Category Archives: Configuration

Cisco DCNM server unresponsive

Cisco MDS

You’re running a DCNM virtual appliance and when you’re trying to log on, it looks like the appliance’s webserver is no longer active.

Connect to the Linux prompt of the appliance and do the following:

Stop DCNM services:

/usr/local/cisco/dcm/dcnm/bin/stopLANSANServer.sh

Open the psql prompt:

/usr/local/cisco/dcm/db/bin/psql dcmdb

log on with root password (user root is assumed)

Type the command:

VACUUM FULL ANALYZE VERBOSE;

This will produce tons of output, but just let it run and eventually it all ends and you just need to start the DCNM services again.

When done, quit the psql prompt:

dcmdb=> \q

Start DCNM services:

/usr/local/cisco/dcm/dcnm/bin/startLANSANServer.sh

Source: cisco.com

Brocade SAN Switch commands that prove to be very useful

Brocade commands are usually very easy to find out. Simply type “help” and look for the command you think you need, but what exactly does each command do? Here’s a list:

Info

uptime                          – Same as unix uptime
date                              – Same as unix date
version                          – Gives versions of firmwares & OS

Hardware State

faultshow                     – Show switch faults
fanshow                       – Show switch FAN faults
psshow                        – Show switch POWER SUPPLY faults
tempshow                    – Show switch TEMPERATURE values
switchstatusshow          – Overall status of switch

Config

agtcfgshow                  – Show SNMP config
configshow                  – Show switch config
gbicshow                     – Show GBIC slots and serial numbers
licenseshow                 – Show license data
supportshow                – Like Sun’s explorer – gets many configs at once
switchshow                  – Show switch ports and connections
zoneshow                    – Show zone and switch aliases

IP

bcastshow                   – Show broadcast routing
ifmodeshow                 – show interface mode (duplex)
ifshow                         – Like unix netstat -s
ipaddrshow                  – Like unix netstat -i
interfaceshow               – Like unix ndd

Performance

ifshow                          – Like unix netstat -s
portperfshow                – Show interface mode (duplex)
portshow #                  – Show stats on a port
portrouteshow #           – Show routes on a port
portstatsshow #            – Show stats (netstat -s) on a port

Misc Show

diagshow                     – Show diagnostics – paged output
errdump                      – Show diagnostics – no paged output
fabricshow                   – Show fabric
fspfshow                      – Show FSPF protocol info
mqshow                       – Show queues
nbrstateshow                – Show FSPF neighbor states
nsshow                        – Show name servers
nsallshow                     – Show all name servers
porterrshow                  – Like mpstat – shows port info
switchstatuspolicyshow  – Show config at when errors are flagged

 

Brocade Data Collection

There are 2 types of Data Collection associated with Brocade switches, “supportshow” and “supportsave”.

If the problem is related to basic “port issues”, a “supportshow” is sufficient. However, a more complex problem that involves Zoning, Performance, Reboots, CP Failovers, Fabric wide problems etc, requires a “SupportSave” type of collection. If the problem is multiswitch related (ISL connections, long distance configurations etc.) we need the supportsave from all relevant switches in the fabric.

The amount and type of information displayed in a supportshow output is depending on the type of switch as well as the collection options enabled in the switch configuration. This can be configured with the “supportshowcfgenable [option]” and “supportshowcfgdisable [option]”.

The output from the “supportshowcfgshow” command shows which subscripts will be enabled. The default groups are always enabled. Only on special occasions you will be asked to enable additional groups if necessary.

 

Performance and Intermittent Error Related Issues

If there is no sign of any obvious physical issue there might be link related problems which can identify performance problems and/or protocol related errors. Brocade counters are cumulative and keep doing so until a certain counter wraps, a switch reboots or the statistics are manually cleared.

Storage vendors for example require in these circumstances that a new baseline is created, a certain run-time has been achieved and separate commands are submitted against the suspected switch or switches.

To create a new baseline with cleared counters do the following:

  1. Log in to the switch via Telnet or SSH
  2. Submit the “statsclear” command
  3. Submit the “slotstatsclear” command

After the agreed (mostly around one hour) run-time capture a new supportsave and upload this to the vendor’s service request.

SupportSave Data Collection (CLI Method)

When the problem is more sophisticated a supportsave from the switch is required. The supportsave command is available as of Fabric OS version 4.4 however, Fabric OS versions (> 6.2.x) provide a significant better collection of logs which represent the status overview of the switch and fabric. If you have a director class switch with two CP’s and/or core plus function blades it will also collect information from all the blades.

The supportsave will upload between 25 and 80 files depending on platform, Fabric OS level and enabled features to an FTP or SCP server. These will not be tarred or zipped into one file so it is important you create such an archive with a meaningful name. (ie. switchname-domainid-fabricid.zip)

Example

Fabosv4.4switch:admin> supportsave -u anonymous -p password -h xxx.xxx.xxx.xxx -d /directory -l ftp

This command collects RASLOG, TRACE, supportShow, core file, FFDC data and then transfer them to a FTP/SCP server or a USB device. You could also simply type the “supportsave” command without any parameters and you will be asked for the extra information before the command is actually executed. The operation will usually take several minutes to complete.
NOTE: supportSave will transfer existing trace dump file first, then automatically generate and transfer latest one. There will be two trace dump files transferred after this command.

OK to proceed? (yes, y, no, n): [no] y
Saving support information for switch:BR4100_IP127, module:RAS…
Saving support information for switch:BR4100_IP127, module:CTRACE_OLD…
Saving support information for switch:BR4100_IP127, module:CTRACE_NEW…
etc……

To upload the files you can specify the FTP parameters inline (as modeled above) or through the supportftp command (see Fabric OS V5.1 command Reference guide) .

  1. Host IP: XXX.XXX.X.X (example 192.168.1.1)
  2. User Name: admin
  3. Password:
  4. Remote Directory: tmp (example: tmp)
  5. Saving support information

SupportShow Data Collection

This is a non disruptive procedure and can be performed by the CE or the customer.

You can use your favorite terminal emulation utility. Refer to the respective documentation how to turn on capturing output to a file.

Telnet or SSH

  1. Telnet or SSH into Brocade switch
  2. Enter username and password
  3. Start logging to file on the Telnet session.
    NOTE: For Windows standard telnet, this is under the terminal pulldown menu.
  4. Enter command: supportShow
  5. Upload the telnet log to TUF.

What is the Principal switch in a Brocade SAN

Principal switches maintain unique domain ID across the fabric. Principal switch ensures that each switch in a SAN have different domain ID. Any ISL of a switch that takes to the Principal switch is a upstream. Any ISL of a switch that goes away from principle switch is a downstream. All non principal switches are called subordinate switch. Zoning updated in a principal switch or a non principal switch will update zoning across the fabric. “date” command will be a read-only if a switch is configured with a Time server – NTP. Principal switch will update the time in all the non principal switches. tsclockserver is the command used to associate a switch to a NTP time server.

To manually set a subordinate switch to a principal switch use “fabricprincipal” command.

To elect a new principal switch:

>fabricprincipal -1         # This command will see the appropriate principal switch and make it as a new principal switch. Also, this will elect new upstream and downstream ISLs

To force a switch to be a principal switch:

> fabricprincipal -f         # This command will force the switch to be the principal switch of the fabric. This will also elect new upstream and downstream ISLs

To see the current settings:

> fabricprincipal -q      # This command will query the current settings and displays

How to check the Principle switch:

  • Fabicshow shows the Princple switchSwitch ID   Worldwide Name           Enet IP Addr    FC IP Addr      Name
  • ————————————————————————-
  • 121: fffc79 10:00:00:05:1e:36:0b:42 10.2.59.52      0.0.0.0        >”IBM_2109_M48_21″
  • 130: fffc82 10:00:00:05:33:cd:a4:c2 10.2.59.8       0.0.0.0         “DS_6505B”
  • The Fabric has 2 switches
  1. Switchshow shows the principle

This Fabric has 2 switches

IBM_2109_M48_21:admin> switchshow

switchName:     IBM_2109_M48_21
switchType:     42.2
switchState:    Online
switchMode:     Native
switchRole:     Principal
switchDomain:   121
switchId:       fffc79
switchWwn:      10:00:00:05:1e:36:0b:42
zoning:         ON (Cfg_080123)
switchBeacon:   ON
blade1 Beacon:  OFF
blade2 Beacon:  OFF
blade3 Beacon:  OFF
blade4 Beacon:  OFF
blade7 Beacon:  OFF

DS_6505B:admin> switchshow
switchName:     DS_6505B
switchType:     118.1
switchState:    Online
switchMode:     Native
switchRole:     Subordinate
switchDomain:   130
switchId:       fffc82
switchWwn:      10:00:00:05:33:cd:a4:c2
zoning:         ON (Cfg_080123)
switchBeacon:   OFF
FC Router:      OFF
FC Router BB Fabric ID: 1
Address Mode:   0

How to Replace the Switch:

  • Load the Licenses to the Switch as of the other switches
  • Assign the Network IP address to the switch and do not connect to the SAN Fabric
  • Save the configuration by issuing the command configupload
  • Upgrade the code to the same Firmware version
  • Update the credentials to the standard credential as per your org standards
  • Disable your switch with ‘switchdisable’
  • Change the domain id of the switch next available
  • Change the principleswitch role to subordinate  by issuing the command principle –f 0
  • Connect the Switch to the SAN  and this will download the config from the core switch or principle switch

How to create a snapview snapshot on an existing LUN

I apologize in advance for this (6 years or so too late) post, since it’s for creating a snapview snapshot on a LUN on a VNX. It’s simply meant as a reminder for the command line syntax:

Examples for creating snapview snapshots (it only defines it, no COFW is happening at this point):
naviseccli -h 172.16.20.96 snapview -createsnapshot 17 -snapshotname VMFS-001-SNAP
naviseccli -h 172.16.20.96 snapview -createsnapshot 18 -snapshotname VMFS-003-SNAP
naviseccli -h 172.16.20.166 snapview -createsnapshot 27 -snapshotname VMFS-002-SNAP
naviseccli -h 172.16.20.116 snapview -createsnapshot 5 -snapshotname VMFS-004-SNAP

To start an actual point in time session (and the start of COFWs):
naviseccli -h [ip address] snapview -startsession [session name] -snapshotname VMFS-001-SNAP

To stop a session:
cnaviseccli -h [ip address] snapview -stopsession [session name]

To activate a snapview session (make the data visible):
naviseccli -h [ip address] snapview -activatesnapshot [session name] -snapshotname VMFS-001-SNAP

To deactivate a snapview session (stop presenting the data to the hosts):
naviseccli -h [ip address] snapview -deactivatesnapshot [session name] -snapshotname VMFS-001-SNAP

Using putty instead of the default ssh from Cisco device manager

DCNM

How do I configure Cisco DCNM so Putty starts when I select to go to the command line of a switch?

right click on switch

If you right click on a switch in the overview section in DCNM, you can go to the command line of that switch, but how do you change the default CLI SSH into Putty (or another telnet / SSH capable tool)?

Read more »

How to add new members to an existing Cisco smart zone

Cisco MDS

DCNM is down. I love the tool, but the downside is that you forget how to use the CLI. I faced downtime of our DCNM appliance and was forced to use the CLI instead. No big deal actually, but I still want to post the commands to use when you need to add new hosts to existing zones.

First you might want to create a new device alias for the new hosts:

device-alias database
device-alias name server1 pwwn 20:11:00:15:b9:00:00:00
device-alias name server2 pwwn 20:11:00:15:b9:00:00:01
device-alias commit

And then you want to add the new aliases to the existing (smart) zone:

config
zone name NameOfZone vsan 123
member device-alias server1 init
member device-alias server2 init
zone commit vsan 123

You don’t need to commit the whole zoneset again, since that one didn’t change. Check it by running:

zoneset activate name ZoneSetName vsan 123

That’s it! there’s not much to it, and can save you a lot of time as well!

Useful EMC VMAX CLI commands

VMAX

Over the years I collected a number of useful CLI commands to control the DMX / VMAX machines I worked with. Even though nowadays Unisphere for VMAX is a useful tool, nothing really beats the command line!
Read more »

How to list Host LUN ids in VMAX Masking Views

VMAX

Creating tdevs and masking them to hosts (storage groups in a masking view) is relatively easy in the Unisphere for VMAX interface, but what if you add a few tdevs of exactly the same size and you want to make sure that the VMware administrator uses the right LUN for each VMFS he’s going to create? One way to make sure he knows which LUN corresponds with what tdev is the Host LUN id. To list the host LUN ids

symaccess -sid 1234 show view mv_some-maskingview-name

or (a bit more verbose)

symaccess -sid 1234 list view -name mv_some-maskingview-name -detail

The second command shows each initiator group nested within other initiator groups as well.

Both generate a table with the following headers:

Sym                                                            Host
Dev     Dir:Port    Physical    Device    Name    Lun    Attr    Cap(MB)

The column under “Host Lun” shows the Host LUN ids.

How to enable SSH on a VMware 5.5 ESXi host using the vSphere client

virtual machine

Recently I needed to make some edits on vmdk files in order to get rid of a reference to the change tracking files, as mentioned in Gabrie’s post: Cannot open the disk and could not open change tracking file. An SSH connection to a VMware ESXi host was required. SHH was disabled and I needed to enable it temporarily.

How to enable SSH on an ESXi host using the vSphere client

In vSphere, select the host you want to enable SSH on.

Select the “Configuration” TAB, then “Security Profile”. In the upper right select “Properties.

A new window opens, now select SSH (you may need to scroll down a bit) and in the lower right select “Options”.

Again a new window opens and here you can either “Start” or “Stop” the SSH daemon.

enable SSH

Free EMC trial software / virtual appliances

virtual machine

A quick heads-up this time about building your own lab environment

Sometimes you just want to run a VNX, Avamar, PowerPath, Data Domain or Isilon as a virtual machine to see how things work, or to write work instructions. And EMC offers a lot of these virtual appliances for free!

Take a look at these:

 

Cisco MDS “fabric merge” and “switch add” prerequisites

Cisco MDS

Prerequisites when adding a new SAN switch to an existing SAN fabric

When using enhanced device aliases, make sure you enable this on the new switch as well:

device-alias mode enhanced
device-alias commit

You can verify if this is enabled on an existing switch by this command:

sh run | grep “device-alias mode”

Read more »

Cisco Smart zoning – part II: examples

Smart zoning examples

In my smart zoning post from last February I already presented the way to get started with Cisco smart zoning. I initially planned to give a more detailed calculation on how much time you can save if you were using smart zoning compared to SIST zoning.

SAN fabric

I was talking to an EMC SAN instructor (Richard Butler) this week and after I did a little white boarding and used my hands to picture how massive a traditional SIST zone environment would be, we agreed smart zoning is the way to go.

Read more »

How to set the NTP server, time and timezone in a Brocade switch

NTP server

Previously I wrote about setting the NTP, time and timezone settings in a Cisco switch and now it’s time for the same in a Brocade switch.

It’s in fact not that hard to do. Log in to the CLI and use the following commands:

tsclockserver 1.2.3.4

or

tsclockserver ntp.domain.ext (make sure the DNS is set up properly first)

This will set the NTP server address in this switch to ip address 1.2.3.4. Set this only on the principal switch, as this switch will propagate the time to the other switches in the fabric.

To set the timezone use the following command:

tstimezone –interactive

This will ask for the region and country the switch is located in.

Choose 8 for Europe and 34 for the Netherlands and after verifying the setting, choose 1 (yes) to set the TZ.

Use the “date” command to verify the current time and date and TZ region:

Wed May 13 01:08:32 CEST 2015

This makes life a lot easier when troubleshooting!

Adding or replacing a Cisco SAN switch in an IVR topology

Cisco MDS

If you have multiple datacenters or a multi tenant fibre channel environment and you’re using Cisco FC switches, it’s a best practice to use VSANs to separate the configurations of each location / tenant. To allow storage arrays and / or hosts in different VSANs to communicate with each other Inter VSAN Routing needs to be used.

If you need to have 2 EMC VNX storage arrays “talk” to each other for MirrorView for example over 2 or more datacenters (for data replication purposes that is) or hosts in one DC talk to storage in another DC, using transit VSANs (and therefore IVR) will keep your VSANs with equipment indoors and the slightly more vulnerable VSAN outdoors. If some farmer with his tractor rips your single mode fiber, only the outdoor VSAN will be fractured and the indoor VSANs remain unharmed. And of course communication between the remote sites is interrupted, but the indoor VSANs / fabrics remain unchanged.

Read more »

Setting Daylight Savings Time on a Cisco MDS switch

Cisco MDS

I recently ran into a log time difference at a customer’s site so I want to repeat my blog from last year (Configuring the timezone and NTP). It’s slightly different on Cisco LAN / Routing equipment, but on Cisco MDS SAN switches the DST configuration is set by using the following commands:

config t
clock summer-time CEST 5 Sun Mar 02:00 5 Sun Oct 03:00 60
exit
copy run start

  • CEST means Central Europe Summer Time (you can use your timezone name instead if you like)
  • the first 5, followed by “Sun” means that the DST will become active on the LAST (5th, sometimes the 4th) Sunday of the month following the day you just named (March)
  • 02:00 is the time of day when the DST will become active (time will go forward)
  • the second 5, followed by “Sun” means that the DST will become active on the LAST (5th, sometimes the 4th) Sunday of the month following the day you just named (October)
  • 03:00 is the time of day when the DST will end (time will go back to normal, backwards, so from 03:00 in my example the clock will go back to 02:00)

You can check the date, time and timezone by typing “show clock”.

Cisco MDS NX-OS zoning on the CLI

Cisco MDS

The Cisco Fabric Manager or the newer DCNM can be great for creating a limited number of zones and aliases, but when the number of zones exceeds 10 or 20 or so, creating, cloning and editing these can be a pain in the B@TT.

FC-SWITCH-01# conf t
Enter configuration commands, one per line. End with CNTL/Z.

FC-SWITCH-01(config)#

Create aliases:

FC-SWITCH-01(config)# fcalias name VM101-HBA1 vsan 45
FC-SWITCH-01(config)# member pwwn 20:21:22:25:B6:00:00:01

Enhanced zone session has been created. Please ‘commit’ the changes when done.

FC-SWITCH-01(config-fcalias)#

FC-SWITCH-01(config-fcalias)# fcalias name VM101-HBA1 vsan 45
FC-SWITCH-01(config-fcalias)# member pwwn 20:21:22:25:B6:00:00:02
FC-SWITCH-01(config-fcalias)# fcalias name VM102-HBA1 vsan 45
FC-SWITCH-01(config-fcalias)# member pwwn 20:21:22:25:B6:00:00:03

FC-SWITCH-01(config-fcalias)# device-alias commit

Create two new zones:

FC-SWITCH-01(config-zone)# zone name VM101-HBA1_VNX-5600-08-SPA6 vsan 45
FC-SWITCH-01(config-zone)# member fcalias VNX5600-08-SPA6
FC-SWITCH-01(config-zone)# member fcalias VM101-HBA1
FC-SWITCH-01(config-zone)# zone name VM102-HBA1_VNX-5600-08-SPB7 vsan 45
FC-SWITCH-01(config-zone)# member fcalias VNX5600-08-SPB7
FC-SWITCH-01(config-zone)# member fcalias VM102-HBA1
FC-SWITCH-01(config-zone)# zone commit vsan 45

Commit operation initiated. Check zone status

Now add the two new zones to a zoneset:

FC-SWITCH-01(config)# zoneset name ZS_VSAN170 vsan 45
FC-SWITCH-01(config-zoneset)# member VM101-HBA1_VNX-5600-08-SPA6
FC-SWITCH-01(config-zoneset)# member VM102-HBA1_VNX-5600-08-SPB7
FC-SWITCH-01(config-zoneset)# zone commit vsan 45

FC-SWITCH-01(config)#

Maximum distances using fiber cable types OM1/OM2/OM3/OM4/OS1

What are the maximum distances using different fiber grades?

 

fiber cable specs graph

The graphs show the direction where OM-specs are going compared to distances.

Read more »

Cisco MDS: Fabric is already locked

Cisco MDS

When you encounter a fabric lock, because you accidentally left the GUI or CLI without committing the changes, you can try the following to clear the lock and retry to apply your changes:

  1. run ‘show cfs lock’ to see who lock`s the fabric
  2. run ‘clear device-alias session’ to clear the lock when you were doing zoning activities

Instead of the “clear device-alias session” in line 2, another common possibility to get the lock cleared is:

  • ‘clear ivr session’ (when you were in the middle of IVR activities)

Other locks can occur, but the device-alias and ivr are probably the most common. At least the ones that I encountered so far.

Cisco zoning: some commonly used – show – commands

Cisco MDS

When you need to look up the current zoning config of a Cisco SAN-switch / VSAN, there’s a number of commands that will help you. Because a “show run” doesn’t always do the trick well, especially if you have a large config. And using the “sh run | i “some text you’re looking for” doesn’t always help as well if you don’t know the exact phrase you’re looking for.

I hope this table helps:

show Command
Description
show zone Displays zone information for all VSANs.
show zone vsan 100 Displays zone information for VSAN 100.
show zoneset vsan 100 Displays information for the zone set in VSAN 100.
show zoneset vsan 2-5 Displays configured zone set information for a range of VSANs (2, 3, 4 and 5 in this case).
show zone name AZone Displays members of zone “AZONE”.
show fcalias vsan 100 Displays fcalias configuration in VSAN 100.
show zone member pwwn 20:00:00:25:b1:34:aa:c2 Displays membership status of a port wwn. Very good if you’re concerned that 1 HBA is used in more than 1 zone!
show zone statistics Displays zone statistics.
show zone statistics read-only-zoning Displays read-only zoning statistics.
show zoneset active Displays the active zone sets.
show zoneset brief Displays brief descriptions of zone sets.
show zone active Displays the active zones.
show zone status Displays zone status.
show zone Displays zone statistics.
show running Displays the interface-based zones.

How to get started setting up ESRS on the latest OE for Block and MCx codes

It’s just another short post on a single command again. This time I was looking for an easy way to get started on ESRS on the latest OE for Block code or the newer MCx code (33.071 or newer).

First of all you need to set up DNS in your VNX machine. In Unisphere, go to settings and click on “configure DNS”.

Also, if there’s a firewall blocking internet traffic, you need to make sure the storage processors can reach *.emc.com over tcp ports 443 and 8443.

After this you can use the following command on the CLI:

naviseccli -h [SPx ip#] esrsconfig -agentProvision -user [Online Support logon name] –password [Online Support super secret password]

Repeat this for the other SP as well.

Read more »

Cisco Smart Zoning: is it really worth the effort? YES!

It’s been available since NX-OS 5.2(6), but is the community ready for a change? It seems the majority of SAN managers are still afraid to start using it, but why? the concept is so simple: it looks like the infamous “default zone” has made it’s way back to the storage area network, but with a twist.

Default zoning

When I was introduced to the world of Fibre Channel, over ten years ago, I remember going through the automatic steps of setting up a freshly powered-on SAN-switch and disabling the default zone, because that one makes that all initiators and targets can communicate with each other. And two initiators talking to each other is not done, because on the midrange arrays an initiator (used for data replication) can also be a target and you don’t want ports logging into each other.

Read more »

%d bloggers like this: