Making cisco MDS switches log to an external (syslog) server

Cisco MDS

Configuring a Cisco SAN switch to have it send logging to an external syslog server by using the GUI is quite easy to do:

Floow the steps as I walk through DM: click “logs”, then “syslog”, then “setup”

If any syslog server is already configured, you’ll find it here:

You can either delete an existing or create a new entry, but you cannot have more than three entries in total!

If you prefer to have an entry using IPv4 or IPv6, choose IPv4 or IPv6, otherwise use DNS and simply type in its name.

I’ve tried this method to change an existing entry and somehow it wouldn’t stick. Deleting three entries, clicking refresh and 2 came right back…. I failed back to the command line.

The CLI is actually easier, but with less overview of what you’re doing. If you need to list the existing syslog servers, type “show logging”. In the extensive sum-up that follows are the servers you’ve configured so far. If a servers needs to be adjusted, don’t bother to first delete it, because a new entry will overwrite the existing line. But if you need to actually delete one, type “no logging server” followed by its name or IP.

A new entry is made by typing

logging server name-of-the-syslog-server.domainname.extension [severity] port 6514 facility syslog

if you want to use the IP of the server, don’t type its name, but the IP, the syntax is the same. Severity is for example “6” so any message of severity “notice(6)” and lower (more important) will be sent. I’ve put port 6514 here as an example for secure syslog, but any other port will do just fine as well.

If you want thee syslog server entries, repeat the “logging server” line three times, one for each syslog target.

Oh, don’t forget to ask the firewall admin to open the port that you will be using 😉

Don’t forget to save the new config. That’s it!

Stretching and unstretching ActiveCluster PODs on Pure arrays

Scenario

Suppose you have two (or more) datacenters and you’re running a true active / active setup, meaning that clusters are spread over both sites and each host has access to both the local volume as well as the identical writable copy on the second site. You’ve accomplished this by setting up ActiveCluster and some PODs with volumes in them and everything is working fine and the PODs are in sync, so volumes can be written to on both locations.

The OTA team is testing a new application on one of the sites without the cluster being spread over both sites. When the testing is done it’s time to go to production and move the application to a production cluster, so formerly local volumes need to be added to a POD, because they need to be written to on the second location as well (or simply need to be replicated to the DR site to have the data on two locations).

Moving one or more volumes to a POD – steps

In order to move one or more volumes into PODS, you can either create a new unstretched POD or – and this can be tricky – you need to unstretch an existing POD first. If a POD contains volumes that are actually being accessed on both sites, you need the hosts to stop using the volumes on the array that’s going to be deleted from the POD for a short while. When hosts are configured correctly (having access to each volume on two arrays), you don’t need to take any action as the I/O will automatically be redirected to the only surviving Pure array. Of course this only applies to the hosts that are actively acessing volumes in the remote POD that you want to unstretch; hosts on the “surviving” array will continue to access that array and will not be able to access the volumes on the second site for a short while. Stopping I/O is necessary because when unstretching PODs, one side loses all access to the volumes in that particular POD.

If hosts are not configured to access volumes on both sites, you will need to failover the resources to the surviving site. When no I/O is going to the array that doesn’t contain a copy of the OTA volumes, the POD that will contain these formerly OTA volumes will need to be unstretched.

If this POD contains a large number of volumes, there’s something interesting to be seen later on! Pay attention!

When you’re sure all I/O to volumes in the POD on the second array is stopped or multi-site access is configured correctly, you can start unstretching the POD: select the POD, delete the remote array as seen from the volume(s) that you need to add to that POD and as soon as the POD contains only 1 array, you can add the volumes that need to be present on both sites.

Resyncing

When all OTA volumes in our example are added to the unstretched POD, you can add the remote Pure array and the resyncing will start.

Depending on the activity on the volumes in that POD, this can be a short transition or somewhat longer (sometimes even hours if it’s a really active POD), but there’s one interesting thing that I noticed during the numerous resyncing activities that I’ve whitnessed so far: when the resync starts, it looks like it’s taking ages to get somewhere and it might even show a disappointing progress, until the resync reaches 20%.

All of a sudden the resync will look like it’s speeding up and as soon the the counter is over 20% it will reach 100% in a matter of seconds (or minutes). I cannot find any proof that this progress has a different meaning other than what it looks like: 20% meaning only 20% is actually resynced, but my assumption is that 20% means it reached 100% and above 20% means it’s catching up on the last few “dirty” blocks on the sides that are being written to.

The good thing is that as soon as you realize that 20 is the new 100%, so waiting for the resync isn’t half as bad as it looks at first!

Happy resyncing, everybody!

How to test the alerting in a Pure Storage FlashArray

When configuring SMTP or syslog for alerting the easiest way to configure is in the GUI, simply because everything you need to configure is right there in plain sight.

For this example, we will assume the syslog servers and SMTP relay host and sender domain have been specified. If not, these can be set from the GUI under Settings > System > Syslog servers and Settings > System > Alert Routing.

But to test if it all works is a different story: there’s no test button!

So we need to log on to the CLI. Use your favorite SSH client and log on the the array.

Syslog

First you can view if the syslog was configured according to your liking by entering the command:

purelog list

You should now see the configured syslog servers and the ports that are used.

To test if syslogging works enter the command:

purelog test

SMTP

For emails the command to view the settings is:

purealert watcher list

You should now see the email addresses that will be used whenever the array needs to send an email. To test this you need to use the following command:

purealert watcher test [email protected]

If all goes well you will then receive an email similar to:

Hello,

This is a test message from your Pure Storage Array.

Controller Serial: PCTFL1953173B
-Pure Storage Array PUREARRAY-027-ct0

Troubleshooting error-disabled issues between a Cisco UCS fabric interconnect and a Cisco MDS

Cisco MDS

When connecting a high speed cisco UCS Fabric Interconnect (like the 6454) to Cisco MDS Fibre Channel switches, you might encounter error-disabled ports causing the port-channel to go down. Specifically for 8 Gbps speeds you need to enter 1 additional command per port that’s participating in the port-channel:

int fc1/17
switchport fill-pattern IDLE speed 8000

For each fc port on the MDS you need to enter this command, followed by a “shut / no shut” and the error disabled problems are most likely gone.

Check to see the port config:

show run int fc1/17

Or whatever port you configured to use the fill-pattern.

Source: Cisco

How to upgrade a Cisco MDS SAN switch

Cisco MDS

After a few years of not having had to perform switch upgrades, I had to perform a few firmware upgrades on MDS switches. I decided to write down the steps I had to take:

  • First make sure you have a reachable ftp server somewhere and write down its ip address
  • Log on to the Cisco MDS SAN switch you need to upgrade
  • Make sure the bootflash: has enough available space to hold the 2 new files

# dir bootflash:

4096 Feb 23 14:58:04 2020 lost+found/
26049024 Feb 10 12:57:55 2020 m9100-s5ek9-kickstart-mz.8.3.2.bin
121121564 Feb 10 12:56:47 2020 m9100-s5ek9-mz.8.3.2.bin

Usage for bootflash://sup-local
463962112 bytes used
2894815232 bytes free
3358777344 bytes total

Read more »