Category Archives: maintenance

Cisco DCNM server unresponsive

Cisco MDS

You’re running a DCNM virtual appliance and when you’re trying to log on, it looks like the appliance’s webserver is no longer active.

Connect to the Linux prompt of the appliance and do the following:

Stop DCNM services:

/usr/local/cisco/dcm/dcnm/bin/stopLANSANServer.sh

Open the psql prompt:

/usr/local/cisco/dcm/db/bin/psql dcmdb

log on with root password (user root is assumed)

Type the command:

VACUUM FULL ANALYZE VERBOSE;

This will produce tons of output, but just let it run and eventually it all ends and you just need to start the DCNM services again.

When done, quit the psql prompt:

dcmdb=> \q

Start DCNM services:

/usr/local/cisco/dcm/dcnm/bin/startLANSANServer.sh

Source: cisco.com

Useful EMC VMAX CLI commands

VMAX

Over the years I collected a number of useful CLI commands to control the DMX / VMAX machines I worked with. Even though nowadays Unisphere for VMAX is a useful tool, nothing really beats the command line!
Read more »

How to enable SSH on a VMware 5.5 ESXi host using the vSphere client

virtual machine

Recently I needed to make some edits on vmdk files in order to get rid of a reference to the change tracking files, as mentioned in Gabrie’s post: Cannot open the disk and could not open change tracking file. An SSH connection to a VMware ESXi host was required. SHH was disabled and I needed to enable it temporarily.

How to enable SSH on an ESXi host using the vSphere client

In vSphere, select the host you want to enable SSH on.

Select the “Configuration” TAB, then “Security Profile”. In the upper right select “Properties.

A new window opens, now select SSH (you may need to scroll down a bit) and in the lower right select “Options”.

Again a new window opens and here you can either “Start” or “Stop” the SSH daemon.

enable SSH

Create and retrieve spcollect files from the command line (VNX)

Hard Drive

Java, it’s a curse. And now you suddenly need to upload the spcollect files to EMC and Java isn’t installed or incompatible and Unisphere won’t start.

Now what?

Make sure you have NAVISECCLI installed and just do it from the CLI!

Read more »

Cisco MDS “fabric merge” and “switch add” prerequisites

Cisco MDS

Prerequisites when adding a new SAN switch to an existing SAN fabric

When using enhanced device aliases, make sure you enable this on the new switch as well:

device-alias mode enhanced
device-alias commit

You can verify if this is enabled on an existing switch by this command:

sh run | grep “device-alias mode”

Read more »

Uptime defined, or what is uptime exactly?

Five nines

You often hear vendors mentioning their system has five 9s of uptime, but what exactly is uptime?

Some define uptime only for their own specific piece of technology. For example, a storage array with five 9s uptime, can only tolerate 5 minutes and 15 seconds of downtime per year, but if your network vendor also has a five 9 uptime specification and your power company and your data center and your internet provider and a whole lot of other components…. do the math!

Read more »

Troubleshooting connectivity issues on a Brocade SAN

Fog

I recently had the “pleasure” to figure out what was wrong with a Brocade based SAN environment. Servers were loosing connectivity on one of the HBAs, but all links were online and further investigation was necessary.

Going through all the error counters on each of the long wave SFPs finally revealed one of the SFPs’ health as marginal (hence it was still online, but very buggy indeed). The webtools GUI showed this particular SFP als orange instead of green. Disabling and re-enabling this SFP didn’t help and I decided to shut this SFP for good. And guess what: all my troubles went away. The trunk this SFP was in went back to a non-redundant, but healthy state and all servers got back to normal operations and got their redundant paths back.

So to summarize the story: look for marginal or even faulted SFPs when vague connectivity issues arise. If links are redundant, shutting the faulty one might help.

How to get started setting up ESRS on the latest OE for Block and MCx codes

It’s just another short post on a single command again. This time I was looking for an easy way to get started on ESRS on the latest OE for Block code or the newer MCx code (33.071 or newer).

First of all you need to set up DNS in your VNX machine. In Unisphere, go to settings and click on “configure DNS”.

Also, if there’s a firewall blocking internet traffic, you need to make sure the storage processors can reach *.emc.com over tcp ports 443 and 8443.

After this you can use the following command on the CLI:

naviseccli -h [SPx ip#] esrsconfig -agentProvision -user [Online Support logon name] –password [Online Support super secret password]

Repeat this for the other SP as well.

Read more »

Tracking the FREAK Attack

Now what? Yet another vulnerability exposed?

On Tuesday, March 3, 2015, researchers announced a new SSL/TLS vulnerability called the FREAK attack. It allows an attacker to intercept HTTPS connections between vulnerable clients and servers and force them to use weakened encryption, which the attacker can break to steal or manipulate sensitive data. This site is dedicated to tracking the impact of the attack and helping users test whether they’re vulnerable.

At the moment it seems that only Firefox is safe to use! And needlessly to say that you should ALWAYS be aware of unsafe content. Try not to click anywhere unless you really trust the website!!!

The FREAK attack was discovered by Karthikeyan Bhargavan at INRIA in Paris and the miTLS team. Further disclosure was coordinated by Matthew Green. This report is maintained by computer scientists at the University of Michigan, including Zakir Durumeric, David Adrian, Ariana Mirian, Michael Bailey, and J. Alex Halderman. The team can be contacted at freakattack@umich.edu.

For additional details about the attack and its implications, see this post by Matt Green, this site by the discoverers, this Washington Post article, and this post by Ed Felten.

Please visit this website to find out more about the vulnerability of your browser or website: https://freakattack.com/

 

Source: https://freakattack.com/

Free Royal TS/X NFR license for EMC Elect, MVP, vExpert, Cisco Champions, CTP

Royal ts

Get your own NFR license for Royal TS/X

I usually don’t like to make too much of a fuzz about free giveaways, but we all need RDP, telnet and SSH clients and this one is worth the extra effort. Try it! So far I like it very much.

You can apply for an activation code on .

vExpertsMVPsCisco ChampionsEMC Elect and CTPs can request a free Royal TS/X NFR license! You need to prove in some way that you are one of those community experts, so they can verify and you’ll get an email with your own key!

If you have questions, you can reach out to @royaltsapp on Twitter.

Cisco releases update for its SAN switches to fix the Shellshock aka Bashbug

Cisco MDS

Although I was triggered by a daily update I received from EMC in this knowledge base article: https://support.emc.com/kb/194669, it was Cisco who finally published an update for various firmwares that did not have a fix yet for the shellshock aka bashbug.
You can find all Release Notes on the Cisco site at http://www.cisco.com/c/en/us/support/storage-networking/mds-9000-nx-os-san-os-software/products-release-notes-list.html.
The EMC version of the Release Notes can be found here:

If you haven’t upgraded yet, I’d plan to do so in the very near future!

How to start copy to hotspare manually

Hard Drive

I recently had to manually invoke a hot spare in a VNX 5200, but in Unisphere the option was greyed out.

Unisphere_No-CopyToHotSpare

On the CLI the command wasn’t supported. Now what?

CopyToHotSpare_fail

According to https://support.emc.com/kb/184890 the proper command is now

naviseccli -h [ip of one SP] copytodisk [source-disk] [hot spare]

CopyToDisk_Success

Using the “getdisk” command will show you the actual rebuild has started.

Bare in mind that the way to address disks is in the format “Bus_Enclosure_Disk”, so for example 1_2_3 means disk 3 (the 4th disk) in enclosure 2 on bus 1.

In Unisphere you can actually see the progress of the rebuild:

Disk Rebuild in Unisphere

How to change the VNX weekly heartbeat date and time

Changing the time for the weekly heartbeat

People with Clariion or VNX systems installed on site know that these arrays will email “home” (that’s EMC/you) once a week on a seemingly random date/time. Ok, once the day of the week and the time are set, each week the “I’m still alive” email will go out at that time. But what if you don’t want to have that email sent out at Thursday at 2:47AM and you want all of your arrays to send out that email on Saturday at noon sharp? You will need to adjust the parameters. I didn’t find a way to change the weekday, so I’m changing the time less than a day before it needs to run. So if I want it to run on Saturday at noon, I could run this script on Friday after noon. It will pick the next available day automatically.

Read more »

Configuring timezone and NTP on a Cisco MDS switch

Timezone and NTP configuration of a Cisco MDS switch

Using a GUI for configuring purposes isn’t always the best thing to do, although it most certainly provides a level of overview that cannot be obtained on the CLI.

I’ve found out the Device Manager default settings for NTP for example are that IPv4 addresses used to point to an NTP entity is set to “peer” and if you don’t use peer synchronization, but have a dedicated NTP server instead, it’s better to place a check mark on the “server” item. Also if you have several time sources, you can set a preferred one by placing a check mark at the particular entity.

Read more »

XP-patches on the black market?

Windows XP patches on the black market?

Despite what several reports say, Windows XP is still an operating system that is widely used, all over the world. And now that Microsoft has stopped its official (and free) support of this succesful OS, a lot of people find themselves in need of a scarce good: XP patches. So what happens when you need a scarce good: a black market!

An official date for the first black market is already known: May 13, 2014, since that would be the first day the formerly regular patch distribution will be no longer be initiated for Windows XP.

Read more »

Are my EMC products affected by the bleeding heart SSL bug?

Bleeding heart

It’s been all over the news this week:

Bleeding heart

Heartbleed OpenSSL bug

OpenSSL versions 1.0.1 through 1.0.1f  as well as 1.0.2-beta1 are indicated to be vulnerable to Heartbeat Vulnerability.

Due to a missing bounds check in OpenSSL during the TLS heartbeat extension, a maximum of 64 KiB of memory can be revealed to a connected client or server. This may potentially allow an unauthenticated, remote attacker to gain access to sensitive information such as private keys, login passwords, and encryption keys (the so-called Secret Keys). As a result of this disclosure of potentially sensitive information, these Secret Keys could be leveraged to decrypt other sensitive information or conduct so-called man-in-the-middle attacks.

References:

I won’t copy/paste the complete list in this post as the list will be updated over time, but in general I can disclose that (according to EMC) Brocade FOS, Centera, Clariion, Connectrix Manager, Control Center, Data Domain OS, ESRS, Isilon OneFS, , Networker, RecoverPoint, Replication Manager, ViPR, VNVe, VNX1, VNX2, VPLEX, XtremIO are not vulnerable.

You should read the article on bit.ly/1hwgFpW for specific other products as there are a few that might need attention.

Make sure you patch your products if you need to and please change your passwords every now and then (and in this case as soon as possible).

Fixing misalignment without moving data

Fixing misaligned partitions

It’s quite an old topic, but I recently found a great tool which allows you to re-align partitions without the need to manually move data from the misaligned disk to a new aligned disk.

First of all I must add that all the credits go to Nicholas Weaver, since he wrote the original blog and the actual tool!

Read more »

Which Cisco NX-OS should I use?

This time a really short reminder-like post. Somehow I often end up trying to locate certain release notes of various equipment, but Cisco organized theirs for the SAN switches in an orderly manner: Release Notes

Every NX-OS version is listed here and in each all supported equipment is named. I found this to be very helpful.

How to set the “The array is alive” on a specific day and time on a VNX

The need for weekly messages

EMC’s Symmetrix already knew this feature for a decade or so (or even longer), but since a few years EMC’s pushing customers to make every array to email home once a week so they can keep track of its pulse. And they’re not joking about its importance either, since once an array skips a beat, a severity 1 ticket is being created to get that fixed as soon as possible. EMC truly seems to care about the arrays they have running all over the world, so they’re indeed in good shape and being monitored actively.

Read more »

Other useful features of USM

Techs focussed on EMC will know “EMC USM” for its use in upgrading storage arrays like Clariion and VNX as well as disk firmwares and installing enablers, but it can be used for other purposes as well.

Gearoid Griffin, a fellow EMC Elect 2013 member, wrote a nice article about it. Go have a look, I find it very interesting!

You can find his article on https://community.emc.com/people/GearoidG/blog/2013/04/04/other-useful-features-of-usm.

%d bloggers like this: