how to upgrade to NSX 6.1.3

OK, I’ve had NSX 6.1.2 deployed in my lab, simulating a complex enterprise environment and the DLR (Distributed Logical Routers) kept on causing numerous issues (one of which I’ve blogged about in a previous article here). Anyways, VMware support has requested that I upgrade to the latest NSX version 6.1.3 which is supposed to have fixed most of the issues I’ve been encountering so I’ve decided to upgrade my environment and thought I’d write a blog about it too, especially since there aren’t very many precise NSX related documentation out there.

Note that despite the title of this article (upgrading 6.1.2 to 6.1.3), these steps are also applicable to upgrading from any NSX 6.0.x version to 6.1.x version.

High level steps involved

First thing to do is to check for compatibility and I have vSphere 5.5 which is fully supported with NSX 6.1.3 as per the compatibility Matrix here.
Then have a look at the 6.1.3 release notes to make sure you are up to date with what’s fixed (not all the fixes are listed here btw), what are the known issues…etc. That is available right here.
Next thing is to download the upgrade media from the NSX / Nicira portal. If you have an active NSX subscription, you may download it from My VMware portal or else, (if you have taken the course and were deemed to be suitable to have access to the eval, you can download it from Nicira portal here. Once you’ve downloaded the Nicira upgrade media (VMware-NSX-Manager-upgrade-bundle-6.1.3-2591148.tar.gz file), you may also download the install & upgrade guide here.(Note that there don’t appear to be a separate set of documentation for 6.1.3 as it appears to be for the whole 6.1.x platform).
If you are upgrading vSphere to v6.0 also at the same time, follow the steps below
1. Upgrade NSX Manager and all NSX components to 6.1.3 in the following order.
  1. NSX Manager
  2. NSX controller(s)
  3. Clusters and Logical Switches
  4. NSX Edge
  5. Guest Introspection and Data Security
2. Upgrade vCenter Server to 6.0.
3. Upgrade ESXi to 6.0. – When the hosts boot up, NSX Manager pushes the ESXi 6.0 VIBs to the hosts. When the hosts show reboot required on the Hosts and Clusters tab in the left hand side of the vSphere Web Client, reboot the hosts a second time. NSX VIBs for ESXi 6.0 are now enabled.
If you are not upgrading vSphere at the same time (this is the case for me), follow the step below
1. Upgrade NSX Manager and all NSX components to 6.1.3 in the following order.
  1. NSX Manager
  2. NSX controller(s)
  3. Clusters and Logical Switches
  4. NSX Edge
  5. Guest Introspection and Data Security

Now lets look at the actual upgrade process of NSX

NSX Upgrade steps

NSX Manager Upgrade
1. I don’t have Data security deployed. If you did, uninstall it first
2. Snapshot the NSX Manager virtual appliance (as a precaution)
3. Login to the NSX manager interface and go to the upgrade tab. Click on the upgrade button on top right and select the previously downloaded VMware-NSX-Manager-upgrade-bundle-6.1.3-2591148.tar.gz file & continue.
4. Select to enable SSH and click upgrade.
5. Upgrade of the NSX Manager appliance will now start.
6. Once the upgrade completes, the appliance will automatically reboot (this is not so obvious unless you look at the console of the appliance). Once rebooted, login to the NSX manager instance and confirm the version details (Top right).
NSX controller(s) Upgrade
1. Notes
  1. Upgrades are done at cluster level and would appear with an upgrade link in NSX manager.
  2. It’s recommended that no new VM’s are provisioned, no VMs are moved (manually or via DRS) during the controller upgrade
2. Backup controller data by taking a snapshot
  1. If the existing NSX version is < 6.1, This has to be done using a REST API call
  2. If the existing NSX version is => 6.1, you can do this using the GUI of the web client as follows
3. Login to the Web client, and go to the Networking & Security -> Installation and click on upgrade available link as shown below
4. You’ll now see the upgrade being in progress (may ask you to reload the web client). Controller(s) will reboot during the upgrade
5. The upgrade should complete within 30 mins normally. If not completed by then, upgrade may need to be launched using the upgrade available link again. Once successful, the upgrade status should state upgraded and the software version should be updated with the new build number. Note that if the upgrade status says failed, be sure to refresh the web client prior to assuming that its actually failed. It may state its failed due to the 30 standard timeout even though the controller upgrade has completed successfully (happened to me couple of times)
Clusters and Logical Switches
1. Go to the host preparation tab and click update
2. Installation and upgrade guide states that each host will automatically be put in to the maintenance mode & rebooted. If the upgrade failed with a Not ready status for each node, you may need to click resolve each time to retry (happened o me once) and it will proceed. Some host may require manual evacuation of VM’s too. Also worth noting that if you keep getting the task “DRS recommends hosts to evacuate” logged in vSphere client, you may want to temporarily disable HA so that the ESXi host can be put in maintenance mode by the NSX cluster installer (Arguably, in a well managed cluster, this should not happen).
3. When the cluster is updated, the Installation Status column displays the software version that you have updated to.
NSX Edge (Including the Distributed Logic Router instances)
1. Go to the NSX Edges section and for each instance, select the Upgrade Versions option from the Actions menu.
2. once the upgrade is complete, it will show the new version of the Edge device, weather that’s a DLR or an Edge gateway device.
Guest Introspection and Data Security
1. Upgrading these components are relatively straight forward and I’d encourage you to look at the install & upgrade guide…
Now, create a NSX backup (You may already have routine backup scheduled on the NSX manager)
If all is well, remove the snapshot created earlier, on the NSX manager appliance.

All in all, it was a fairly straight forward upgrade but the time it took due to somewhat finicky nature of some of the components (especially the cluster upgrade where a number of manual retries were required) unnecessarily added time. Other than that, all appears to be well in my NSX deployment.

Cheers

Chan