Recently I needed to upgrade a Cisco Unified Computing System (UCS) system and while Cisco does have documentation about it. The process can be scary and could use more screenshots. Here is my step by step guide on how to upgrade a Cisco UCS via the Firmware Auto Install with the Cisco UCS Manager.
Fully planning your Cisco UCS upgrade is very important. There are a lot of things that need to be checked to make sure the upgrade is a success and that everything is compatible. In the past I’ve needed to upgrade my ESXi to as high as they can go based on the VMware and Cisco Compatibility Matrix and then upgrade the Cisco UCS. Then go back and upgrade ESXi again and then upgrade Cisco UCS again.
Quick Cheat Sheet
- Resolve any faults in UCS Manager
- Plan your Upgrade path
- Backup the configuration
- Run UCS Manager Health and Pre-Upgrade Check Tool
- Download Firmware Software Bundles
- Upload Firmware Software Bundles
- Verify Maintenance Policy
- Verify Fabric Interconnects
- Verify Hardware
- Disable Call Home
- Infrastructure Firmware Upgrade
- Server Firmware Upgrade
Upgrade Path Planning
- Resolve any faults in UCS Manager
- Review the Cisco suggested release for your UCS in the Cisco Software Downloads
- Review UCS Hardware and Software Compatibility Matrix
- Review the Cisco upgrade guide for UCS Manager
- Review the Release Notes for the version of UCS you selected.
- Review the Deprecation Notices in the Release Notes
- Review the Upgrade and Downgrade Guidelines in the Release Notes
- Review the Deprecated Hardware and Software in the Release Notes
- Review the Cross-Version Firmware Support in the Release Notes
- Review the Internal Dependencies in the Release Notes
- Review the Open Caveats in the Release Notes
- Review the Known Behavior and Limitations in the Release Notes
- Plan your upgrade path and double check your upgrade patch with Cisco Technical Assistance Center (TAC).
Backup
- Click on Admin > Backup Configuration
- Click on Create Backup Operation
We need to take an All Configuration backup and a Full State backup.
For the All Configuration backup set the following
- Select All Configuration
- Select Preserve Identities
- Select Local File System
- Give it a name
It should look something like this.
For the Full State Backup set the following
- Select Full Sate
- Select Local File System
- Give it a name
It should look something like this.
UCS Manager Health and Pre-Upgrade Check Tool
Before we perform an upgrade we should use the Cisco UCS Manager Health and Pre-Upgrade Check Tool as it helps confirm the upgrade path is valid and that everything is healthy.
- Install Python on the system that you want to run the tool on.
- We will need an export of a backup file from Cisco UCS Manager.
- Download and install the UCS Manager Health and Pre-Upgrade Check Tool from Cisco.
Here’s the output from the UCS Manager Health and Pre-Upgrade Check Tool when I ran it.
Download Firmware Software Bundles
Now we can download the Firmware Software Bundles we need.
I’ve found that using the Cisco suggested release tends to work out nicely. At the time of writing this the Cisco UCS was 4.0(4n) and the Cisco suggested release was 4.2(1m)
My Fabric Interconnects are the 6200. I will download the Infrastructure Software Bundle that contains them.
I also need to download the blade server software bundle.
In my setup I don’t manage any C-Series rack mounted server so I can skip this part. If you mange any C-Series rack mounted server you would need to download this too
Upload Firmware Software Bundles
Now that we have our Firmware Software Bundles downloaded we need to upload them to UCS Manager.
- Click on Equipment
- Click on the Firmware Management tab
- Click on Download Firmware
- Upload the Firmware Software Bundles that we downloaded earlier.
- If we click on Download Tasks we can see what exactly is going on.
Verify Maintenance Policy
The Default Maintenance Policy is set to auto reboot the servers. If we don’t tweak this and apply the blade server firmware, all of the servers will reboot at the exact same time and we don’t want that.
- Click on Servers > Policies
- Click on Maintenance Policies > default
- Change the Reboot Policy to be User Ack
- Save the changes
To see where the policy is applied click on Show Policy Usage. If you don’t find it in use double check the service profile templates or service profiles applied to the servers to find where it is getting the maintenance policy settings and make the tweak if needed to that policy.
Verify Fabric Interconnects
- Click on Equipment > Fabric Interconnects and select one of the Fabric Interconnects
- Expand High Availability Details and make sure it is Ready and and the State is Up
- Repeat this on the other Fabric Interconnect.
Verify Hardware
- Click on Equipment > All and select one of the Chassis
- Click on Servers
- Verify that the servers are all functional
- Click on IO Modules
- Verify that IO Modules are fully functional
- Select one of the Servers
- Select Inventory
- Select Adapters
- Verify the Adapters are fully functional
- Repeat the steps for each server in that chassis.
- Repeat all the steps on the next chassis.
Disable Call Home
There will be many alerts while doing the upgrade. To prevent alert spam we’ll want to disable call home.
- Click on Admin > Communication Management > Call Home
- Set the Admin State to Off
- Save Changes
Infrastructure Firmware Upgrade
We are now ready to install the Infrastructure Firmware.
- Click on Equipment > Equipment
- Click on Firmware Management
- Click on Firmware Auto Install
- Click on Install Infrastructure Firmware
The only warning we have is about the modules and we’ve already confirmed that everything is supported in the new version so we can click Ignore All.
- Click Ignore All
- Select the Infra Pack we uploaded earlier.
- Select Upgrade Now or schedule an upgrade time.
- Click Finish to start the upgrade
Due to not having or needing a service pack we get a warning.
- click Yes
- Click Ok
The Infrastructure Firmware upgrade process has begun.
- If we click on the FSM tab we can watch the progress.
One of the first things it does is upgrades the UCS Manager Web UI.
We will see a message like this which is perfectly normal.
It will take a few minutes before we can use the Web UI again. This is normal.
- Once the Web UI is back up login again to monitor the progress from the FSM tab.
- The next item that will be upgraded is the subordinate Fabric Interconnect.
If you have everything configured correctly there should be no impact to your traffic.
- The next item that will be upgraded is the primary fabric interconnect.
- It will prompt us prior to rebooting the primary fabric interconnect.
- Wait for the alerts in the UCS Manager to clear before giving the all clear for rebooting.
- After the alerts have cleared click on the bell icon to bring up Pending Activities.
- Click on Fabric Interconnects and click Reboot now
- Click Yes
- Click Ok
As it upgrades the primary Fabric Interconnect we will loose access to UCS Manager again. This is normal.
We will see a message like this which is perfectly normal.
It will take a few minutes before we can use the Web UI again. This is normal.
- Once the Web UI is back up login again to monitor the progress from the FSM tab
Once FSM says success we are done the Infrastructure Firmware upgrade and can start on the Server Firmware.
Server Firmware Upgrade
- In Firmware Auto Install Click on General and click on Install Server Firmware
- Click Next
- Select the Firmware Version
- Select which Host Firmware Packages you want to use. In my case it’s default.
- Review the selections and when ready click Next
- Review the impact and click Install when ready
- Click Confirm Install
Due to the tweak that we did on the maintenance policy none of the blade servers will reboot unless we tell them to.
- Click on the Pending Activities.
- When ready select the blade server you want to reboot and click Apply
We can monitor the progress on each blade server.
- From Equipment click on the Chassis and then click on the Server
- Click on the FSM tab
- Repeat the process until all the blade servers are upgraded.
- Once everything is completed, turn on call home if you disabled it.
That is all it takes to upgrade a Cisco UCS. It can be a long process but with proper planning you can do this with zero downtime.
