Lab Hosting Averts Migration Outages for International Airport

WWT's Lab Hosting services were used to manage, construct and host the proposed lab solution to mitigate potential service outages.

Challenge

After experiencing a challenging data center migration, a major international airport turned to WWT prior to migrating to a new Nexus 7706 platform in their data center and integrating a new secondary data center.

As part of the project scoping process, the airport's technical staff toured WWT's Advanced Technology Center (ATC) to explore lab services and capabilities. The customer was immediately convinced that the ATC facility and WWT technical engineers and architects were ideal for hosting their data center migration lab. The team saw that the ATC could host their development environment at scale. This assured them that their production environment wouldn't be impacted during the actual migration and past migration challenges wouldn't resurface.

Solution

A jointly reviewed test plan documented a methodical approach that encompassed all areas of validation needed for successful integration and migration. A documented, robust and repeatable test plan ensured that all steps were executed and all observations were captured for accuracy. Specific device configurations were duplicated to fully reproduce the airport's data center environment on-site at WWT.

WWT provided logistics and expertise to gather and construct the hardware needed to host the airport's current production environment as well as the proposed secondary data center. This environment allowed the airport to test the steps needed to complete the migration and upgrade scenarios prior to implementation activities and observe any impact on the production network. Additionally, WWT provided quality assurance engineers to oversee the project during the three weeks of on- and off-site testing.

The customer's actual new hardware was delivered to WWT's ATC and used for testing, thereby eliminating the potential of field DOAs and reducing the time to integrate into the customer's production network. This process also saved the customer from having to reconfigure devices a second time.

Execution

The first test case was to mimic the current data center. The environment consisted of UCS servers, fabric interconnects, Nexus 2K/5K/7K and Catalyst switches. All configurations and firmware precisely mimicked the airport's main data center. Then, the new data center equipment was introduced — four Nexus 7706 switches and a Catalyst 6800 core device.

In the initial test case, the team turned up the new campus 6800 core router. During this process, the links between the core and their 7010s were immediately shut down due to Bridge Assurance. This unanticipated outage exposed a configuration anomaly between their production environment and test environment and highlighted a configuration change that needed to be included in the migration plan. After adding the additional code, the links between the core and 7010s came up as anticipated. This addition caused six packets to be lost and 200-millisecond delays due to converging routing protocols. This was deemed acceptable by the customer IT team and easily handled by the Transmission Control Protocol (TCP).

The second step was standing up the new 7706 switches that will replace the existing 7010 switches. When first executed several links were disabled, again due to Bridge Assurance. During troubleshooting, WWT and airport engineers discovered that the vPC peer links were not added to FabricPath. As a result, the order of operations in the procedure was corrected.

Next, the 7706 links were turned up as part of FabricPath with success. The issue and fix were also applied to the 7706s slated for their new secondary data center, and a third data center outage was averted.

Initially, the airport team wanted to use Hot Standby Router Protocol (HSRP) with Spanning Tree filtering to allow the secondary data center HSRP group to be separate from the main data center HSRP. This would allow traffic to be forwarded more efficiently. But through risk analysis, the airport engineering team ultimately decided to use HRSP Anycast, which allowed all traffic to be forwarded efficiently with less risk to the environment. Had it not been for the WWT lab environment this feature would not have been discovered.

Crucial to testing was the ability to send traffic through the data center environment using Ixia to simulate traffic and measure the degree of latency and duration. This provided the deltas and data points needed to prove the migration strategy was successful.

Results

The lab tests were deemed a complete success. They provided valuable lessons learned during the Lab Hosting process and provided the airport with assurance that the data migration in the production environment would be flawless.

Because the airport had shipped its equipment to WWT for its Lab Hosting environment, we were able to pre-configure the new equipment and ship it back ready for installation.

The customer expects to engage WWT further after the positive experience with their new data center migration.

– Airport Lead Architect 

If your organization is ready to upgrade its data center, click here to learn about how we can work with you to help you design the solution that meets your business needs.