United Air Lines Saves Through Optimized Storage

When you tell him that on average, storage admins manage between 30TB and 60TB each, Samuel Turner smiles. As United Air Lines' manager of storage utility services, he has good reason: his staffers each manage triple that amount. "They manage nearly 200TB per person," Turner says, noting that although his staff is pushed, such amounts are doable, at least for the time being.

Turner says he can get away with those numbers because over the past few years United has worked hard to simplify and virtualize its storage environment. "Now, we're a utility service. And much like a utility, I work to optimize the storage capacity and resources within the organization so we can better manage our dollars associated with storage," he says.

"We look at the needs not only of the high-end users but also the low-end services and applications with an eye toward trying to centralize, optimize and drive higher efficiencies through reuse of structured services. And that lets us reduce our overall cost per unit of storage for the whole environment." But getting to this point hasn't been easy. Like many large organizations, United had a hodgepodge of storage arrays and disks in place as it took advantage of vendors that were cutting each other on price. "The rub is we needed to have resources trained on the nuances for each particular brand we used because they all operate just a bit differently," Turners says.

The Flight Plan

Faced with such inefficiencies, United made a strategic decision to narrow its pool of storage players to one or two high-level brands. The idea was to reduce costs by simplifying. Turner says his team gravitated toward EMC and Hitachi Data Systems. Eventually, it decided to go with Hitachi as its main vendor, primarily due to the high-level features inherent in Hitachi's TagmaStore Universal Storage Platform and its new Tiered Storage Manager (TSM) software. The TagmaStore is a large array that can handle 332TB of internal storage and as much as 32 petabytes of total storage capacity. The unit not only offers virtualized internal storage suitable for Tier 1 high performance storage services, it also can virtualize attached storage from a variety of vendors on the back end, enabling United to build virtualized Tier 2 and Tier 3 storage services. "The Hitachi became the front line, the gateway and entry for all of our equipment in the storage network, as well as all of the hosts and servers out there," Turner says. "It lets us shop and somewhat commoditize the disk storage below the Tagmas, but it also let us integrate it so that way we can still maintain that single pane of glass. We can manage our environment holistically."

ILM Spells Savings

ILM Spells Savings

A key to building this tiered infrastructure at United was understanding the airline's storage requirements and applying smart information lifecycle management strategies. This meant understanding what data is generated by the applications, how much is stored, for how long, how critical it is and how quickly users need access to it. Unfortunately, United's various lines of business had trouble providing the desired level of detail. "Parts of the organization are not yet at a point where they can communicate those requirements or even begin to understand the nature of the data itself enough to be able to share those requirements with us," Turner says.

Turner's group decided to tackle the problem another way, using the Hitachi TSM software to gauge the activity of the various storage volumes and build out a tiered infrastructure from there. The software draws patterns and, based on criteria United provides, indicates which data is eligible for tiering. More active volumes fall into the higher tier while data that's not used often may be relegated to a lower-cost, lower-speed tier, or even archived. Turner says he can use the tiering infrastructure to purchase storage smarter. "Instead of spending $50,000 and getting 2TB of high-performance disk, I'm going to spend $50,000 and get 10TB or 15TB of Tier 3 disks," he says. "Or instead of getting 2TB of high performance, I'll get 5TB of Tier 2 and 10TB of Tier 3, or whatever makes sense."

SAN Smarts

United also is playing smarter on the storage-area network (SAN) side of the equation by leveraging Onaro's SANscreen tool to track and analyze SAN connections, ensuring optimal performance. "Before we had SANscreen, we were constantly having server admins call us saying they lost the SAN connection, or that something's wrong with the SAN," Turner says. "We'd scramble around to figure out what's going on, only to find that the problems were due to changes taking place at the host level. They would change drives, swap out HBA cards, or sometimes disconnect themselves from the network, and then forget a path was missing." With SANscreen in place, those problems are instantly obvious. "SANscreen sets a baseline of the SAN," he says. United defines rules that say how many connections or paths a given server should have to the SAN and then SANscreen checks the paths every few minutes to make sure all is well.

"If something changes, we get an alert, and the SANscreen engine does an analysis and gives us its best guess about what could be the issue. It's a nice tool." United further stretches its admin dollars by providing read-only, browser-based SANscreen access to server admins. "They can do some self service and take a look and see the configuration of the SAN and if their server still has its path," he says. "And that also lets me minimize the number of team members I have to support the SANs." Turner says the next step for United and its storage utility strategy is to offer service-level agreements to users based on their specific requirements. "So it's more about what your needs are for response time, availability, redundancy and the like, and then we map that to our infrastructure," he says. "Ideally, we'll come up with several packages of services that are appropriate for our company, and then offer those packages as needed. That's what we're working toward."