Houston Texans Score Big with Major Tech Plays

When the Houston Texans joined the National Football League in 2002, the team had to build a fan base. Doing so meant winning games, but it also required good customer service.

“From the start, we’ve catered to the needs of our season ticket holders and sponsors,” says Jeff Schmitz, the team’s vice president of IT. “How we communicate with our customers is a big part of our business. We’re definitely centered on unified communications.”

Ticket sales representatives, for instance, rely on call center technology to help fans and business sponsors choose or upgrade their seats and luxury suites. And on gameday, customer service representatives staff a hotline to handle questions and requests.

Today, fan loyalty is strong. The Texans, winners of consecutive division championships in 2011 and 2012, have sold out 122 straight games. Top-notch customer service has remained a constant throughout — thanks in no small part to the team’s unified communications (UC) infrastructure.

In 2010, the Texans’ IT department began a campaign to modernize its IT and communications systems so it could continue to meet the needs of customers and employees. First, IT beefed up its data center and upgraded the organization’s network. Then, in 2012, the staff replaced an aging Voice over IP phone system with a new Cisco VoIP system and call center services to provide high-quality voice capabilities and other new features.

The new infrastructure — installed at NRG Stadium and the headquarters of the McNair Group, the financial and real estate firm of Texans owner Robert McNair — provides employees with many UC capabilities, including video calls, instant messaging and the ability to send office calls to mobile phones. It supports customer service needs and lets em­ployees from business operations communicate and collaborate effectively, boosting productivity, Schmitz says. It is also much easier to manage, resulting in time and cost savings for the IT team.

The football operations staff also welcomes the new technology. Using IP phones, coaches regularly make video calls to one another. And thanks to the new bandwidth, IT was able to build a fast wireless network for players and coaches at both the stadium and the team’s nearby practice facility. The wireless LAN has allowed the team to replace paper playbooks with Apple iPad devices, a trend throughout the NFL. Coaches can now deliver digital playbooks to players — a big change that allows them to spend more time developing the game plan, says Russell Joyner, the Texans’ director of football information systems.

“In our business, we only have one week to prepare for games, so every hour saved is one extra hour of productivity,” he says.

The Red (Hot) Zone

In summer 2010, the Texans’ IT team faced a pair of related challenges: It needed to incorporate servers and storage from the NFL to meet league requirements, while at the same time boosting its own server count to support future growth. It became clear that the combined additions would push the data center beyond its cooling capabilities, Schmitz says.

Already, the team’s facilities faced occasional brownouts. When that happened, existing uninterruptible power supply systems would sometimes fail, causing servers to shut down. “Some of the UPS units weren’t made for the environment,” Schmitz says.

IT solved the data center power and cooling problems by purchasing a high-end APC Symmetra PX UPS and four new APC air-conditioning units. They redesigned the data center to include hot and cold aisles, which improved airflow and also reduced cooling needs.

With the data center ready, Schmitz tackled the network. In late 2010, CDW engineers replaced one core switch with two Cisco Catalyst 6504-E switches. The upgrade not only introduced much-needed redundancy, but also increased backbone network speeds from 1 gigabit per second to 10Gbps.

Beyond the core, the Texans and CDW also upgraded the network edge with four new Cisco Catalyst 4506-E Series switches and eight new Cisco Catalyst 3750X Series switches at NRG Stadium and the nearby practice field, as well as new Cisco Catalyst 3750X Series and Cisco Catalyst 4507-E Series switches at the McNair Group headquarters.

The faster, more robust network not only ensures good voice quality, but also improves application performance and speeds video distribution throughout the stadium, Schmitz says.

Photo: Phoebe Rourke-Ghabriel

“It absolutely gives us a reliable phone system with the added benefit of instant messaging, accessing voicemail in email and call clarity.” says Jeff Schmitz, Vice President of IT.

UC Kickoff

As if on cue, with the high-performance network in place, the phone system needed attention. By 2012, voice quality had deteriorated and the existing system suffered frequent (though controlled) outages. According to Schmitz, the IT team had resorted to taking the phone system offline after 5 p.m. on a regular basis to replace parts and keep it operational.

Moreover, because the communications system lacked redundancy, the McNair Group offices would occasionally lose phone and voicemail service when the network connection to NRG Stadium (where the primary phone system resides) was interrupted.

“We did a lot of after-hours work and nursed it to keep it going those last two years,” Schmitz says. Enter the Texans’ new Cisco UC system and VoIP service.

Five CDW engineers helped the Texans install the new phone system and call center over a six-week period leading up to the team’s first 2012 regular-season game. They deployed the UC software on a Cisco UCS C-Series server and equipped each employee with an IP videophone.

The team architected the system to ensure uptime and quality of service. It installed a backup UC system on a second server at the McNair Group office so that if the main phone system should go down, the backup will ensure that employees in both locations have a dial tone. Network engineers also added fiber connections to each edge switch to prevent future connectivity failures.

To keep quality of service high, engineers also implemented policy-based routing between the stadium and the McNair Group. “We are better able to integrate with our remote office and keep voice traffic separate from data,” Schmitz says.

Perfect Handoff

When the new phone system went live, Schmitz says, employees loved it.

“The sound quality is night and day” compared with the old system, he says.

For business operations, employees use Cisco Jabber. The application lets staff launch all UC services, including instant messaging and voice and video calls, and check voicemail. The system’s call forwarding and unified messaging features quickly improved customer service. On gameday, customer service staffers can forward calls from their desk phones to their smartphones while roaming the stadium, which means there’s no interruption to the service they provide fans, says John Schriever, senior vice president of ticketing and event services.

The corporate development team benefits in the same way. They prefer face-to-face meetings with sponsors at the stadium, but if sponsors call with a need, team members can use call forwarding to retrieve messages on their smartphones and respond quickly.

The new Cisco call center software also allows management to adjust staffing as call volume increases. For example, if the team is selling playoff tickets and the center is flooded with calls, customer service agents can log in to the ticket sales call center to help, Schriever says. “People jump in and help during peak demand.”

This summer, the Texans’ IT department plans to upgrade to the latest version of Cisco UC and equip employees with a Jabber app for their smartphones to further improve communications on the go.

Overall, the new technology has been worth the investment, Schmitz says.

“It absolutely gives us a reliable phone system with the added benefit of instant messaging, accessing voicemail in email and call clarity,” he says.