Jul 22 2015

NYSE Outage Highlights Just How Essential Business Continuity Is for Every Company

The high-profile outage of the New York Stock Exchange should cause organizations in financial services to pause and reflect.

The world moves at a fast pace. But few organizations in the world move at a more blistering pace than the New York Stock Exchange (NYSE).

We’ve all seen the Hollywood movies that depict the frenetic activity that is the NYSE’s trading floor. But in reality, much of the trading action these days happens machine to machine, not human to human. Bob Ivry, a Bloomberg News financial journalist, says that 50 to 70 percent of trades in the U.S. are done by robots, reports Marketplace.

While machine trading has advantages, namely speed, it also means that a technical glitch, like the one that happened to the NYSE on July 8, can bring everything to a screeching halt.

The outage, which lasted for nearly four hours, according to The Washington Post, has led to a great deal of analysis, concern and reflection from those who work in financial services and fintech.

What lessons can be gleaned from this headline-grabbing outage, and how can IT play a better role in getting ahead of such outages?

Ensure Decision-Makers Are Accessible and Empowered

In moments of crisis, process can be useful, since it attempts to control what is a naturally chaotic situation. Bureaucracy, however, can be a paralyzing bottleneck. A TechRepublic article reflecting on the NYSE outage advises organizations to build out discipline-specific response teams and ensure that executives are included on each team to “blow up” gatekeepers or make swift purchasing decisions, if needed.

A crisis demands urgent action; having to figure out who is capable of making a decision in an emergency and where they are at that moment could cost your business time that it can’t afford to lose.

Make Sure There’s Failover

Without a doubt, the NYSE’s outage was a big deal. But the world didn’t stop because of it. The global exchange markets are fragmented in many ways to avoid standstills, according to a report from Reuters:

Many traders said that it did not matter that the NYSE was down. That's because there are 11 U.S. stock exchanges, including those run by Nasdaq OMX Group (NDAQ.O) and BATS, along with more than 40 private stock-trading venues, so the trading of NYSE-listed stocks was uninterrupted.

"This is one of the rare cases where the fragmented markets we live in actually serve a purpose," said Dave Nadig, director of exchange-traded funds at FactSet Research Systems. "If this happened at (the London Stock Exchange), you would just be sitting staring at a blank screen."

In a similar manner, whether it’s through technology or structure or both, companies must plan for outages and ensure that the world doesn’t stop when systems go down.

Does the Cost of New Tech Outweigh the Risk of Implementation?

The reason for the massive NYSE outage has been pinned on a software update to the Securities Information Processor. The SIP “is a consolidated feed that aggregates the direct data from three key datacenters housing the 12 exchanges plus the ADF and TRFs,” according to Traders magazine.

The software enhancement to the SIP was apparently done in an attempt to make electronic trading — which already operates at the millisecond level — track data at the microsecond level.

In a Traders opinion piece on the outage, David Weisberger, managing director and head of market structure analysis of RegOne Solutions, argues that the benefit of the microsecond enhancement wasn’t worth the cost of the subsequent outage. To make his case, he revisits the impact that financial IT systems had to bear back when the Y2K bug cast its shadow across the global economy and notes that since the SIP is a single point of failure for the entire global financial system, the enhancement likely wasn’t worth the risk.

In order to understand the level of expense we're talking about, consider that Y2K was an addition of two characters to a date field and this change is adding three significant digits to a time field.

While I am not suggesting it will be as expensive, since the systems involved are more modern, it is not going to be cheap. Thanks to the NYSE, we also know that it carries serious operational risk.

I suppose we should be thankful that it was "only" the NYSE that failed and not the SIP itself, as that would have essentially caused a market wide shut-down. As previously noted, the SIP is a single point of failure in our market since many systems rely upon it to validate trading.

Whether you agree or disagree with Weisberger’s take on the value of the SIP upgrade, IT leaders in financial services must always make informed cost/benefit comparisons before jeopardizing operations for something that might not be worth the collateral damage. Perhaps more importantly, the possibility of outages must be considered, and recovery tactics must be planned, in case the unthinkable happen.

Spencer Platt/Getty Image News/ThinkStockPhotos