How Capital One Controls Cloud Costs

BIZTECH: Tell me about Capital One’s cloud journey and how things have evolved over the past few years.

JOHNSTON: Our journey to the cloud started in 2014. That’s when we made our first steps out of data centers and we said, “OK, we’re going to experiment with this thing called AWS.” And over the course of a few years, we have transitioned fully to the cloud. We did that focusing on speed to exit out of data centers; there was quite a bit of lift and shift versus refactoring.

We fully exited our domestic data centers in 2020. We went all in on the cloud because what our customers were expecting from us were personalized experiences. To deliver those personalized experiences, we had an incredible amount of data, but we just couldn’t physically continue to store it in data centers.

Now our customers can have those experiences. When you log on to our app, it really feels tailored to you.

BIZTECH: How did your technical team adapt to the change?

JOHNSTON: There was also a workforce transformation happening in parallel with our tech transformation. The skills that you need in the cloud are similar to what we had for the data center, but not exactly the same.

We transformed our way of working, and that was also the beginning of our agile transformation. Our teams needed to operate in a way that was not an annual release cycle, we needed to learn how to deploy continuously in order to really take advantage of the cloud.

Today, what we can say is that we are fully taking advantage of what the cloud has to offer us. We can scale, we can handle peak loads that we just couldn’t fathom doing in a data center, and our workforce can handle that and help us innovate for the future on things like generative AI and what’s ahead.

RELATED: Experts share their recommendations to improve a modern data center. 

BIZTECH: And you’re still mostly on AWS?

JOHNSTON: About 90%-95%, yes. And the reason why we went all in with a single provider was because if you think about the various capabilities that each cloud platform provides, they’re very similar in nature, but they’re also different. The way you have to build your applications is different. Our point of view is that we are going to remove arbitrary uniqueness.

We decided to focus on a single provider and to take advantage of serverless and be serverless-first. If we were multicloud or even polycloud, we probably wouldn’t have been able to take advantage of the native tools and truly invest our energy into making those work for us. So, how can we take advantage of whatever is native and use that to its fullest extent?

BIZTECH: How did you help your teams make that transition?

JOHNSTON: We had to change our way of thinking about the art of the possible. Within a data center environment, there were very strict boundaries. You were limited in terms of how you could scale and what you could do based on the capacity that was available on the hardware that you had at the time.

You operated within those boundaries. In the cloud, those boundaries are gone. From a workforce standpoint, we had to do quite a bit in terms of encouraging associates through hackathons and innovation spikes and those types of things to help get those boundaries changed. When a team is solving a problem, they have the freedom to not just focus on what they’re working on today but to imagine what the future could be, and really to redefine where they could go.

DIG DEEPER: Make the most of your cloud budget with help from AWS and CDW. 

BIZTECH: How did you manage cloud costs — especially early on?

JOHNSTON: When we started our cloud journey, we were not worried about optimization and how efficient things were. We were concerned with getting to get the cloud. When we got there, a lot of folks did start asking questions about ROI: Are we getting the value out of the cloud that we thought that we would?

Because when we were looking at our costs — and this is once the data centers were closed and we were looking at our projections — it was a steady increase up and to the right in a way that we didn’t expect. We were asked: Is this just the way things are going to be, or are there things that we can do to truly optimize?

We’ve done quite a bit with our FinOps practice in terms of showing how much things cost as well as chargebacks, especially where we have shared platforms. We’ve done a lot in terms of showing where our biggest opportunities to save are. And what powers that and gives it the weight that’s needed is the leadership support and buy-in that we do actually need to prioritize efficiency.

It is a balancing act between efficiency and customer features, and that balance has always been the way of DevOps. But when you are able to strike that right balance and when you’re able to do it with the right tooling that gives you the right information at the moment you need it, you can make those right decisions such that the balance doesn’t feel quite as heavy. And the engineers can make the right decisions in terms of what is reasonable from a spending standpoint.

Click the banner below to learn how financial services can be smarter and more strategic in the cloud.

BIZTECH: How have you gotten your engineers on board with careful spending?

JOHNSTON: We produce quite a bit of reporting. Today, we don’t have this, but where there has been egregious overspending, there’s visibility on it. And thinking about it as good spending and bad spending is how we’ve categorized it.

We’ve gamified it: We give you a score on how well you’re doing. And that is published across all of our divisions, and no one wants to be the last one on that list.

EXPLORE: Why cloud optimization translates to real business value. 

BIZTECH: What do you use to get such clear visibility into your monthly spending?

JOHNSTON: We started with looking at it in retrospect. There’s information coming out of AWS that we ingest. It’s a couple of days behind, but it’s as real-time as we need it. You shouldn’t make in-the-moment spending decisions; they should be looked at as trends over time.

We ingest that data and we look at it raw. It gives you information but not insights. What we did is turn that information into insight by marrying it up with our configuration management database, so that we could say this application, which is an aggregation of X amount of services and X amount of usage within AWS, costs this much, which was based on the information we ingested from AWS. That was step one. We can show how much something costs in a meaningful way and not just in the way that we get it from AWS.

The CMDB is essentially our asset inventory, and that is baseline. If that is wrong, then everything on the stack ends up being wrong. We put an incredible amount of energy into making sure that’s right. That provides a mapping of who owns this application, what this application does and where this application is running.

BIZTECH: How are you measuring success?

JOHNSTON: There are two measures of success that we look at, and these are actually the only two that we measure from our FinOps practice. There’s so much that we could measure, but really it just boils down to these two.

The first one is around efficiency. Basically, are you rightsized? We look at it from both a computing and a data standpoint, and that is using all of the telemetry that’s coming off the boxes.

The second measurement that we use is spending predictability, because if we’re able to predict how much we are going to spend, then we’re able to manage it. We measure each application because we do forecasting every year and we want to know how close we were to the forecast. Our goal is to be within a couple percentage points of spending predictability. If these two things are in check, we’re doing the right things. We know how much things will cost and we are managing it. Those are essentially the goals of our FinOps practice.

Click the banner below to subscribe to our newsletter for the latest financial services IT insights.