AI Production Readiness: Build the Foundation Before You Scale

BIZTECH: An organization has been running an AI pilot, and everything seems to be going well. What might they be overlooking as they prepare to move into production?

ZAJDEL: The first thing I see frequently is that organizations have overlooked the governance that’s required. It’s easy to do a pilot with a controlled, curated, massaged data set that you’re using to test the concept. Then, you run it in the wild and discover that you don’t have that same level of quality and curation. It can become the Wild West.

What gets skipped is specific: Nobody owns the data, the business terms aren’t defined and there’s no lineage an end user can see.

The other thing we see is that people try to tackle the biggest problems first. At scale, we see that it’s better to start small and expand. We hear a common refrain: “We’re spending a ton of money on tokens. We invested a ton of money getting our data ready and everything else, but we’re not getting the value out of it.” Often, that’s because they’re trying to do too much.

Keeping it small and expanding from there allows you to go through the pitfalls, solve for them, and understand how you bring your people, processes and data sets along with you. At CDW, we call this minimum viable data governance: ownership, definitions, lineage and guardrails. Do those four things for the use case in front of you, then expand. 

Everybody who represents an AI solution is telling you, “The water is warm, jump in.” That might be true, but you have to make sure you’re ready as an organization and that you’re trying to solve something you’re ready to solve. Keeping your wins achievable will allow you to progress quicker than going for the big win, coming up short and then having to start over.

BIZTECH: Security is another major concern as AI moves into production. What should organizations be considering before they put their data at risk?

ZAJDEL: One of the big areas we see often is that people underestimate the speed at which AI can expose their data to risk, especially if they’re setting up agents that can be ungoverned. You can have automated agents doing what they were programmed to do without necessarily considering how your data is being exposed. An agent inherits the permissions of whoever built it, and too often the builder selected every table because that was faster than deciding which ones the agent actually needed. Scope the data an agent can see before you scope what it can do.

Tokenization of data is one step we believe is necessary before you put your data at risk with an AI application. If you’re setting up a large data set to be used in conjunction with AI use cases, you should be tokenizing the whole data set. The moment data crosses into your organization, you should assume there will be losses, and there will be a breach. You need to protect yourself in the likelihood of that happening, so that if someone gets that data without access to the token vault, the data is useless.

You have to have the mindset that things will progress at a speed you’ve never seen before with AI. As soon as you’re creating agents and giving them autonomy, development will happen at an extraordinary pace. You should assume your data is going to be put at risk, so protect yourself from the start. Build that resilience into the system at the beginning.

BIZTECH: How can organizations assess whether their data is ready for AI, particularly when they have years or decades of legacy data and governance challenges?

ZAJDEL: We’ve taken a lot of steps to aid and assist companies with that journey. There are data quality assessments and readiness workshops that we can engage in to help organizations figure out where they are on that continuum. It’s not necessarily about saying you’re ready or not ready. It’s about understanding where you are on a maturity continuum. Are you five steps away? Four steps away? Are you right there? Are you ready to go?

Organizations also struggle with keeping their governance current. Projects have a beginning, a middle and an end. But governance is a way of life. You have to commit to that forever. If you have turnover in people or organizational structure, you can’t let up and say, “We did that already. We’re governed.” If you let up, it will stop.

Click the banner below to learn how a modern data platform supports smart decision making.

A lot of organizations feel that if they have governance 75% of the way there, they’re better than most. But AI is going to expose that remaining 25% that hasn’t been buttoned down. If you look at a report and say, “That can’t be right. I don’t trust that number,” you already have a data quality problem. AI will simply automate and make that problem much more efficient and much faster.

Older organizations can face an especially daunting challenge because they may be trying to address data quality issues accumulated over decades. But you don’t necessarily have to fix everything going back to the beginning. You can establish robust measures and governance and quality metrics going forward. Start using AI with your current data, and then figure out how to address those past issues.

DIVE DEEPER: Learn what you’ll need to build a foundation for scalable artificial intelligence.

BIZTECH: Are there warning signs during the pilot stage that should tell an organization it isn’t ready for production?

ZAJDEL: In a pilot, a lot of problems can be obscured because you’re working with a small data set or proving only part of a concept. Customers can be fooled by successes in a pilot that don’t translate into value once they move into production.

I also think you want to be broad in building consensus and getting buy-in from the business. One mistake I see often is a small group building a pilot without getting input from the business about whether it will add significant value and be worth the money invested.

People may think, “If I can prove this out, I’ll be able to get more budget for the next project.” But if the business doesn’t see enough value to justify the additional expenditure, the project can stall. That communication is really important. You need buy-in across the entire business.

Data silos are another warning sign. You need common, agreed-upon semantic layers. What is revenue? How do you define it? Is it the same definition everyone else uses? What about sales, profit or territory? If everyone has different definitions, that can become a problem. The model will always answer the question. It answers with whichever definition of revenue it found first, and the person carrying that number into a board meeting has no way to know. Define the terms once and make them portable across platforms, which is what the emerging open semantic standards are built for.

And you need a clear understanding of the costs involved. I’ve seen successful pilots peter out because they turn out to be far more expensive than anyone thought they would be. That needs to be built into the plan. Pilots are cheap because token volume is small. Production cost scales with users, retries, context size and agents calling other agents. Treat AI spending like cloud spending: cost-per-query guardrails and budget alerts per agent from day one.

BIZTECH: What about the people side of AI production readiness? Are organizations prepared with the skills and training they need?

ZAJDEL: Training your coworkers and employees is an absolutely vital step. People need to understand how to use the different types of AI tools, what they’re good for and how to use them. And they need to understand when not to use a particular tool. Using the wrong tools for the wrong types of tasks is a way to run up bills very quickly.

It’s amazing to me that people know AI but treat it a lot like an amped-up search engine, trying to get answers to things. When you start building agents and workflows, the understanding of how things work definitely falls off. Training and that understanding are vital. The platform is rarely the blocker. The humans are. Change management has to be scoped into the project, not bolted on after leadership asks why adoption is flat.

UP NEXT: Learn what you'll need to build a foundation for scalable AI.

Organizations also need a common dictionary of terms and what they mean. And they need to understand the impacts of what they’re asking AI to do. I’ve seen reports where someone is called a “power user” because they run up a big bill. Maybe they are a power user and doing fabulous things. But if they’re running up a big bill, maybe they’re asking AI to do a bunch of things that are simply a waste of time.

You need to validate that activity and make sure everyone is on the right track. AI production readiness isn’t just a technology issue. It’s about making sure your people, processes and data are ready to support what you’re trying to accomplish.

BIZTECH: What’s the most important thing organizations should remember about AI production readiness?

ZAJDEL: Having a solid foundation for your data is the first and primary step. A lot of companies view that like asking their kids to eat their broccoli. It’s not very exciting, and they may not like it, but it’s a necessary step.

There are a lot of folks who will walk in and sell the sexy part first. There’s a lot of sexy stuff in AI that is very, very cool. But you need to have that solid foundation in place. Get five things right: data structure, data quality, semantic layers, governance around usage and tokenization to secure the data. Then the value comes much, much faster.