Many organizations that operate in the cloud have different technology needs than they had when they made their initial investments. As organizations re-evaluate their cloud environments, they need to consider how artificial intelligence will factor in. A McKinsey Global Survey found that the percentage of organizations using an AI tool for at least one business function jumped to 72% this year, up from 55% in 2023. Generative AI use nearly doubled in the same time frame: 33% of organizations used it in 2023, compared with 65% in 2024.
To accommodate this shift, organizations must consider the data users are feeding into artificial intelligence models and how that data is governed. Here’s what they need to know.
Click the banner to tailor a cloud solution to your organization’s needs with CDW’s help.
The Risk of Sharing Sensitive Data with Public AI Platforms
As businesses embrace generative AI platforms, IT professionals must find ways to ensure that sensitive data isn’t being shared publicly. Users need ways to explore large language models without disclosing any of their data.
“First, we do a data governance check. What kind of data are you going to be using? What are the controls around that data? Then we can design a solution that allows you to keep your data in-house and not expose any of it,” says Roger Haney, chief architect for software-defined infrastructure at CDW.
Data governance is key for organizations looking to prepare their infrastructure and users for AI and LLMs. “We have a workshop called Mastering Operational AI Transformation, or MOAT,” Haney says.
In this course, businesses learn why AI technology threatens to breach the metaphorical castle of the enterprise, and how to manage the complexity of these risks so IT leaders take the right precautions to safeguard their data.
“You’re drawing a circle around the data that we don’t want to get out. We want it to be internally useful, but we don’t want it to get out,” Haney adds.
Working with a tech partner can also bolster data security and help IT leaders curate cloud solutions that don’t rely on public LLMs. This gives them the benefits of generative AI without the inherent risk.
“We’re getting a lot of businesses coming back to us, asking, ‘How do we create governance not only on the data but also on equipment usage?’ They spent a lot of money. They’ve gotten some early successes, but now they really need to keep an eye on that bottom line. We’ve had some challenging quarters, and the businesses are feeling that. So, how do they best utilize their resources and do this in a cost-effective manner?”
One solution is “to set up your cloud in such a way that we’re able to use a prompt to make a copy of an LLM,” Haney explains. “We build private enclaves containing a chat resource to an LLM that people can use without a public LLM learning the data they’re putting in.”
65%
The percentage of organizations using generative artificial intelligence in 2024
Source: mckinsey.com, “The state of AI in early 2024: Gen AI adoption spikes and starts to generate value,” May 30, 2024
When to Host AI Databases in the Cloud
Organizations’ plans for generative AI will determine how they should prepare their infrastructure for the future of this technology. Haney says most users want to communicate with their data for retrieval or analytical purposes.
“Chatting with your data doesn’t require a new data store. You don’t have to build a huge data lake or warehouse,” he says. “If you have data, then we add another model that can create the query in SQL, do the query and pull the data back. Then you can ask it questions, using that data as part of your prompt, and you can ‘talk’ with your data.”
Creating a retrieval-augmented generation database allows businesses to ask a simple question and get two or three top answers fast.
“If you’re going to do 20 queries per second, for example, you probably could do that on-premises,” Haney says. “If you’re going to do 200 queries or — if you’re a company the size of CDW and you’re building an HR bot — 500 queries per second, you want to do that with resources that are scalable. That’s where the cloud comes in.”
“With a fine-tuned model, you need heavy GPU resources because now you’re embedding that information into the model itself,” Haney explains. “We do most of that work in the cloud, where we’re able to rent a GPU or a TPU [tensor processing unit], and it’s a lot less expensive.”
When it comes to determining how you’ll prepare your cloud infrastructure for AI, think first about how you want to use AI, how you want to use your data and what that will require internally for workloads, training and preparation. If IT leaders ask themselves where they want AI to take their digital infrastructure next, they will be better poised for success as they reverse engineer to make it happen on an enterprise level.
UP NEXT: When is the cloud right for organizations deploying artificial intelligence?
