The Demands of AI on IT Infrastructure
As AI becomes more widely used, the millions of applications that rely on it will create vast amounts of data. “There is insatiable demand” for running large language models, Nidhi Chappell, general manager of Microsoft Azure’s AI infrastructure, told the The Wall Street Journal, especially in industries such as finance and manufacturing.
Organizations need IT infrastructure that can support this additional weight. This includes networks and storage that can handle huge amounts of data and computing chips that can meet the processing demands of AI.
“The biggest thing I'm asking most about right now is the impact of AI on IT operations,” said Steve McDowell, principal analyst at NAND Research, in an interview last year. “There's a lot of hype around AI, and that impacts IT. [They] need to figure out how to build the infrastructure to support it because traditional compute does not account for it. We're seeing thousands of experiments across organizations, and IT has to step up and support those often in short order, so it's a challenge operationally.”
Businesses can rely on cloud providers to shoulder some of this load. According to The Wall Street Journal, hyperscale cloud providers — including Amazon Web Services, Microsoft Azure and Google Cloud Platform — are taking steps such as deploying AI-optimized server clusters to help them prepare for the demands of AI applications.
Companies looking to use the cloud to handle AI workloads must choose the right provider because many applications rely on different microservices for different functions, and a problem with one provider could have far-reaching effects. Solutions that can establish seamless connectivity between cloud and on-premises environments, such as IBM’s Hybrid Cloud Mesh, can help companies mitigate such issues.
DISCOVER: How businesses can unlock the value of generative artificial intelligence.
As they increase their investments in AI and the cloud, companies need to take steps to enhance their chances of success. To begin, they need to improve the levels of staff expertise in AI. “Business and tech leaders must build for the future now,” notes a post on CIO.com. “Working with people managers to develop the skills stack to support the tech stack ensures organizations can take advantage of current and future AI capabilities — all powered by data.”
Businesses also should focus their hybrid cloud strategies to support their AI initiatives. A strategic approach can help companies optimize their AI workloads for efficiency and productivity. “Bursty workloads can reside in the public cloud,” writes Dion Hinchcliffe of Constellation Research, “while mission-critical, always-on AI workloads can be migrated to a private cloud environment, optimizing cost and performance.”
UP NEXT: How are U.S capital firms using the hybrid cloud?
Finally, organizations also should consider the compliance implications of their cloud AI efforts. For example, the European Union earlier this year adopted the Artificial Intelligence Act, which establishes rules for data quality, transparency, human oversight and accountability in the use of AI, and sets steep penalties for noncompliance.
As they move forward with a variety of AI initiatives, businesses will face some challenges in hosting these workloads in the cloud. They must approach these efforts strategically to optimize their outcomes and maximize success.
