Why AI Is Forcing Capital Markets to Rethink Compute Strategy, Again

AI Changes the Economics of Computing

One of the most misunderstood dynamics in AI adoption is cost behavior. Training large language models is expensive but episodic. Inference is different: Once models are embedded into workflows, inference becomes persistent, high-frequency and latency-sensitive. It touches every trade, every alert, every decision loop.

In capital markets, that means:

• Continuous scoring of transactions for surveillance
• Intraday risk recalculation
• Real-time pricing and scenario analysis
• AI-augmented market data interpretation

These workloads don’t scale like Software as a Service applications. They scale like on-premises infrastructure. Firms that assumed “We’ll just run it all in the cloud” are discovering that recurring inference, constant application programming interface calls, data egress and performance tuning can produce cost curves that are volatile — and in some cases, structurally misaligned with business value. This isn’t an indictment of cloud — it’s an architectural mismatch.

DISCOVER: What is CDW’s strategic application modernization assessment?

In capital markets, latency is not a key performance indicator; it is a hard boundary. Certain workload types — execution analytics, market surveillance, intraday risk and pricing — simply cannot tolerate unpredictable latency. Even small variances can compound into operational risk, missed opportunities or regulatory exposure.

That reality drives companies to require their computing resources to be closer to market data sources; trading venues and colocation facilities; and high-performance, on-premises or private environments. AI amplifies this constraint. Models that depend on real-time data feeds and immediate response loops demand deterministic performance. Public cloud excels at elasticity, but determinism is not its native strength.

The result is not a binary decision, but a placement problem. What runs near the edge? What runs centrally? What bursts to cloud? What must remain controlled?

Click the banner to sign up for our newsletter and receive more business IT insights.

Why Data Gravity Matters More Than Ever

AI follows data, not the other way around, and capital markets data is fragmented, sensitive and heavily governed. It includes proprietary models, client information, trading records, market data entitlements and regulatory artifacts that cannot simply be centralized without consequence. Modern AI increases — not decreases — the need for:

• Fine-grained access controls
• Lineage and auditability
• Policy-driven data movement
• Separation of duties

This is why the conversation is shifting away from “Where do we run models?” to “How do we architect data so models can run anywhere safely?”

The answer increasingly includes hybrid lakehouse architectures, data fabrics that abstract location, open formats that avoid lock-in, and governance that is embedded at the data layer, not bolted on later. AI is forcing firms to mature their data architecture faster than most modernization roadmaps anticipated.

As AI moves into regulated transaction flows, many firms are quietly relying on tokenization and format-preserving data protection to run analytics and inference on sensitive data without moving or exposing the underlying records — further reinforcing why hybrid architectures persist.

Capital markets firms understand operational risk better than most. But AI introduces new dependencies: compute platforms, model services and data pipelines that can quietly concentrate risk if left unmanaged.

Cloud concentration risk is no longer hypothetical. Regulators are paying attention. Boards are asking harder questions.

Hybrid architectures, when designed correctly, allow firms to avoid single-provider dependency; isolate critical workloads; design for failure, not perfection; and maintain their options as technology evolves. Resilience is no longer just about uptime; it’s about architectural freedom under stress.

DISCOVER: Get the tech trends impacting financial services organizations in 2026.

The Operating Model Is the Hidden Bottleneck

Technology is rarely the limiting factor. People and process are. Many firms struggle not because they chose hybrid infrastructure but because they never updated their operating model to support it. Hybrid environments require:

• Platform teams, not ticket queues
• Guardrails instead of bespoke approvals
• Clear workload classification standards
• FinOps discipline tied to business outcomes

AI makes this unavoidable. You cannot scale AI responsibly without a platform mindset that spans on-premises, cloud and edge. This is where many transformations stall: It’s not due to lack of vision, but to organizational inertia.

The wrong question continues to dominate technology decision points: “Should we move this to the cloud?”  The right question is far more strategic: “Where should this workload live to optimize performance, cost, resilience and control, now and over time?”

That question has different answers depending on a range of factors, including latency sensitivity, data sensitivity, frequency of execution, regulatory exposure, cost elasticity and more. Hybrid compute is simply the acknowledgment that capital markets do not have homogeneous workloads, and AI is making that heterogeneity impossible to ignore.

DIVE DEEPER: Get help breaking down financial services data silos.

Designing for Reality, Not Narrative

AI is not pushing capital markets firms backward. It is forcing them to continually mature architecturally. The firms that will lead are not those that declare allegiance to a single platform, but those that: 

• Design with constraints, not slogans
• Treat compute placement as a strategic discipline
• Invest in data foundations before scaling models
• Build resilience deliberately, not accidentally

Hybrid is not a compromise. It is a recognition of reality. And AI is making that reality unmistakable.

This article is part of BizTech's EquITy blog series.