Building AI Factories for Financial Services: From Strategy to Scale
A practical guide to deciding where AI workloads should run, designing the infrastructure to support them and preparing for production.
The new rules of transformation
Over the course of the past several years, generative AI (GenAI) has moved from the edges of the IT agenda to its center and has expanded to include facilities, data scientists, application developers and business owners in addition to IT. For financial services firms, the implications are especially significant: GenAI is no longer a capability being evaluated in innovation labs. It is being embedded into fraud detection, credit underwriting, AML transaction monitoring and customer-facing services. These workloads run at scale, touch regulated data, and carry direct business and compliance consequences. The phase of cautious experimentation is over.
McKinsey's 2026 State of AI survey found that 44% of organizations are now scaling AI across the enterprise, up from 38% a year earlier. Yet only 37% attribute any earnings impact from AI. Further, just 6% of organizations qualify as AI high performers, meaning they see significant financial impact from their AI initiatives. In financial services, the gap is compounded: scaling AI requires not just infrastructure capacity, but infrastructure that satisfies regulators, protects customer data, and meets the auditability standards that model risk frameworks demand. The gap is not ambition. It is infrastructure — and governance.
Production-grade AI has fundamentally different requirements from the environments most organizations built for experimentation. It is not an upgrade to existing data center strategy. It is not a larger version of a workload. It is a new category of infrastructure, with its own architecture principles, its own operational requirements, and its own set of decisions that, once made, are difficult to reverse. Infrastructure and IT operations (I&O) leaders are being asked to make those decisions now, often without the frameworks or precedents that have guided every previous generation of technology investment.
This playbook is written for those leaders. It addresses two things directly: a decision framework for determining where AI workloads should run, and a reference architecture for building the infrastructure to run them.
Why AI programs stall before they scale
The gap between a working AI prototype and a production AI system is wider than most organizations anticipate. Prototypes are built to demonstrate capability. Production systems are built to deliver it, repeatedly, at scale, under operational and regulatory constraints that no proof of concept ever must satisfy. In financial services, those constraints are among the most demanding of any industry, including data privacy obligations under GLBA and audit trail requirements that compliance teams and regulators will scrutinize. Additionally, while GenAI and agentic models are explicitly excluded from model risk management under the Federal Reserve's updated SR 26-2 guidance, financial institutions are expected to establish their own GenAI governance frameworks. The NIST AI Risk Management Framework provides a strong foundation for doing so.
One common failure mode is trying to scale AI workloads through infrastructure that was never designed to carry them. General-purpose cloud environments, legacy compute clusters, and shared networking architecture can support experimentation. They cannot reliably support the data throughput, latency requirements, and governance demands of production AI at enterprise scale. For financial services firms, the consequences are amplified: a fraud detection model that cannot scale to production throughput is not a delayed project — it is a risk management gap. The result is a pattern that has become familiar across financial services: a successful pilot, a decision to scale and then months of unplanned infrastructure remediation before anything reaches users.
A second failure mode is sequential decision-making. Many organizations approach AI infrastructure the way they approached previous technology generations: evaluate, select, procure, deploy, repeat. However, this approach can lead to bottlenecks that carry greater consequences in an AI Factory versus traditional environment. In an AI Factory, higher potential performance and greater complexity are paired with substantially higher acquisition and operating costs. A GPU cluster under-supported by storage throughput will not perform to specification. A high-performance storage environment on the wrong network fabric will not deliver its rated throughput. The performance of the system is determined just as much by how the components work together as it is by the specification of individual parts. That shift in approach is what compresses deployment timelines and reduces the technical and commercial exposure that comes with first-of-kind implementations.
Part 1: The onshoring decision framework
Where AI workloads run is one of the most consequential infrastructure decisions a financial services technology leader will face. The default assumption of the previous decade that public cloud is the appropriate home for most enterprise workloads is under active review across every segment of the financial services industry. For banks, insurers, asset managers and broker-dealers alike, three converging forces are driving that shift: regulatory pressure, operational requirements and commercial logic.
This framework gives you a structured way to make the placement call, workload by workload, across all three.
Step 1: Map your AI workload estate
Before applying the framework, establish a complete inventory of your current and planned AI workloads. This estate-mapping step aligns with WWT's AI Technology Capability Framework, which uses workload inventory as the entry gate for placing organizations on an AI Factory or pre-Factory adoption pathway. For each workload, capture:
- Workload type (data preparation, training, fine-tuning, inference, agentic AI)
- Data classification (what data the workload touches and its sensitivity)
- Latency and performance requirements (real-time, near-real-time or batch)
- Expected volume and growth trajectory over three to five years
- Current hosting environment and any known constraints
- Quality/accuracy of response
- Required time-to-market
Step 2: Apply the three-trigger test
For each workload, apply the three triggers below. A single 'yes' to a regulatory question requires a dedicated, sovereign architecture.
Trigger 1: Regulatory
| Question | If yes |
|---|---|
| Does this workload process data covered by GLBA, PCI DSS or applicable state privacy laws (e.g., CCPA, NYDFS)? | Dedicated / onshore infrastructure required |
| Does inference or model training involve customer financial data, PII, transaction records or credit information? | Map against applicable residency obligations before proceeding |
| Is your organization subject to FFIEC guidance, OCC model risk expectations, FINRA Rule 3110 or SEC AI oversight requirements? | Sovereign, on-premises or co-located infrastructure strongly indicated |
| Do you operate under SR 26-2 model risk management expectations, and do those models require auditable, controlled compute environments? | Engage legal and compliance before finalizing architecture |
| Does this workload serve customers or data subjects in the EU, UK or other non-U.S. jurisdictions (e.g., GDPR, UK GDPR, EU AI Act)? | Map to jurisdiction-specific residency and AI obligations and engage legal |
For financial services firms, the regulatory exposure is already acute: GLBA requires robust safeguards for customer financial information, federal banking supervisors expect existing model risk, third-party and information security guidance to apply to AI, and the SEC's examination priorities on AI governance and disclosure are raising compliance expectations.
That pressure is building globally. Gartner predicts that by 2028, 65% of governments worldwide will introduce technological sovereignty requirements to improve independence and protect against extraterritorial regulatory interference. Firms with customers or operations in the EU face AI-specific rules that are phasing in: the EU AI Act classifies AI systems used to evaluate creditworthiness or establish credit scores for individuals as high-risk, which brings stricter requirements for data governance, documentation and human oversight, though systems used to detect financial fraud are excluded from the creditworthiness category.
Organizations yet to map their AI workloads against these obligations, at home and abroad, are carrying risk they have not fully priced — and that their regulators are already starting to ask about.
Trigger 2: Operational
| Question | If yes |
|---|---|
| Does this workload have hard latency requirements? (e.g., real-time fraud scoring, trading risk calculations, AML screening) | Dedicated infrastructure can provide consistent latency |
| Is it embedded in operational processes where downtime or variability has direct business or compliance impact? (e.g., payment authorization, loan decisioning, regulatory reporting) | Onshore deployment removes dependency on external connectivity |
| Does the workload depend on low-latency connectivity between your core banking or trading systems and AI compute? | Colocation or on-premises infrastructure removes a single point of failure |
| Have you experienced latency spikes or throughput constraints that affected real-time decisioning or customer-facing SLAs? | Strong indicator that production workloads require dedicated architecture |
Some production GenAI workloads require consistent, predictable performance. In financial services, latency variability is not merely a performance concern — it is a risk management issue. A fraud detection model that introduces variable latency into payment authorization creates both customer experience degradation and potential regulatory exposure. The dependency on external cloud connectivity between core systems and AI compute is a potential point of failure that some CROs and technology risk teams have not yet fully assessed. For firms operating in the EU, the Digital Operational Resilience Act (DORA) turns that dependency into a regulatory matter, requiring financial entities to manage ICT third-party risk, including reliance on cloud providers that support critical functions.
Validate the complete dependency path between core systems and AI compute for the proposed placement. Record the external connections and third parties that remain, the effect of their failure and the tested recovery approach. Treat continuity as a demonstrated system outcome, not an assumed consequence of choosing onshore, on-premises or colocation infrastructure.
Trigger 3: Commercial
| Question | If yes |
|---|---|
| Is this workload projected to run at significant volume for three or more years? (e.g., transaction monitoring at scale, real-time credit scoring) | Commercial case for dedicated infrastructure is positive over time |
| Have you modeled total cost of ownership for dedicated vs. cloud infrastructure over a realistic horizon? | Dedicated infrastructure may offer a TCO advantage at production scale |
| Are inference costs already material and growing as you move toward production scale? | A signal that cloud economics at scale are no longer optimal |
| Would a 40–60% reduction in inference cost (achievable on dedicated infrastructure) materially improve unit economics — for example, in high-volume AML screening or customer AI applications? | Strengthens the case for dedicated infrastructure for high-volume inference |
The onshoring decision should not be treated as binary. A hybrid architecture, in which certain workloads remain in cloud environments while production inference and sensitive training workloads move to dedicated infrastructure, is the right answer for most organizations. The framework below maps trigger outcomes to placement recommendations.
For an operational-positive workload with negative regulatory and commercial triggers, placement remains an architecture decision rather than an automatic cloud default. Compare candidate environments against the documented latency, throughput and continuity requirements, then record the cost of meeting those requirements before approving placement.
Step 3: Determine placement
Each row below maps to a combination of Step 2 trigger outcomes. The framework is conservative on regulatory exposure: a single regulatory "yes" sends the workload to a sovereign architecture regardless of how the operational and commercial triggers score, because regulatory risk cannot be priced away by commercial savings. Treat the placements below as the starting point for the I&O, compliance and architecture conversation, not as the final call.
| Trigger outcome | Recommended placement | Action |
| All three triggers positive | Dedicated / onshore | Proceed to Part 2 to evaluate the AI Factory architecture |
| Regulatory trigger positive (regardless of others) | Dedicated / onshore required | Non-negotiable — proceed to Part 2: AI Factory architecture |
| Operational and commercial positive; regulatory negative | Hybrid — this workload onshore | Architect for hybrid; prioritize low-latency workloads on dedicated |
| Commercial only positive | Cloud for now | Reassess at scale; monitor TCO trajectory |
| No triggers positive | Public cloud | Retain in cloud; revisit as workload volume grows |
Part 2: The AI Factory reference architecture
An AI Factory is a purpose-built infrastructure environment designed to support the full AI production lifecycle: data ingestion and preparation, model training and fine-tuning, inference at scale, and the operational monitoring required to maintain model performance over time. For financial services firms, this is not simply an infrastructure choice — it is a governance choice. A purpose-built AI Factory provides the data isolation, access controls, audit logging, and compute determinism that model risk frameworks and financial regulators require. Unlike general-purpose cloud environments, it can be designed for throughput, latency, residency and operational continuity at production scale.
The following is one example of an AI Factory reference architecture. It brings together four integrated components designed to work as a unified system, using NVIDIA for compute and networking while the security, data platform and colocation technologies can vary based on an organization's requirements. This architecture has been deployed in production and tested in WWT's Advanced Technology Center (ATC) and AI Proving Ground.
Component | Role in the stack | Why it matters at production scale |
NVIDIA | Compute and AI network fabric | GPU architecture directly determines model training speed and inference performance. NVIDIA's CUDA™ ecosystem underpins most production AI frameworks. The NVIDIA networking platform, spanning NVIDIA Spectrum-X™ Ethernet fabric and NVIDIA Quantum InfiniBand, complete the picture with purpose-built interconnect designed to sustain GPU utilization at scale. |
An enterprise security provider | Enterprise security and reference architecture | The security layer provides the access controls, segmentation, monitoring and auditability required for regulated AI environments. A validated reference design can help integrate these controls across the architecture rather than treating security as a separate layer added after deployment. |
An AI data platform | Storage layer | Storage is frequently an overlooked constraint in AI deployments. Training runs require rapid access to large datasets, while inference at scale often requires high-throughput retrieval. The data platform must provide the throughput and access characteristics required to support these workloads at production scale. |
A colocation provider | Physical infrastructure | Carrier-neutral, sovereign data center facilities can provide the physical security, compliance certifications and geographic flexibility required for financial services AI deployments. For organizations with data residency requirements, a qualified colocation partner can help keep workloads within the required jurisdiction. High-density power and cooling infrastructure supports the compute intensity of production AI. |
For the proposed configuration, record which components and versions were tested together, which workload profiles were used and which acceptance criteria were met. Identify any substitutions from the tested configuration and the additional validation they require. A reference architecture is the baseline for this evidence, not a substitute for it.
The components in detail
NVIDIA: Compute and AI network fabric
The choice of GPU architecture matters. Not all AI workloads have the same compute profile, and the gap between a correctly specified GPU environment and an incorrectly specified one is significant in both performance and cost terms. NVIDIA's ecosystem provides the software stack, including NVIDIA CUDA, that underpins most production AI frameworks, reducing integration friction and simplifying operational management of the compute environment.
What is less commonly understood is that NVIDIA's networking portfolio is an equal part of the production AI story. The GPU is only as effective as the fabric that feeds it. NVIDIA's network components are engineered specifically for the traffic patterns AI workloads generate: high-volume, latency-sensitive, all-to-all communication between accelerators that no general-purpose network architecture was designed to carry.
| Category | Component | Primary use case |
| Switches | NVIDIA Spectrum-X Ethernet / NVIDIA Quantum InfiniBand | AI Ethernet fabric (Spectrum-X) and InfiniBand lossless switching (Quantum) — purpose-built for GPU-to-GPU all-to-all traffic |
| Adapters | NVIDIA ConnectX™-8 SuperNIC | RDMA-capable network adapter providing high-bandwidth, low-latency connectivity between accelerators and the fabric |
| Management | NVIDIA DOCA / NetQ | Unified telemetry, observability and programmability layer across the full NVIDIA network fabric |
Spectrum-X: Ethernet AI fabric
Spectrum-X is NVIDIA's Ethernet fabric designed for AI workloads in hyperscale, multi-tenant and agentic inference environments. Its adaptive routing technology dynamically reroutes traffic in microseconds when congestion is detected, eliminating the head-of-line blocking that degrades GPU utilization on conventional Ethernet. For organizations that need AI networking over standard Ethernet infrastructure rather than InfiniBand, Spectrum-X Ethernet delivers the performance consistency that production inference requires without requiring a full fabric replacement.
NVIDIA Quantum InfiniBand: InfiniBand for large-scale training
NVIDIA Quantum InfiniBand switching provides lossless, high-radix connectivity for the largest training workloads. The Quantum-X800 generation supports tens of thousands of GPUs in a single fabric domain. For organizations running distributed model training at scale, InfiniBand's combination of ultra-low latency and near-zero packet loss is the standard against which every alternative is benchmarked. NVIDIA Quantum InfiniBand is the fabric of choice for the world's largest AI training clusters.
Enterprise security
Security and compliance controls are important for AI infrastructure in regulated industries. The security layer should provide access control and segmentation across the environment, monitor AI traffic and model interactions, enforce applicable policies, and provide the auditability required by internal governance and compliance frameworks.
WWT's AI Readiness Model for Operational Resilience (ARMOR) provides a framework for evaluating security across seven domains, including governance, model protection, AI operations, software development, infrastructure, identity and data. Applied at the reference architecture stage, ARMOR can help organizations identify security requirements and ownership before an AI environment reaches production, when architectural changes become more difficult and costly. For example, organizations can establish workload segmentation, administrative access and other controls as part of the architecture rather than retrofitting them after deployment.
AI data platform and storage
AI workloads are storage-intensive in ways that many organizations do not anticipate. Training runs require rapid access to large datasets. Inference at scale often requires high-throughput retrieval. The storage layer should provide the throughput and access characteristics these workloads demand while supporting the scale and availability requirements of a production AI environment.
Colocation and physical infrastructure
A qualified colocation partner provides carrier-neutral, sovereign data center facilities with the physical access controls, compliance certifications and geographic flexibility required for financial services AI deployments at scale. For organizations with data residency requirements, a colocation partner can help keep workloads within the required jurisdiction while providing the high-density power and cooling infrastructure needed to support production AI compute.
Why integration is the critical variable
Selecting the right components is necessary. Making them perform as a system is what determines production outcomes. AI workloads are interdependent in ways that punish siloed procurement: a GPU cluster under-supported by storage throughput will not perform to specification; a high-performance storage environment on the wrong network fabric will not deliver its rated throughput.
For financial services technology leaders where data privacy, regulatory compliance, supply chain reliability and operational resilience sit at the top of every agenda, the difference between a reference architecture and a running AI Factory comes down to how integration work is sequenced and executed.
World Wide Technology operationalizes this validated stack for financial services environments. WWT validates the full system before it reaches the customer environment — including security controls, network segmentation and access management relevant to regulated workloads — compressing deployment timelines and reducing the technical risk that comes with first-of-kind implementations. Technology leaders get a single, accountable path from architecture decision to live deployment, with components arriving on site already tested and validated as a system.
Part 3: Your next move
The financial services firms that will lead in the AI Factory era are not those with the largest budgets. They are the ones making the right decisions in the right sequence — decisions that satisfy regulators, protect customers and deliver the economics that justify the investment.
Step 1: Complete your workload inventory
Use the estate mapping criteria in Part 1. If you do not have a current picture of what AI workloads you are running or planning, start here before any infrastructure decision.
Step 2: Apply the three-trigger framework
Run every candidate workload through the regulatory, operational and commercial triggers. Document the outcome for each. This becomes your placement map and your business case for infrastructure investment.
Step 3: Identify your sovereign workloads
Any workload that triggers a regulatory 'yes' requires a dedicated, sovereign architecture. In financial services, this is likely to include fraud detection models trained on customer transaction data, AML screening systems and credit decisioning models. These are non-negotiable and should be prioritized in your infrastructure roadmap regardless of commercial trigger outcomes.
Step 4: Validate architecture against your workload profile
Use the reference architecture in Part 2 as your baseline. The WWT-validated stack has been tested against production AI workloads in regulated-industry environments, including the security segmentation and access control requirements that financial services compliance teams expect. Validate it against your specific regulatory and workload requirements before committing to procurement.
Step 5: Test the full system before you deploy
The full stack does not have to be validated on site. Instead, WWT recommends validating it in a production-equivalent environment first — that environment is the WWT AI Proving Ground, with production-equivalent labs across compute, networking, storage and workload management. This is what compresses deployment timelines and prevents the costly remediation that derails most AI infrastructure programs between pilot and production.
Leading in a new era
The financial services firms that will lead in the era of the AI Factory are not necessarily those with the largest budgets or the most aggressive timelines. They are the ones making the right decisions, in the right sequence, with a clear view of the regulatory, operational and commercial factors that shape each call. In an industry where model risk, data privacy and operational resilience are non-negotiable, the leaders who engage with these questions seriously — and who bring the right partners alongside them — are the ones who will be running AI Factories while their peers are still running pilots.
This report may not be copied, reproduced, distributed, republished, downloaded, displayed, posted or transmitted in any form or by any means, including, but not limited to, electronic, mechanical, photocopying, recording, or otherwise, without the prior express written permission of WWT Research.
This report is compiled from surveys WWT Research conducts with clients and internal experts; conversations and engagements with current and prospective clients, partners and original equipment manufacturers (OEMs); and knowledge acquired through lab work in the Advanced Technology Center and real-world client project experience. WWT provides this report "AS-IS" and disclaims all warranties as to the accuracy, completeness or adequacy of the information.