Two years ago, the hard question in AI infrastructure was whether you could get GPUs at all. In late 2026 the question has changed: there are more places to buy accelerated compute than ever, and the options differ sharply in price, contract terms, compliance posture and risk. Hyperscalers sell reserved and on-demand GPU instances alongside their own accelerators. A fast-growing class of specialist GPU clouds, usually called neoclouds, sells large clusters on multi-year terms. Colocation providers offer power and cooling for customers who want to own hardware, and some enterprises are building on-premises AI clusters again.
This guide is for infrastructure, platform and finance leaders who must decide where their AI training and inference capacity will come from through 2030. It explains how the market has shifted, compares the four main sourcing models, offers decision criteria and a scoring approach, and closes with a hypothetical example and an outlook.
How the GPU Market Shifted in 2025–2026
Neoclouds moved from niche to mainstream remarkably quickly. According to Synergy Research Group figures reported by Trending Topics in July 2026, neocloud revenue exceeded 25 billion dollars in 2025, and Synergy projects the segment could approach 400 billion dollars by 2031. ABI Research, in a December 2025 analysis, forecasts that neoclouds will earn around 250 billion dollars from GPU-as-a-Service by 2030, and that inference, not training, will make up the large majority of that market by the end of the decade.
The same sources also highlight the risks. ABI Research notes heavy dependence on a small number of very large customers, citing Microsoft’s share of CoreWeave’s 2024 revenue, and warns that neoclouds without direct enterprise relationships could be squeezed into a commodity role. The Trending Topics analysis points to high depreciation relative to revenue and to substantial GPU-backed debt maturing between 2026 and 2028. Meanwhile, hyperscalers have expanded short-term reservation products, such as Amazon EC2 Capacity Blocks for ML, which let customers reserve GPU clusters for fixed windows up to eight weeks ahead.
For buyers, the result is a market with real choice and real counterparty risk. That combination calls for a portfolio approach rather than a single-vendor bet.
Treat GPU capacity like a portfolio, not a purchase: match each workload’s shape and risk to the sourcing model that fits it, and keep the ability to move.
The Four Sourcing Models
1. Hyperscaler GPU instances
AWS, Microsoft Azure and Google Cloud offer GPU instances on demand, through reservations and commitment discounts, and through scheduled products such as Capacity Blocks. They also offer their own accelerators, such as AWS Trainium and Google TPUs, which can be cheaper for workloads that support them.
- Strengths: integration with your existing identity, networking, data, security and compliance controls; managed AI platforms; global regions; mature enterprise contracts.
- Weaknesses: on-demand GPU pricing is typically the most expensive option; the newest GPUs can be hard to obtain at scale without commitments; data egress fees apply if data moves elsewhere.
- Best for: inference close to production applications and data, regulated workloads, bursty experimentation and teams that value managed services over raw unit price.
2. Neoclouds
Specialist GPU providers such as CoreWeave, Nebius, Lambda, Crusoe, Together AI and many regional players focus on accelerated compute. They tend to offer large, tightly networked clusters, Kubernetes- or Slurm-based environments, and increasingly managed inference and training software.
- Strengths: often lower unit prices for large clusters; fast access to recent GPU generations; engineering focus on high-performance networking and storage for AI.
- Weaknesses: fewer adjacent services; contracts may require multi-year commitments and prepayments; counterparty and financing risk; data must move to and from the provider; regional and compliance coverage varies.
- Best for: large training runs and fine-tuning, high-volume batch inference, and teams with strong in-house ML platform skills.
3. Colocation with owned or leased hardware
You buy or lease servers and place them in a colocation facility that provides power, cooling and connectivity. Many facilities now offer liquid cooling and high-density racks designed for AI.
- Strengths: lowest long-run unit cost if utilization is high; full control over hardware and software; predictable capacity once installed.
- Weaknesses: large up-front capital or lease commitments; long lead times for hardware, power and cooling; responsibility for operations, spares and refresh; hardware depreciates as new GPU generations arrive.
- Best for: organizations with steady, high-utilization workloads and the skills to run clusters.
4. On-premises AI clusters
Some enterprises are building AI capacity inside their own data centers, often for sovereignty, data gravity or latency reasons. Vendor-validated reference architectures and “AI factory” bundles make this more accessible than it once was, but power and cooling remain the constraint in many older facilities.
- Strengths: maximum control over data location and access; reuse of existing facilities and security operations; no egress costs for on-premises data.
- Weaknesses: facility upgrades may be required; capacity cannot burst; staffing and lifecycle costs are often underestimated.
- Best for: sensitive data that must stay on site, steady inference near operational systems, and organizations with existing data center investments.
Decision Criteria That Actually Matter
Unit price per GPU-hour is the most visible number and the most misleading one on its own. Use a broader set of criteria and weight them for each workload.
- Workload shape. Is it a one-off training run, recurring fine-tuning, steady inference or spiky inference? Steady loads favor commitments or owned hardware; spiky loads favor on-demand or scheduled reservations.
- Utilization you can really achieve. Owned or committed capacity is only cheap if it is busy. Be honest about scheduling maturity and pipeline readiness.
- Data gravity and egress. Where does the training data live, and where must inference results go? Moving large datasets repeatedly can erase a unit-price advantage.
- Compliance and sovereignty. Data residency, sector rules and contractual audit rights may rule some providers out, or require specific regions.
- Counterparty risk. Assess financial strength, customer concentration and contractual remedies. Ask what happens to your capacity if the provider is acquired, restructures or misses delivery dates.
- Contract flexibility. Term length, prepayment, ability to swap GPU generations, termination and step-down rights, and service credits.
- Software and operations. Kubernetes or Slurm support, storage performance, observability, security integration and how much of the stack your team must run.
- Time to capacity. Lead times for hardware, power and cooling for owned deployments versus provisioning times for cloud options.
A simple scoring approach
For each major workload, score each sourcing model from one to five on the criteria above, multiply by weights agreed with finance, security and engineering, and compare total landed cost over three years under at least two utilization scenarios. Include egress, storage, networking, staff time, support, power and refresh, not only compute. The point is not precision; it is to make trade-offs explicit and repeatable.
Building a Blended Capacity Portfolio
Most organizations with meaningful AI workloads will end up using more than one model. A common pattern looks like this:
- Baseline inference on committed hyperscaler capacity or on-premises clusters, close to applications and data.
- Large training and fine-tuning runs on neocloud clusters or scheduled hyperscaler reservations, sized to the run and released afterwards.
- Burst and experimentation on on-demand cloud capacity, with strict budgets and automatic shutdown.
- Steady, high-utilization workloads moved to owned or leased hardware once demand is proven and predictable.
Portability is what makes this portfolio work. Containerize training and serving stacks, keep model artifacts and datasets in open formats and object storage you control, use infrastructure as code for every environment, and standardize on an orchestration layer that runs in all of them. Kubernetes with a common set of operators is the usual choice, though Slurm remains common for large training jobs.
Governing the portfolio
A blended portfolio needs an owner and a rhythm, or it degrades into a collection of unrelated contracts. Assign a capacity owner, usually in the platform or infrastructure team, who works with FinOps and procurement. Hold a monthly capacity review that compares forecast demand by workload with committed capacity across all providers, flags idle reservations and upcoming expiries, and decides where new demand should land. Track a small set of metrics consistently across providers:
- Allocated versus actually used accelerator hours for every reservation and owned cluster.
- Cost per unit of useful work, such as cost per thousand inference requests or per completed fine-tuning run.
- Queue time for training jobs, which reveals when capacity is genuinely short rather than poorly scheduled.
- Time to move a workload between providers, tested periodically so that portability is proven, not assumed.
Tie capacity decisions to the model roadmap. When a team adopts a smaller model or a more efficient serving engine, the capacity owner should see it before the next commitment is signed, not after.
Contract and Risk Checklist for Neocloud and Large Reservation Deals
- Delivery commitments: dates, penalties for late delivery and what happens if hardware arrives in a different configuration.
- Technology refresh: rights to move to newer GPU generations and on what terms.
- Data protection: encryption, key management, deletion guarantees and audit rights, plus where data and backups physically reside.
- Security: tenant isolation model, vulnerability management, incident notification timelines and alignment with your regulatory obligations, such as DORA or NIS2 if applicable.
- Exit: data export procedures, transition assistance and termination rights on change of control or financial distress.
- Financial health: review published financial statements, customer concentration and financing arrangements, and consider step-in or escrow arrangements for critical capacity.
- Support: response times, named engineering contacts and escalation paths for cluster failures during long training runs.
A Hypothetical Worked Example
A hypothetical European insurer plans three AI workloads for 2027: a customer-service assistant serving steady traffic, quarterly fine-tuning of a domain model on claims documents, and exploratory research by a small data science team.
- Assistant inference. Data residency rules and integration with existing systems point to a hyperscaler region in the EU with committed capacity sized to baseline demand, plus on-demand headroom for peaks.
- Quarterly fine-tuning. Each run needs a large cluster for about two weeks. The team compares scheduled hyperscaler reservations with an EU-based neocloud offering. The neocloud is cheaper per GPU-hour, but data transfer, security review and contract terms narrow the gap. The insurer chooses the neocloud for two runs as a pilot, with a fallback plan to use scheduled hyperscaler capacity.
- Research. The team uses on-demand instances with per-project budgets and automatic shutdown policies.
- Portability. All three workloads run on the same container images and orchestration tooling, so moving a workload between providers is an operational task rather than a rebuild.
The example is illustrative. The key design choice is that no single provider is a single point of failure for the insurer’s AI roadmap.
Pitfalls to Avoid
- Buying for peak. Long commitments sized for a hoped-for peak become stranded cost if demand or model efficiency changes.
- Ignoring model efficiency gains. Smaller, distilled and quantized models can cut inference compute dramatically. Revisit capacity plans whenever your model strategy changes.
- Underestimating operations. Owned clusters need hardware maintenance, firmware management, scheduling and failure handling. Budget for people, not only machines.
- Single-sourcing on price. The cheapest provider today may not be the most reliable through 2030. Diversify critical capacity.
- Forgetting the CPU and storage around the GPUs. Data loading, preprocessing and checkpoint storage can bottleneck expensive accelerators.
Outlook: 2027–2030
Several trends are likely to shape GPU sourcing through the end of the decade. Inference will overtake training as the dominant demand, as ABI Research projects, which favors capacity close to users and data. Regional and sovereign AI capacity will grow, particularly outside North America. Accelerator diversity will increase as hyperscaler chips, AMD GPUs and challenger silicon gain software support, rewarding teams that keep their stacks portable. Consolidation among neoclouds is plausible given their capital intensity, which makes contract protections and exit plans more important. And power availability will remain a hard limit on how fast new capacity can come online anywhere.
Conclusion
There is no single right answer to where AI compute should come from. The right answer is a deliberate portfolio: hyperscaler capacity where integration and compliance matter most, neoclouds or scheduled reservations for large and time-bound runs, owned hardware only where utilization is proven, and portable stacks that let you rebalance as prices, models and providers change. Organizations that make these decisions explicitly, with finance and security at the table, will spend less and carry less risk than those that simply buy wherever capacity is available this quarter.
How Hanoi Technology Can Help
Hanoi Technology helps organizations design portable AI infrastructure, compare GPU sourcing options and operate blended capacity across cloud and on-premises environments.