Cloud security has long protected data at rest with encryption and data in transit with TLS. The weak point has been data in use: while a model trains on customer records or answers a user’s prompt, the data sits decrypted in memory, where privileged software, a compromised host or an insider could, in principle, reach it. For most workloads that residual risk has been acceptable. For AI, it is becoming harder to accept. Training sets contain personal and proprietary information, prompts carry confidential business questions, and model weights are themselves valuable intellectual property.
Confidential computing closes that gap by running workloads inside hardware-based trusted execution environments (TEEs) that keep memory encrypted and isolated from the host, the hypervisor and the cloud operator, and that can prove their integrity through attestation. Until recently it covered CPUs only, which limited its use for AI. That changed when NVIDIA’s H100 introduced a confidential computing mode for GPUs, and in 2025–2026 the major clouds expanded confidential GPU options. This article explains how the technology works, what is available, where it makes sense and how to adopt it.
How Confidential Computing Works
Confidential computing combines three capabilities.
- Memory encryption and isolation. CPU technologies such as AMD SEV-SNP and Intel TDX encrypt a virtual machine’s memory with keys held in the processor and prevent the hypervisor from reading or tampering with it. The whole VM becomes a trust domain.
- Extending the boundary to the GPU. With confidential GPUs, the TEE extends from the confidential VM to the attached accelerator. In Google Cloud’s description of its A3 confidential VMs, the CPU prevents the GPU from accessing VM memory directly; instead the NVIDIA driver uses an encrypted bounce buffer in shared memory for commands and data, with encryption handled transparently by the CUDA driver and GPU firmware.
- Attestation. Before secrets are released to the workload, a verifier checks cryptographic evidence that the hardware, firmware and software stack are genuine and in the expected state. Keys for data or model weights are released only if attestation succeeds.
The result is that data and models can be processed in the cloud while remaining protected from the infrastructure that runs them, and that relying parties can verify this rather than take it on trust.
Confidential computing turns “trust the cloud operator” into “verify the hardware.” For AI, that matters most when the data, the prompts or the model belong to someone who does not fully trust the platform.
What the Clouds Offer in 2026
Offerings change quickly, so verify current regions, sizes and preview status before planning. As of late 2026, the landscape looks like this.
Microsoft Azure
Azure’s confidential GPU documentation lists the NCCadsH100v5 series: confidential VMs on AMD 4th Gen EPYC processors with SEV-SNP, attached to NVIDIA H100 GPUs, with the TEE spanning both. Microsoft notes practical constraints, including no support for resizing, a limited set of qualified guest images and customer responsibility for high availability and disaster recovery design. Azure also offers CPU-based confidential VMs and confidential containers for workloads that do not need GPUs.
Google Cloud
Google Cloud introduced confidential VMs with NVIDIA H100 GPUs on the A3 machine series and confidential VMs using Intel TDX, including C3 instances that support Intel AMX for CPU-based AI. According to a July 2026 summary of Google Cloud announcements, confidential G4 VMs with NVIDIA RTX PRO 6000 Blackwell GPUs entered preview, NVIDIA Hopper GPU support in Confidential Space became generally available, Intel Trust Authority became generally available for independent attestation in Confidential Space, and Google released open source SDKs for encrypting inference prompts and responses end to end within TEEs.
AWS
AWS approaches the problem through the Nitro System, which is designed so that operators cannot access customer instance memory, and through Nitro Enclaves for isolating highly sensitive processing within an instance. Organizations evaluating confidential GPU workloads on AWS should review the current documentation for the isolation guarantees and attestation options available on accelerated instances.
Specialist providers
A number of GPU cloud providers and confidential AI platforms now offer confidential GPU instances or private inference services built on the same hardware features. They can be useful where hyperscaler options are limited by region or GPU type, but apply the same attestation and due diligence standards.
Where Confidential AI Pays Off
Confidential computing adds cost, constraints and engineering work, so target it where the risk reduction is meaningful.
- Training and fine-tuning on sensitive data. Health records, financial transactions, customer communications and trade secrets can be processed with stronger guarantees, which can support privacy and contractual obligations.
- Private inference. Protecting prompts and responses matters for assistants used by lawyers, clinicians, bankers or executives, and for services that promise users their queries are not visible to the operator.
- Protecting model weights. Model providers deploying proprietary models into customer or third-party environments can keep weights encrypted except inside attested TEEs.
- Multi-party collaboration. Organizations that want to train or evaluate models on combined data without exposing raw data to each other, such as hospitals or banks fighting fraud, can use confidential clean rooms with verifiable policies.
- Sovereignty and regulated industries. For organizations worried about access by foreign authorities or cloud insiders, confidential computing combined with customer-controlled keys and attestation is a technical control that complements contractual and legal measures.
Where it is usually not needed
- Workloads on public or low-sensitivity data, where standard encryption and access controls are sufficient.
- Early experimentation, where confidential constraints would slow learning without a matching risk benefit.
- Cases where the main risks lie elsewhere, such as prompt injection, data leakage through model outputs or weak identity controls. Confidential computing does not solve those.
Understanding the Limits
Confidential computing is a strong control, but it is not a complete AI security strategy. Be clear about what it does not do.
- It does not protect against the application itself. If code inside the TEE logs prompts, sends data to an external service or is vulnerable, the TEE will faithfully protect a leaky application.
- It does not stop model-level attacks. Prompt injection, jailbreaks, training data extraction and membership inference remain risks and need separate controls.
- Side channels exist. Researchers continue to find side-channel attacks on TEEs. Vendors patch them, which is why keeping firmware current and verifying it through attestation matters.
- Trust shifts, it does not vanish. You still trust the hardware vendors, their firmware and the attestation service. Independent attestation services reduce dependence on the cloud provider alone.
- Performance and operations. Encrypted CPU-GPU transfers and attestation add overhead, and some platform features, such as resizing or certain migration capabilities, may be restricted. Measure the impact on your workload.
An Adoption Playbook
- Classify AI workloads by sensitivity. Identify which datasets, prompts and models are sensitive enough to justify confidential computing, based on regulation, contracts, intellectual property value and threat models.
- Define the threat model. Be explicit about whom you are protecting against: cloud insiders, compromised hosts, other tenants, foreign legal access, collaboration partners or your own administrators.
- Choose the platform. Compare confidential GPU availability by region, GPU type, price, supported images and attestation options across your clouds.
- Design key release around attestation. Store data encryption keys and model keys in a key management service or HSM that releases them only to workloads presenting valid attestation evidence matching your policy.
- Harden the workload. Minimize the software inside the TEE, remove debug access, control outbound network paths and review logging so sensitive content is not written out in plain text.
- Pilot and measure. Run a representative training or inference workload in confidential mode and compare throughput, latency, cost and operational effort with the non-confidential baseline.
- Operationalize. Integrate attestation verification into CI/CD and runtime monitoring, keep firmware and drivers current, and document the control for auditors and customers.
- Communicate assurance. If you offer AI services to customers, explain what confidential computing guarantees, how they can verify it and what it does not cover.
Architecture Patterns for Confidential AI
Most confidential AI deployments follow one of a few patterns. Choosing the right one early avoids redesign later.
Pattern 1: Confidential training enclave
Encrypted datasets are stored in object storage. A training job runs in confidential GPU VMs, proves its identity through attestation, receives data keys from a key management service and writes encrypted model artifacts back to storage. This suits fine-tuning on sensitive corporate or customer data and is the simplest pattern to start with.
Pattern 2: Confidential inference service
A model server runs inside confidential VMs or confidential containers. Clients verify the service’s attestation, or rely on a gateway that does, before sending prompts. Prompts and responses are encrypted end to end into the TEE, and logs contain only metadata. This suits assistants handling privileged or regulated information.
Pattern 3: Confidential clean room
Several parties contribute encrypted data and agree on a policy describing exactly which code may process it and what outputs may leave. A neutral, attested environment enforces the policy. This suits cross-organization model training, benchmarking and analytics where no party should see the others’ raw data.
Pattern 4: Protected model deployment
A model owner deploys encrypted weights into an environment it does not control, such as a customer’s cloud account or an edge location. The weights are decrypted only inside attested TEEs, protecting intellectual property while allowing the model to run close to the customer’s data.
Whichever pattern you choose, the common foundation is the same: hardened workload images, a key management service that enforces attestation policies, and monitoring that alerts when attestation fails or policies change.
Decision Checklist
- Is the data, prompt or model sensitive enough that exposure to infrastructure operators is an unacceptable risk?
- Do regulations, contracts or customers require stronger protection of data in use?
- Are confidential GPUs available in the regions and sizes you need, at an acceptable cost?
- Can your team operate attestation-based key release and keep the trusted stack up to date?
- Have you addressed higher-priority AI risks, such as access control, prompt injection and output leakage, first?
- Will you be able to explain and evidence the control to auditors, regulators or customers?
A Hypothetical Worked Example
A hypothetical European healthcare analytics company wants to fine-tune a language model on clinical notes from several hospital partners and then offer a summarization service to clinicians. Partners require that their data not be accessible to the company’s administrators or the cloud operator.
- The company selects confidential GPU VMs in an EU region and designs a pipeline in which each hospital encrypts its data with keys held in a key management service.
- Keys are released only to training workloads whose attestation evidence matches an agreed policy, including approved images, code hashes and firmware versions. Hospitals can review the policy and attestation logs.
- Fine-tuning runs inside the TEE; only the resulting model weights, encrypted with a separate key, leave it.
- The summarization service also runs in confidential VMs, with prompts and responses encrypted end to end and logging limited to metadata.
- Benchmarks show some overhead compared with non-confidential instances, which the company accepts given the contractual requirements. Prompt injection testing, output filtering and access controls are implemented separately.
The example is illustrative; actual designs depend on the provider’s features and regulatory advice.
Outlook
Confidential computing is moving from a niche capability toward a default for sensitive AI. Newer GPU generations support confidential modes, cloud providers are extending confidential options to more machine types and regions, and tools such as prompt encryption SDKs and independent attestation services are making the technology easier to use. Over the next few years, expect confidential inference to become a selling point for AI services handling sensitive data, confidential clean rooms to support more cross-organization AI, and regulators and customers to ask more specific questions about how data in use is protected. Organizations that build attestation-based key management and hardened workload images now will be positioned to adopt these capabilities as they mature.
Conclusion
AI raises the stakes for protecting data in use. Confidential VMs and confidential GPUs let training data, prompts and model weights stay protected from the infrastructure that processes them, with attestation providing verifiable evidence. Apply the technology where sensitivity, regulation or collaboration demands it, understand its limits, design key release around attestation and pair it with controls for application and model-level risks. Done this way, confidential computing becomes a practical foundation for trustworthy AI in the cloud.
How Hanoi Technology Can Help
Hanoi Technology helps organizations design and operate confidential computing environments for sensitive AI training and inference across Azure, Google Cloud and AWS.