Confidential computing can enable better frontier AI auditing
- 4 days ago
- 4 min read
The first public measurement of confidential-computing overhead on an instrumented evaluation of a frontier AI model shows the cost is manageable and the remaining gap is a solvable engineering problem.
Author: Alejandro Tlaie Boria
Executive SummaryWe present the first public measurement of confidential-computing (CC) overhead on an instrumented evaluation of a frontier AI model. Running five interpretability workloads on a 744-billion-parameter model inside hardware-attested enclaves, we report three main results:
As we laid out in our May 2026 white paper, these were the three main empirical uncertainties we identified. Therefore, governed deep evaluations of frontier AI models are technically unblocked today, and the next natural step is a follow-on pilot with an evaluator body, a governance partner, and a frontier lab. |
Safety claims that black-box testing cannot reach
Regulators and evaluators are increasingly asked to verify safety claims that black-box evaluation cannot settle. Does a model harbour hidden capabilities? Are training-time backdoors present? Did unlearning of dangerous knowledge actually work? Answering these questions requires visibility into model internals: activations, expert-routing decisions, gradients. But granting that visibility exposes model providers to trade-secret risk, since the same access channels could leak model weights.
In our previous white paper (May 2026), we proposed a way through this impasse. Evaluators run interpretability tools against the model inside a trusted execution environment (TEE) whose hardware attestation hides the weights from everyone, including the infrastructure host, while a governed export pipeline releases only bounded, signed evidence bundles. We call this de facto glass-box evaluation: analytical access to model internals without raw weight access.
The white paper left three empirical prerequisites open: (1) Is CC overhead bounded on the workloads that matter? (2) Is a constrained export pipeline operationally viable? (3) Does per-request governance work correctly?
What we measured
All measurements were collected on a production CC platform (Tinfoil Containers, Intel TDX, 8× NVIDIA H200 GPUs) running GLM-5.1, a frontier open-weights mixture-of-experts model with 744 billion parameters. The five workload conditions span the interpretability techniques a credible deep evaluation requires: activation capture, representation steering, expert-routing analysis, and input gradients.
Finding I: Production CC overhead is +5–7%. On stock inference, confidential computing adds 5–7% to wall time, roughly 40–80 ms per request. Under interpretability instrumentation, the overhead rises to +33–38%. A targeted ablation shows this is almost entirely an instrumentation artefact: the tooling used in this study (vllm-lens, developed by the UK AI Security Institute) disables a key GPU optimisation (CUDA Graphs) to attach its measurement hooks. Of the total gap, 27 percentage points come from that missing optimisation; only 5–7% is intrinsic CC cost. Restoring the optimisation under instrumentation is a well-defined engineering task for the follow-on pilot.
Finding II: Governed export saves time rather than costing it. Instead of shipping 4 MB of raw activation data over the network, the evaluation pipeline compresses each capture into a 60 KB signed evidence bundle inside the enclave. The compression costs 238 ms of compute but eliminates over a second of network transfer, for a net saving of 879 ms per request. Aditionally, the bundle size stays bounded at 60 KB even as raw payloads grow six-fold.
Finding III: Governance is invisible. A governance layer built on PySyft (developed by OpenMined) authenticates evaluators, enforces per-evaluator export budgets, and logs every request. The total cost is roughly 1 millisecond per request. To put it in relative terms, that is 10,000x faster than the model forward pass it wraps.
Why this matters
For regulators and evaluation bodies: the assumed performance trade-off of confidential computing has been the standing objection to mandating deeper-than-black-box access. A measured +5–7% production overhead removes that objection. The evaluation-access provisions in the EU AI Act's GPAI Code of Practice (Safety and Security chapter), the UK AISI's pre-deployment testing efforts, and the international network of AI safety institutes all presuppose infrastructure of this kind but have not yet grounded it empirically. In the case of the EU Code of Practice, this method could become 'State of the Art' and eventually an industry standard for frontier AI risk management.
For frontier AI providers: the architecture shows that granting evaluators analytical access need not mean exposing weights. Every export is bounded, aggregated, signed, and budget-capped; raw tensors never cross the enclave boundary. The overhead figures allow providers to cost a deployment precisely rather than estimate it.
For evaluators and interpretability researchers: the brief documents which techniques run inside the enclave today, at what cost, and with a concrete optimisation path for the instrumentation layer that would bring instrumented-mode overhead close to the production floor.
From measurement to pilot
In this technical brief, governance at production scale is demonstrated at single-session level and some features (multi-evaluator concurrency, cross-session export budgets) are deferred to a pilot. The measurement harness is released alongside the brief for independent replication.
Organisations working on frontier model evaluation, confidential computing for AI, or AI governance infrastructure are welcome to reach out at info@pourdemain.ngo.
Read the full technical brief:
Read the companion white paper: Securing Deep Frontier AI Evaluations.
