Anthropic reports progress on provable inference for large language models
News
Anthropic says FlashProve can generate proofs for Llama 4 Maverick at about 110 tokens per second on one H100 GPU.
Anthropic said it has made progress on provable inference for large language models, releasing a research prototype called FlashProve that is designed to cryptographically prove a response came from a specified model. In its Nov. 2, 2026 announcement, the company said the system can generate proofs for Llama 4 Maverick at around 110 tokens per second on a single H100 GPU, with a proof size of about 19 MB per 1,024-token request.
What Anthropic says FlashProve does
According to Anthropic, provable inference is intended to address a basic verification problem: users today cannot reliably tell from output alone whether a response came from the model they intended to query. The company said the approach could support consumer applications, internal security use cases and efforts to verify compliance with AI pacing requirements. Anthropic framed FlashProve as an open-source prototype built around zkSNARKs, a type of zero-knowledge succinct argument that can hide model weights while allowing verification of an inference result.
Anthropic said the system was tested on a large model, Llama 4 Maverick, described in the post as a 400-billion-parameter mixture-of-experts model using 4-bit quantized weights and integer arithmetic in prefill. The company said proving with FlashProve takes about 70 to 95 times the GPU time of running the model itself on vLLM, a common open-source inference library. The announcement did not provide an independent benchmark or say that the results had been externally reproduced.
Artifacts and verification claims
Anthropic said it is releasing five artifacts: the research post, a technical report, a protocol description, executable code that includes a prover and two verifiers, and Lean proofs for top-level mathematical statements, along with an interactive visualizer for auditing the theorems. The company said the code, Lean proofs and documentation are being released under the Apache 2.0 license.
The post also said the work is a progress update rather than a completed research agenda. Anthropic said it is using top-level auditable statements in Lean to support the claimed security guarantees, and noted that the commitment to the model and the inference proofs are intended to hide the model’s weights. The announcement did not say that the system hides the architecture of the model.
Context and limitations
Provable inference is aimed at computational integrity, not at determining whether a model’s answer is accurate, safe or unbiased. Anthropic’s post says the approach is relevant for independent evaluators who need to know that the model being tested is the one being served, for customers using a third-party routing service, and for cases where an attacker might tamper with the token stream.
The company’s reported throughput and proof size are specific to the benchmark it described, and benchmark comparisons across prior systems require care because model sizes, hardware, numerical representations, batching and proof coverage differ. Anthropic did not disclose a date for a next verified product release or a commercial launch in the announcement.
