← All evidence
P-005 · GPU Saturation
22× stock TensorRT-LLM on the same silicon.
Production
Measured
Verified
Inference Fabric saturates NVIDIA silicon by fixing the supply chain around the kernels. Measured on production workloads on a single RTX 5090: 96.9% mean SM utilization, 250K peak tokens per second, 0.42% padding waste.
Production telemetry, captured during sustained saturation runs on the development host. Numbers are published in the site's canonical fact-sheet (/llms.txt) and cross-referenced in the Inference Fabric page hero + evidence section.
SIR (Saturated Inference Runtime) wraps TensorRT-LLM with shape-pure batching, a zero-allocation hot path, eight-level backpressure, and class-keyed KV reuse. Producers fire and forget into the buffer; the GPU only ever sees clean, homogeneous batches.
GPU SATURATION · PRODUCTION TELEMETRY 96.9% mean SM utilization (100% peak) 250K tokens/sec peak, single RTX 5090 160K+ tokens/sec sustained, single GPU 22× throughput vs stock TensorRT-LLM ~95% lower marginal cost per million tokens at saturation 99.9% KV cache hit rate 0 shape switches 0 XID errors 0.42% padding waste (industry typical: 15–40%) Hardware path: NVIDIA Blackwell (SM 120) · CUDA 13 · TensorRT-LLM · FP8/FP4