Self-hosted · your GPU quoteCloudflare Workers AISelf-hosting costs less
BLACKWELL HARDWARE · NVFP4
NVFP4 inference planner
GLM · DeepSeek · Qwen / 2, 4, 8 × B200 or B300
01 — 06
Two GPU families.
2, 4, or 8 GPUs.
01 Compare serving costs
Own the GPUs, or buy the tokens.
80% means 20% of billed time produces no tokens.
API cache reuse; independent of self-hosted prefix cache.
Observed usage · Claude Code and Codex
| Session group | Calls | Input / output | Fresh / output | Cached input |
|---|
Hosted APIs USD per million tokens
| Provider | Input | Cached input | Output | Effective / 1M output |
|---|
Self-hosted rental selected GPU configuration
| Vendor | / GPU-hour | / setup-hour | 730-hour month | Effective / 1M output |
|---|
When does self-hosting cost less?
Break-even utilization for this setup
Price assumptions and exclusions
Rental costs use the modeled throughput and include idle paid time. API and rental comparisons exclude taxes, tools, cache creation/storage, router fees, network and operating costs. API precision, quality, revision and latency may differ. Published rates do not guarantee available hardware or the required interconnect.
03 Explore the hardware
Six ways to serve it.
| Configuration | Output throughput tok/s · whole setup | Fair share tok/s · request | Evidence |
|---|
∅
No exact NVFP4 measurement at this setting.
Turn off “Measured only” to explore source-backed estimates.
Fair share = total output throughput ÷ active requests. It is a derived capacity metric, not measured decode speed.
04 Make the assumptions yours
The math is on the table.
Estimates keep the source workload and apply the factors below. These are planning scenarios, not calibrated predictions.
The range is a sensitivity band around each estimate. It is not a statistical confidence interval.
05 Follow the evidence
Every number has a starting point.
How extrapolation works
Load: exact published points are preserved. Between points, output throughput is interpolated linearly against log₂ concurrency. Beyond the last observed load, throughput is held at the last observation; overload or out-of-memory failures are not modeled.
GPU count: GLM and DeepSeek use TP2/TP4 and two TP4 replicas for 8 GPUs. At the default same load per replica, two replicas serve twice the total requests and project twice the output with 100% replica efficiency. Same total requests splits fixed demand between active replicas. TP4 does not imply twice TP2 speed: the published GLM C16 rates are 1,760 and 1,773 tok/s respectively. GLM B300 multiplies each corresponding B200 anchor by an assumed hardware factor; it is not a B300 measurement. Qwen SGLang and agentic estimates use 2/4/8 TP1 replicas. Published Qwen vLLM always uses whole-setup concurrency and preserves its measured C24 results. Missing TP2 configurations assume successful fit and required offload.
Precision and runtime: DeepSeek uses same-engine native-MXFP4 measurements with an editable NVFP4 ratio. GLM on vLLM uses SGLang NVFP4 measurements with an editable engine ratio. A ratio of 1 means assumed parity, not measured equivalence.
Hardware: Published Qwen/SGLang and DeepSeek/SGLang profiles have separate B200/B300 baselines. Agentic sources have their own GPU baseline; transfers use the editable B300/B200 ratio or its inverse. Nominal HBM excludes no runtime reservations and does not prove memory fit.
128K / 256K agents: these mean 128,000 / 256,000 prompt tokens plus the selected output. Stage estimates combine source decode rates with uncached-input prefill work, serialized without overlap. Prefill rate is held constant; decode is multiplied by the chosen retention for each context doubling beyond the reference length. Cache eviction, memory limits, queueing and tool waits are unmodeled. All such results are conditional estimates, including isolated prefill time.
Boundaries: GLM anchors span community NVFP4 checkpoints and tuning phases. Qwen long-input decode prompt lengths are unspecified. DeepSeek stage proxies come from native MXFP4 TPOT/TTFT, with unverified NVFP4 transfer. Source endpoints and pricing revisions can differ. No exact 128K/256K matrix or production memory-fit guarantee is claimed.