How fast will a model run on your hardware?
Choose a model and your hardware. See whether it fits and how fast it could run.
Free · runs entirely in your browser · nothing to install
Or try an example
Estimated speed. Select a setup to explore it.
1
Model What will run
2
Hardware Where it runs
3
Run mode How it runs
4
Prediction Speed and why
Model Llama3 8B · 4-bit
Start with a model and weight precision.
Model
Custom
Llama 3 8B
Llama 3 70B
Llama 3.3 70B
Llama 3.2 3B
Llama 3.2 1B
Llama 3.2 90B Vision
Llama 3.2 11B Vision
DeepSeek R1 (671B MoE)
DeepSeek V3 (671B)
DeepSeek R1 Distill 70B
DeepSeek R1 Distill 32B
DeepSeek R1 Distill 14B
DeepSeek R1 Distill Qwen 7B
DeepSeek R1 Distill 1.5B
Qwen 2.5 72B
Qwen 2.5 32B
Qwen 2.5 14B
Qwen 2.5 7B
Qwen 2.5 3B
Qwen 2.5 Coder 32B
Mistral 7B
Mistral Large 2 (123B)
Mistral Nemo 12B
Mixtral 8x7B
Mixtral 8x22B
GLM-4 9B
Command R+ 104B
Gemma 3 27B
Phi-3 14B
Phi-3 3.8B
SDXL Base (3.5B)
Flux.1 Dev (12B)
SD3 Medium (2B)
Large Model (400B+)
Very Large Model (1T+)
Search or import a model
Quantization
4-bit (recommended)
5-bit · Q5_K
6-bit · Q6_K
3-bit · Q3 / IQ3
2-bit · Q2 / IQ2 (quality risk)
INT8 / Q8
FP8
FP16
BF16
FP32
Checkpoint & memory
Exact format optional
Published checkpoint sizes
Use the format above
Lookup table placement
Device memory · include the whole checkpoint
Host RAM · sparse lookup offload
Requires lookup offload support in the chosen runtime. Experts are planned separately. Unified-memory devices still need space for both.
Sizes the weights from the real file format instead of the family average: Q4_K_M is 4.9 bits per weight, IQ4_XS 4.4, NVFP4 4.5.
Llama3 8b @ Q4
Model architecture & advanced settings
Distribution & optimization
Distribution across devices
AUTO — pick the best strategy
Pipeline (layers split across devices)
Tensor (each layer split across devices)
Data (replica per device, separate requests)
Expert (MoE experts split across devices)
Sequence parallelism
Context parallelism
Hybrid TP+PP
Hybrid TP+DP
Inference optimization
Standard inference
Speculative decoding
Continuous batching
PagedAttention (vLLM)
Flash Attention
EXO phase split (prefill/decode)
KV cache compression
None (fp16 KV)
8-bit KV (llama.cpp q8_0, vLLM fp8)
4-bit KV (llama.cpp q4_0)
TurboQuant 3.5-bit
TurboQuant 2.5-bit
Applies to the KV cache only. Halving KV bytes helps most at long context and when many users share the device.
Runtime
Runtime framework
AUTO — match hardware
llama.cpp
MLX / MLX-LM
Ollama
vLLM
SGLang
TensorRT-LLM
EXO / MLX Distributed
Specialized engines & measured results
These engines use different placement and kernels. Follow their hardware and model requirements; the stock runtime estimate above does not predict their speed.
Advanced execution assumptions
Optional supplied values replace only the matching estimate. They are treated as unverified assumptions unless you independently measured and documented them; the planner never promotes them to benchmark evidence.
Bytes/pass can describe mixed precision across attention, experts, heads, compressors, and routers. Resident weights controls capacity separately; sustained bandwidth is set on each device. Values inferred from an observed tok/s result are circular and should not be presented as validation.
Scenarios
Load a scenario preset
— none —
RTX 5090 + DDR5 Overflow (36GB model)
RTX 5090 x16 + RTX 5060 Ti x1 (slow PCIe)
RTX 5090 x8 + RTX 5060 Ti x8 (balanced)
RTX 5090 x16 + GPU via Oculink (x4)
2x RTX 5090 x16 (theoretical)
RTX 4090 + DDR5 Overflow
RTX 4090 + RTX 4080 (both x8)
Mac M4 Max (128GB)
Mac M3 Ultra (512GB)
4x Mac M3 Ultra (Thunderbolt 5)
4x Mac M3 Ultra (TB5 RDMA)
Single H100
8x H100 (NVLink)
4x Intel Arc Pro B70 + MiniMax M2.7 (Q4 plan)
4x Intel Arc Pro B70 + DeepSeek V4 REAP (unverified plan)
Cost assumptions (power pricing)
Model internals (override preset)
float32
bfloat16
float16
int8
fp8
q4
Adjust context and users alongside your estimate.
Choose hardware
Hardware NVIDIA RTX 5090 · 32 GB
Choose your hardware Add the devices that will run this model.
Devices in this plan Click a device to edit it
Back
Runtime options
Show estimate →
Your estimate
Planning estimate
Your workload Change it to see the trade-offs
Memory uses the combined context window; timing is split into prompt processing and decode.
Concurrent users
1 user
2 users
4 users
8 users
16 users
32 users
64 users
128 users
Try
Quick chat
Long answer
Long document
Execution How this runs on your hardware
Why: the model execution map
How the model maps onto your devices — what each one stores, where they communicate, and where the milliseconds go.
Physical topology
Device connections
Device topology diagram; see the model execution map for the same information as text.
Status
Bar segments: M=Memory · C=Compute · L=Local BW · N=Network BW · click a node to edit
Per-device utilization chart
Memory, compute, local bandwidth, and network
Per-device utilization chart; the same percentages appear as text in the per-device breakdown.
Model intelligence
Find a model, then plan the hardware.
Compare base models, fine-tunes, quantizations, active parameters, and measured performance without digging through a giant select menu.
Search model or organization Family All families Architecture Dense + MoE MoE only Dense only Size Any size Under 10B 10–70B 70B+ Sort Most measured Fastest observed Name Parameter size
Loading evidence…
Leave-one-out validation
Measured versus predicted
How the targets are built
Model + comparable real runs
The physical roofline is only a sanity bound. Projected real uses the median correction from similar runs; the optimized target applies strong-run benchmark efficiency to a roofline that still retains modeled communication latency.
Model / setup Observed Optimized Projected Error
Same author, kept apart from the gold set and never used to calibrate the engine. Stock and baseline rows are compared with the projection for that setup; tuned rows show how much of the optimized target a tuned software stack actually reached. Every row links to its measurement.
Run / setup Stack Measured Projected Optimized Comparison
Published benchmarks for reference
click to expand
Real-world throughput plus official model-card task scores from vendor, community, and research sources. Throughput rows are used for configuration alignment; task-score rows are for comparison only.
Result interpreter
Compare a measured run with the plan.
Paste a result or enter its speed to compare it with the projected result, an attainable optimized target, and the changes most likely to help.
Waiting for a measurement
Your active plan supplies the model and hardware.
We will compare the entered result with the primary projection and an evidence-capped optimized target.
View the normalized result JSON
Portable result One payload, with its assumptions made explicit. Copy this normalized result into a report or a future API call.
{
"observedTokS": 32.2,
"configuration": "active-plan",
"return": ["optimizedTokS", "physicalRooflineTokS", "expectedTokS", "headroom", "recommendations"]
}
