Which open-weight LLMs fit on an NVIDIA RTX 5060 Ti 16GB GPU?

An NVIDIA RTX 5060 Ti (16 GB GDDR7, 448 GB/s) comfortably runs Qwen 3.5 14B Instruct in Q4_K_M (~10.8 GB total VRAM with 16K context at ~32–42 tok/s) or Q5_K_M (~12.3 GB), Mistral Small 3.2 24B in Q4_K_M (~16.2 GB at 16K context or ~15.6 GB at 8K context), and Qwen 3.5 32B Coder in IQ3_M (~15.9 GB).

How is local LLM GPU VRAM calculated for GGUF / AWQ / FP16 models?

Total GPU VRAM equals Model Weights (Total Parameters × Bytes per Parameter for the chosen quantization) plus KV-Cache memory (Context Tokens × KV-Cache per 1K tokens × KV quantization scale) plus CUDA/Metal runtime activation overhead (~0.7 GB to 2.1 GB).

How much does 1M to 10M context cost on GPT-6.1 Terra, Mirendil Fern, and Gemini 3 Pro?

With prompt caching enabled (85%–90% discount on repeated repository prefixes), Mirendil Fern 10M costs $0.18 per 1M cached input tokens, Google Gemini 3 Pro costs $0.15 per 1M cached input tokens, and OpenAI GPT-6.1 Terra costs $0.25 per 1M cached input tokens.

Interactive Engineering Ledger · October 2026

AI Model Pricing, Context & GPU VRAM Calculator

Compare monthly API token economics (including prompt caching discounts) across frontier models, or calculate exact local GPU VRAM footprints, KV-cache overhead, and generation speeds (tok/s) on hardware from an NVIDIA RTX 5060 Ti 16GB to an 8× H200 cluster.

Hardware Preset:

GPU / System Profile

Available VRAM 16 GB

Quantization Format

Context Window

Open-Weight Model	Params (Total / Active)	Weights VRAM	KV + Runtime	Total VRAM	Hardware Fit	Est. Speed
Moonshot Kimi K3.1 Pro (3T MoE) Moonshot AI · Modified MIT Open-Weight	3,000B Total / 96B Active	1800.0 GB	2.9 GB	1802.9 GB (11268%)	REQUIRES MULTI-GPU	~1 tok/s
DeepSeek V3.2 / R2 (671B MoE) DeepSeek · MIT Open-Weight	671B Total / 37B Active	402.6 GB	2.7 GB	405.3 GB (2533%)	REQUIRES MULTI-GPU	~3 tok/s
Meta Llama 4 Scout (109B MoE) Meta AI · Llama 4 Community License	109B Total / 17B Active	65.4 GB	3.1 GB	68.5 GB (428%)	REQUIRES MULTI-GPU	~6 tok/s
Qwen 3.5 72B Instruct Alibaba Qwen · Apache 2.0	72.7B Dense	43.6 GB	4.1 GB	47.7 GB (298%)	REQUIRES MULTI-GPU	~1 tok/s
Qwen 3.5 32B Coder Alibaba Qwen · Apache 2.0	32.5B Dense	19.5 GB	3.0 GB	22.5 GB (140%)	REQUIRES MULTI-GPU	~3 tok/s
Mistral Small 3.2 (24B) Mistral AI · Apache 2.0	24B Dense	14.4 GB	2.2 GB	16.6 GB (104%)	REQUIRES MULTI-GPU	~4 tok/s
Qwen 3.5 14B Instruct (RTX 5060 Ti 16GB Sweet Spot) Alibaba Qwen · Apache 2.0	14.8B Dense	8.9 GB	2.0 GB	10.9 GB (68%)	FITS IN VRAM	~32 tok/s

Model	Tier	Max Context	Input ($/1M)	Cached ($/1M)	Output ($/1M)	Cost / 1K Reqs	Est. Monthly (30d)
OpenAI GPT-6.1 Terra OpenAI	FRONTIER API	2,000,000 tokens	$2.00	$0.25	$8.00	$32.70	$1,471.50/mo
OpenAI GPT-6.1 Sol OpenAI	FRONTIER API	2,000,000 tokens	$7.50	$1.50	$30.00	$131.40	$5,913.00/mo
Mirendil Fern (10M Context) Mirendil Labs / OpenAI Ecosystem	FRONTIER API	10,000,000 tokens	$1.40	$0.18	$5.60	$22.97	$1,033.56/mo
Anthropic Claude Duet (4.6 Sonnet) Anthropic	FRONTIER API	1,000,000 tokens	$3.00	$0.30	$15.00	$52.38	$2,357.10/mo
Google Gemini 3 Pro Google DeepMind	FRONTIER API	5,000,000 tokens	$1.25	$0.15	$5.00	$20.34	$915.30/mo
Moonshot Kimi K3.1 Pro (3T MoE) Moonshot AI	OPEN-WEIGHT	2,000,000 tokens	$0.90	$0.12	$3.20	$14.23	$640.44/mo
DeepSeek V3.2 / R2 (671B MoE) DeepSeek	OPEN-WEIGHT	256,000 tokens	$0.27	$0.07	$1.10	$5.01	$225.45/mo
Meta Llama 4 Scout (109B MoE) Meta AI	OPEN-WEIGHT	1,000,000 tokens	$0.18	$0.09	$0.55	$3.74	$168.35/mo
Qwen 3.5 72B Instruct Alibaba Qwen	OPEN-WEIGHT	131,072 tokens	$0.35	$0.15	$0.90	$6.63	$298.35/mo
Qwen 3.5 32B Coder Alibaba Qwen	OPEN-WEIGHT	131,072 tokens	$0.14	$0.05	$0.30	$2.41	$108.27/mo
Mistral Small 3.2 (24B) Mistral AI	OPEN-WEIGHT	65,536 tokens	$0.10	$0.04	$0.30	$1.91	$86.13/mo
Qwen 3.5 14B Instruct (RTX 5060 Ti 16GB Sweet Spot) Alibaba Qwen	OPEN-WEIGHT	131,072 tokens	$0.07	$0.03	$0.20	$1.36	$61.02/mo

Model	Architecture & Parameters	Context Window	API (In / Cache / Out)	Recommended Hardware	Spec & Dispatch
OpenAI GPT-6.1 Terra OpenAI · Proprietary API	Proprietary Frontier MoE Sparse MoE + Hierarchical Sparse Attention (HSA)	2,000,000 tokens	$2.00 / $0.25 / $8.00	Cloud API Only (Managed HSA Cluster)	Datasheet · Article →
OpenAI GPT-6.1 Sol OpenAI · Proprietary API	Proprietary Reasoning Tier Test-Time Compute Reasoning MoE + Verification Loop	2,000,000 tokens	$7.50 / $1.50 / $30.00	Cloud API Only	Datasheet · Article →
Mirendil Fern (10M Context) Mirendil Labs / OpenAI Ecosystem · Commercial API & Enterprise VPC	Ultra-Long-Context Specialist Hierarchical Sparse Attention (HSA) + Ring-KV Distributed Memory	10,000,000 tokens	$1.40 / $0.18 / $5.60	Cloud API or 8× H200 VPC Appliance	Datasheet · Article →
Anthropic Claude Duet (4.6 Sonnet) Anthropic · Proprietary API	Proprietary Frontier Model Constitutional Proactive Agent Architecture + Stateful Memory	1,000,000 tokens	$3.00 / $0.30 / $15.00	Cloud API Only (AWS Bedrock / Vertex / Anthropic)	Datasheet · Article →
Google Gemini 3 Pro Google DeepMind · Proprietary API	Proprietary Multimodal MoE Native Multimodal Sparse MoE (Text, Audio, Video, Code)	5,000,000 tokens	$1.25 / $0.15 / $5.00	Cloud API Only (Google TPU v6e / Trillium)	Datasheet · Article →
Moonshot Kimi K3.1 Pro (3T MoE) Moonshot AI · Modified MIT Open-Weight	3,000B Total / 96B Active Ultra-Sparse MoE (3,000B Total / 96B Active) + MLA + MuonClip	2,000,000 tokens	$0.90 / $0.12 / $3.20	2× 8×H200 Node (FP8) or 8× H200 / B200 Node (INT4/AWQ)	Datasheet · Article →
DeepSeek V3.2 / R2 (671B MoE) DeepSeek · MIT Open-Weight	671B Total / 37B Active Fine-Grained MoE (671B Total / 37B Active) + Multi-Head Latent Attention	256,000 tokens	$0.27 / $0.07 / $1.10	8× H100/H200 (FP8) or Mac Studio M4 Ultra 192GB + Offload (IQ3_M)	Datasheet
Meta Llama 4 Scout (109B MoE) Meta AI · Llama 4 Community License	109B Total / 17B Active 16-Expert MoE (109B Total / 17B Active) + iRoPE Long Context	1,000,000 tokens	$0.18 / $0.09 / $0.55	NVIDIA H100 80GB, Dual RTX 5090 (64GB), or Apple M4 Max/Ultra (64GB+)	Datasheet
Qwen 3.5 72B Instruct Alibaba Qwen · Apache 2.0	72.7B Dense Dense Transformer (72.7B) + GQA (8 KV Heads) + YaRN	131,072 tokens	$0.35 / $0.15 / $0.90	Dual RTX 4090/3090 (48GB) or Apple M4 Max (64GB)	Datasheet
Qwen 3.5 32B Coder Alibaba Qwen · Apache 2.0	32.5B Dense Dense Transformer (32.5B) + GQA + Fill-in-the-Middle (FIM)	131,072 tokens	$0.14 / $0.05 / $0.30	NVIDIA RTX 4090 (24GB) or RTX 5090 (32GB); fits 16GB at IQ3_M	Datasheet
Mistral Small 3.2 (24B) Mistral AI · Apache 2.0	24B Dense Compact Dense Transformer (24B) + Tekken Tokenizer + GQA	65,536 tokens	$0.10 / $0.04 / $0.30	NVIDIA RTX 5060 Ti (16GB) at Q4_K_M (8K ctx) or RTX 4090 (24GB)	Datasheet
Qwen 3.5 14B Instruct (RTX 5060 Ti 16GB Sweet Spot) Alibaba Qwen · Apache 2.0	14.8B Dense Dense Transformer (14.8B) + GQA + 128K Context	131,072 tokens	$0.07 / $0.03 / $0.20	NVIDIA RTX 5060 Ti (16GB) — Fits Q4_K_M (32K+ ctx) & Q5_K_M cleanly!	Datasheet

VRAM Sizing & Bandwidth Methodology

Model Weights VRAM: Computed from total parameter count multiplied by effective bytes-per-parameter including block quantization metadata (FP16 = 2.00 B/param, Q8_0 = 1.06 B/param, Q5_K_M = 0.70 B/param, Q4_K_M = 0.60 B/param, IQ3_M = 0.43 B/param).
KV-Cache & Context Scaling: Accounts for Grouped-Query Attention (GQA) on dense models and Multi-Head Latent Attention (MLA) on DeepSeek V3.2 and Kimi K3.1 Pro, plus CUDA/Metal compute buffer overhead (0.65 GB – 2.15 GB).
NVIDIA RTX 5060 Ti 16GB Recommendation: For 16GB GDDR7 cards (448 GB/s), Qwen 3.5 14B Instruct (`Q4_K_M` or `Q5_K_M`) is the daily-driver sweet spot (~10.8 GB total VRAM at 16K context, ~32–42 tok/s). For 24B–32B models on 16GB VRAM, enable quantized KV-cache (--cache-type-k q8_0) or select IQ3_M.