Mistral Small 3.2 (24B)
Low-latency 24B Apache-2.0 model engineered for fast function calling, structured JSON extraction, and tight 16GB–24GB consumer GPU deployments.
Provider
Mistral AI (Apache 2.0)
Architecture
Compact Dense Transformer (24B) + Tekken Tokenizer + GQA
Parameters
24B Dense
Context Window
65,536 tokens (Max Output: 16,384 tokens)
API Pricing
Input: $0.10 / 1M · Cached: $0.04 / 1M · Output: $0.30 / 1M
VRAM (16K Ctx)
Q4_K_M: 16.62 GB · Q8_0: 27.79 GB · FP16: 51.5 GB
Hardware Target
NVIDIA RTX 5060 Ti (16GB) at Q4_K_M (8K ctx) or RTX 4090 (24GB)
Interactive Token Cost Estimator (Mistral Small 3.2 (24B))
GPU Hardware Compatibility Matrix (Mistral Small 3.2 (24B) at 16K Context)
| GPU / Hardware Profile | VRAM | Q4_K_M (4.8-bit) | Q8_0 (8-bit) | Est. Speed (Q4) |
|---|---|---|---|---|
| RTX 5060 Ti 16GB 448 GB/s | 16 GB | REQUIRES MULTI-GPU (16.62 GB) | REQUIRES MULTI-GPU (27.79 GB) | ~4 tok/s |
| RTX 4090 24GB 1008 GB/s | 24 GB | FITS IN VRAM (16.62 GB) | REQUIRES MULTI-GPU (27.79 GB) | ~45 tok/s |
| RTX 5090 32GB 1792 GB/s | 32 GB | FITS IN VRAM (16.62 GB) | FITS IN VRAM (27.79 GB) | ~80 tok/s |
| M4 Max 64GB 546 GB/s | 52 GB | FITS IN VRAM (16.62 GB) | FITS IN VRAM (27.79 GB) | ~24 tok/s |
| 2× 24GB (48GB) 920 GB/s | 48 GB | FITS IN VRAM (16.62 GB) | FITS IN VRAM (27.79 GB) | ~41 tok/s |
| H100 80GB 3350 GB/s | 80 GB | FITS IN VRAM (16.62 GB) | FITS IN VRAM (27.79 GB) | ~150 tok/s |
| M4 Ultra 192GB 819 GB/s | 164 GB | FITS IN VRAM (16.62 GB) | FITS IN VRAM (27.79 GB) | ~37 tok/s |
| 8× H200 (1,128GB) 4800 GB/s | 1128 GB | FITS IN VRAM (16.62 GB) | FITS IN VRAM (27.79 GB) | ~215 tok/s |
Architectural Strengths & Target Workloads
- 24B Q4_K_M weights (~14.4 GB) fit inside a 16GB RTX 5060 Ti with quantized KV-cache (`-ctk q8_0`)
- Ultra-low layer count delivers rapid token generation and snappy agentic tool calling
- Apache 2.0 licensed with zero usage restrictions
Primary Engineering Workloads: 16GB–24GB Local Workstation LLM · Low-Latency Function Calling · Structured Log Parsing.
Frequently Asked Questions (Mistral Small 3.2 (24B))
How much does Mistral Small 3.2 (24B) cost per 1M tokens?
Mistral Small 3.2 (24B) is priced at $0.10 per 1M uncached input tokens, $0.04 per 1M cached input tokens, and $0.30 per 1M output tokens (Mistral La Plateforme API or Local 16GB–24GB GPU).
What is the context window of Mistral Small 3.2 (24B)?
It supports up to 65,536 tokens (65,536 tokens) and a maximum generation output of 16,384 tokens.