How to Run Mistral Large 4 Locally (2026 Guide)

Mistral released the weights of its flagship model, Mistral Large 4 — nicknamed “Le Chonk” — on Hugging Face on 29 October 2026. It is a roughly one-trillion-parameter model you can download for free, which sounds like an invitation to run a frontier-class model at home. The invitation is real, but the hardware bill is the story: the official checkpoint is about 1.06 TB, and even the smallest usable community build needs 233 GB of memory. This guide works out exactly what each route costs, step by step, so you can decide in ten minutes whether to download anything at all.

Quick answer: Mistral Large 4 needs 233 GB to 1.06 TB of memory to run locally, so most people should use the API — the hardware maths, the licence catch and the setup steps.

What Mistral Large 4 actually is

Mistral Large 4 is a natively multimodal mixture-of-experts model with about one trillion total parameters and about 52 billion active per token — 49 billion before embeddings and output layers are counted. The details come from Mistral’s own model card on Hugging Face, which we read in full on 9 November 2026. Mistral trained it from scratch on 3,800 NVIDIA Grace Blackwell GPUs in its own data centres in Europe, and it accepts text and image input with text output.

Specification	What Mistral states
Total parameters	About 1 trillion
Active parameters per token	About 52 billion (49 billion excluding embeddings and output layers)
Architecture	Mixture of experts: 128 routed experts, 4 active per token, plus 1 shared expert
Context window	1 million tokens
Vision	Pixtral-style vision encoder, 1.6 billion parameters; text and image input
Languages	160+ languages, including every official language of the EU
Reasoning	Three efforts — low, medium, high (default high); temperature 1.0, top_p 0.95 recommended
Speed aid	Embedded DSpark speculative-decoding draft head; Mistral measures up to 1.9x throughput

Mistral positions the model as a generalist with particular strength in cybersecurity, finance and law, and says it surpasses frontier closed models on visual grounding. It is also the model behind Mistral’s own API offering, served as mistral-large-4. The weights are ungated on Hugging Face — no approval step, no waitlist — under the Mistral Ecosystem Open-Weight License, which we come back to below because it contains the one clause that matters most.

Can you run Mistral Large 4 locally?

On an ordinary laptop or desktop: no, not in any useful way. The honest answer is a memory table, because “local” means something different at this size than it does for a 7-billion-parameter model. Every figure below is the published download size or the publisher’s own memory guidance, checked 9 November 2026.

Build	Download size	What it needs	Engine
Official FP8 checkpoint	About 1.06 TB	8 data-centre GPUs (8x H200 / B200 / B300 class)	vLLM, SGLang, Transformers
Official NVFP4 checkpoint	About 610 GB	Blackwell GPUs (8x B200 / B300)	vLLM, SGLang
GGUF Q8_0 (community)	About 1.12 TB	Multi-GPU server	llama.cpp
GGUF Q4_K_M (community)	About 635 GB	High-memory workstation or server	llama.cpp
GGUF Q2_K (community)	About 381 GB	One Apple Silicon Ultra machine with 512 GB unified memory	llama.cpp
GGUF IQ2_XXS (community)	About 298 GB	512 GB unified memory	llama.cpp
Unsloth UD-IQ1_S, 1-bit	About 233 GB	256–270 GB of combined RAM and VRAM, per Unsloth	llama.cpp

Three caveats before you pick a row. First, the 1-million-token context is advertised, but the official vLLM recipe validates serving at 256K — do not plan a production workload around the full 1M until you have tested it yourself. Second, Ollama lists only a cloud tag (mistral-large-4:cloud) that bills per token; there is no downloadable Ollama tag, so Ollama is not a local route for this model. Third, the smallest builds are heavily quantised: they run, but they are not the model Mistral benchmarked. If none of these rows fits hardware you already own, skip to Route 1 — that is the right answer for almost everyone.

Route 1: Use the API or a hosted endpoint (no hardware)

This is the route Mistral itself recommends if local serving is subpar — and for nearly every reader, it is the whole guide. The model id is mistral-large-4, and Mistral’s published API pricing is $0.70 per million input tokens, $0.07 per million cached input tokens and $2.10 per million output tokens.

Try it free first. Hugging Face’s model card notes official demos run on the Mistral API with a small shared free budget per visitor, and Mistral’s own demo gallery (the “Le Chonk Gallery”) lets you test coding, vision, tool calls and JSON output without an account or a download.
Get an API key. Create a key in the Mistral console and call the model as mistral-large-4 from any OpenAI-compatible client — the same endpoint shape as the vLLM server in Route 2, so code you write against the API ports directly to a self-hosted server later.
Use a hosted provider if you need deployment without operations. Mistral lists OpenRouter, NVIDIA-hosted endpoints and a NIM container, and Microsoft Foundry, which also offers private deployments on Azure. Hugging Face Inference Providers offer the same model without any hardware of your own.
Set the reasoning effort per job. The model card recommends reasoning_effort="low" for fast chat replies, "medium" for day-to-day use and "high" (the default) for complex prompts and agentic work, with temperature 1.0 and top_p 0.95.

Worked cost: 10 million input tokens and 5 million output tokens in a month costs $7.00 + $10.50 = $17.50 through the API. Keep that number in mind when you read Route 2 — it is the baseline every self-hosting plan has to beat.

Route 2: Self-host on an 8-GPU server with vLLM

If you already run GPU servers, vLLM is Mistral’s recommended engine, and every official serving example assumes one node with eight GPUs and tensor parallelism of eight. There is no official single-GPU recipe. The model card’s own commands, verified on 9 November 2026, are:

docker pull vllm/vllm-openai:mistral-large-4

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 \
  --ipc=host \
  vllm/vllm-openai:mistral-large-4 \
  --model mistralai/Mistral-Large-4-1T-A52B \
  --tensor-parallel-size 8 \
  --tool-call-parser mistral \
  --reasoning-parser mistral \
  --enable-auto-tool-choice \
  --gpu-memory-utilization 0.85

Then point any OpenAI-compatible client at http://localhost:8000/v1. Four things to know before you run it:

Use the dedicated image. As of the October 2026 recipe, no regular vLLM release or nightly wheel includes Mistral Large 4 support, so you need the vllm/vllm-openai:mistral-large-4 image (or the ROCm image on AMD), with mistral_common[image] 1.13.0 or later.
The two parser flags are required, not optional. --tool-call-parser mistral and --reasoning-parser mistral are what separate reasoning from content and make tool use work; without them, agentic calls will misbehave.
FP8 needs 8 GPUs; NVFP4 needs Blackwell. Split across eight GPUs, FP8 needs about 128 GiB of weights per GPU, which rules out a single node of 80 GB H100s; NVFP4 targets Blackwell hardware, so on Hopper plan around FP8 and multi-node serving.
If memory runs short, shrink the context first. Lower --max-model-len and --max-num-seqs (concurrency) rather than raising --gpu-memory-utilization above 0.85 — and remember the recipe validates 256K context, not the advertised 1M.

SGLang is the alternative engine, with the same 8-GPU shape, and Transformers can serve the FP8 checkpoint via torchrun across eight GPUs. Fine-tuning is supported from day one: the model card points to TRL, with a LoRA example training the FP8 variant on a node of 8 B300 GPUs. If you are comparing open-weight flagships, our Kimi K3.1 Pro open-weights guide works through the same decision for a 2.8-trillion-parameter model with a 1.56 TB download — the hardware maths there is even steeper.

Route 3: The Mac Studio route with llama.cpp

The only realistic single-machine route is a Mac Studio with 512 GB of unified memory, running a community GGUF quant through llama.cpp. The llama.cpp support landed in pull request 31376, merged on 29 October 2026, and the first release build containing it is b11998 — use that build or newer, and do not trust older binaries with these files.

Pick your quant by memory, not by hope. Q2_K (381 GB) is the documented fit for a 512 GB machine; IQ2_XXS (298 GB) leaves more headroom; the 1-bit Unsloth build (233 GB) is the smallest working option Unsloth publishes, needing 256–270 GB of combined memory. Community MLX builds are also reported at 338 GB (2-bit), 459 GB (3-bit) and 590 GB (4-bit) — the 4-bit build needs two machines.
Download from the community repos. The official Hugging Face repository (mistralai/Mistral-Large-4-1T-A52B) holds the FP8 and NVFP4 checkpoints; the GGUF quants come from community publishers such as Unsloth and ggml-org. Verify the file hashes the publisher lists before you serve anything.
Expect slow, usable speeds — not data-centre speeds. Hugging Face measured about 90 tokens per second for prefill and 15 tokens per second for generation on a single M3 Ultra running Q2_K. That is fine for one person chatting or coding; it is not a shared service.
Do not plan a business on this route. A single machine serving one user is a hobby setup. The moment other people depend on it, you are back in Route 2 territory — and in the licence clause below.

The licence catch nobody mentions

Mistral Large 4 is genuinely free to download, modify and use commercially — until you sell access to it as a service. The weights ship under the Mistral Ecosystem Open-Weight License, which Mistral’s own model card describes as “a modified version of the MIT license, for both commercial and non-commercial use — operating a Model-as-a-Service business requires a separate commercial license with Mistral.”

“We release this model under the Mistral Ecosystem Open-Weight License: a modified version of the MIT license, for both commercial and non-commercial use — operating a Model-as-a-Service business requires a separate commercial license with Mistral.” — Mistral AI, Mistral Large 4 model card, Hugging Face

What that means in practice: building an internal tool, a product that calls the model behind your own features, or a fine-tune you deploy for your own company is covered. Standing up a hosted endpoint and charging other people to call it is not — that needs a separate commercial agreement with Mistral, and there is no revenue threshold that exempts a small operation. This is the same shape of catch as the Kimi K3.1 Pro licence, which our Kimi guide covers, but Mistral’s version has no published dollar threshold at all. Read the licence text on the model card before you spend anything on hardware; it is short enough to read in one sitting, and it is the cheapest mistake in this guide to avoid.

What the benchmarks actually say

Every benchmark number for Mistral Large 4 comes from Mistral itself — no independent replication has been published yet, so treat the figures as the vendor’s claims, not settled facts. On the Artificial Analysis Coding Agent Index, the model card reports 48%, effectively tied with the leading open-weight model GLM-5.3, while using 47% fewer tokens per task on average. On the Artificial Analysis Intelligence Index v5, it scores 26 — the highest among western open-weight models, with the leading Chinese open-weight models scoring higher on that aggregate index. Mistral also reports 41% on Terminal-Bench 4.0, up from 28% for the October preview. For context on how much vendor benchmarks can flatter a model, our SemiAnalysis GDPx piece showed the best AI in the world scoring just 12% on real work — benchmark headlines and real-world performance are different things, and the honest test for this model is your own workload, run through the free API tier first.

Our take: the 1.06 TB test

Before you download a single gigabyte, answer these five questions — if any answer is no, use the API. This is the original element of this guide: a decision test built from the numbers above, not a repeat of Mistral’s announcement.

Do you already own the hardware? Eight data-centre GPUs, or a 512 GB Mac Studio. If you would buy hardware for this model, stop — the API is cheaper by orders of magnitude.
Does your data legally have to stay on your network? This is the strongest real reason to self-host. If the answer is “we would prefer it”, the API’s data controls are usually enough.
Will steady volume keep a node busy? At the worked example above — 10M input and 5M output tokens a month — the API costs $17.50. An 8-GPU node costs thousands a month to rent. You would need roughly a hundred times that volume before self-hosting starts to look competitive, and even then only if the node stays busy around the clock.
Do you need a fine-tune you own? LoRA training on the FP8 weights is supported from day one. If you do not have a fine-tuning plan, this reason does not apply.
Have you read the licence? If anyone else will call your deployment, you need Mistral’s commercial agreement first.

Our verdict: for almost everyone reading this, the right way to “run Mistral Large 4 locally” is not to run it locally at all — it is to call the API, or a hosted endpoint, and spend the hardware budget on the product around it. The weights being open still matters: it means the model cannot be taken away, repriced or quietly changed underneath you, and it means a team with real GPU infrastructure can self-host on its own terms. But openness is not the same as cheapness, and at one trillion parameters, the gap between those two ideas is about a terabyte wide.

Frequently asked questions

Can I run Mistral Large 4 on a normal laptop?

No. The smallest published build, Unsloth’s 1-bit quant, is about 233 GB and needs 256–270 GB of combined memory. A normal laptop has 16–32 GB. The realistic single-machine route is a 512 GB Mac Studio running a GGUF quant through llama.cpp.

How much RAM do I need for Mistral Large 4?

It depends on the build: about 1.06 TB of weights for the official FP8 checkpoint on an 8-GPU server, about 610 GB for the NVFP4 checkpoint on Blackwell GPUs, 381 GB for the Q2_K community quant on a 512 GB Mac, or 233 GB for the 1-bit Unsloth build. Add room for the KV cache and runtime overhead on top of the weight files.

Is Mistral Large 4 free to use commercially?

Yes, for commercial and non-commercial use under the Mistral Ecosystem Open-Weight License — with one exception. If you operate a Model-as-a-Service business, serving the model to third parties, you need a separate commercial licence from Mistral. There is no published revenue threshold.

What is the cheapest way to try Mistral Large 4?

Use the free demo budget on Hugging Face’s demo gallery, or call the Mistral API as mistral-large-4 — $0.70 per million input tokens, $0.07 cached, $2.10 per million output tokens. There is no local Ollama tag; Ollama’s only listing is a cloud model that bills per token.

How does Mistral Large 4 compare with Kimi K3.1 Pro?

Both are open-weight flagships released within days of each other, but Mistral Large 4 is the smaller download: about 1.05 trillion total parameters and a 1.06 TB FP8 checkpoint, against Kimi’s 2.8 trillion parameters and roughly 1.56 TB download. The licences differ too — Kimi uses a custom licence with a $20M revenue condition for hosted services, while Mistral uses a modified MIT licence with a separate commercial agreement for model-as-a-service. Our Kimi K3.1 Pro guide works through that model’s hardware maths in the same way.

Sources and methodology

Primary source: Mistral AI, “Mistral Large 4 — Le Chonk” model card, Hugging Face (huggingface.co/mistralai/Mistral-Large-4-1T-A52B), read in full on 9 November 2026 — architecture, licence text, benchmarks, vLLM commands and DSpark figures all come from this card. Hardware sizes and community builds: Apidog, “Mistral Large 4 Open Weights” (apidog.com) and Axiomi, “How to Run Mistral Large 4 Locally” (axiomi.si), both checked 9 November 2026. All benchmark figures are Mistral’s own, as stated on the model card; no independent replication was found at the time of writing. Method: desk research only — no 8-GPU node or 512 GB Mac Studio was available to test on, and no hands-on run is claimed. Related reading: our methodology page (how we test AI tools). Verification date: 9 November 2026.

Govind Dheda

Govind Dheda covers AI tools, guides and prompts for OpenAIMaster — what is new, what is worth using, and how to put AI to work.