Edge0-145B is the most ambitious “big model, small hardware” release of November 2026: a 145-billion-parameter agent model whose weights you can download from Hugging Face and run on a single consumer graphics card. The headline is real — but it comes with two catches that the announcement does not lead with. First, “one GPU” still means roughly 61 GB of memory in total once system RAM is counted. Second, the weights are open-weight, not open-source, and the licence draws a hard line between using the model and selling with it. This guide walks through the hardware, the setup, and the licence check, in that order.

Quick answer: Edge0-145B is a 145B agent model that runs on one consumer GPU — but it needs 61 GB of memory and a licence check first. Hardware, setup and limits inside.

What is Edge0-145B?

Edge0 released Edge0-145B and a smaller sibling, Edge0-120B, on 12 November 2026, with the weights on Hugging Face and its inference framework (edge0-infer) and deployment tools (edge0-strata) on GitHub. It is a latent sparse mixture-of-experts model: 145 billion total parameters, but only about 10.12 billion active per token, across 48 layers with 512 experts per layer — 10 routed experts plus one shared expert per token. It offers up to 256K of context and was trained natively at 4 bits, which is the detail that makes the single-GPU claim possible at all.

The model is derived from Qwen3.8-Flash-Next, Alibaba’s model — a fact the Hugging Face model card states openly. But it is not a relabelled Qwen model and not a re-quantisation of one. Edge0 rebuilt the expert layer in all 48 layers, replacing the routed mixture-of-experts with a latent version: each token is projected down to a narrow latent width, the experts run there, and the result is projected back. The model was then post-trained on Edge0’s own data. That rebuild is why the headline numbers look the way they do: routed expert parameters fall from 120.8 billion in the parent model to 84.6 billion, and expert parameters per token fall from 2.36 billion to 1.65 billion.

Two builds are available. The Q3_K build is 40.81 GiB and the NVFP4 build is 47.79 GiB. The practical difference is speed versus hardware: NVFP4 is the faster build on a modern NVIDIA card, Q3_K is the one that also runs on Apple Silicon. Both ship with a vision encoder (mmproj-BF16.gguf) that turns pictures into embeddings the language model can read — though reading images needs a small helper program, strata-vision, that is not built from the Edge0 source tree and has to come from the prebuilt engine archive or a Strata checkout.

What hardware do you actually need?

This is where the announcement and the model card tell slightly different stories, and the model card is the one to trust, because it is the document that lists the measured conditions.

Edge0’s announcement quotes a 29 GB peak memory figure. The model card’s own residency numbers tell a fuller story: for the NVFP4 build on an RTX 5090 D with 24 GB of video memory, the model used 23.25 GiB of VRAM plus 38.27 GiB of host RAM — about 61.5 GiB in total. The experts stream from host memory as they are needed, which is the whole trick of the design, but streaming still needs the RAM to stream from. “One GPU” is true. “One GPU and nothing else” is not.

SetupBuildAgent decodeChat decode16K prefill
RTX 5090 D, 24 GB VRAMNVFP498.3 tok/s79.6 tok/s3,165.7 tok/s
RTX 5090 D, 24 GB VRAMQ3_K121.2 tok/s99.7 tok/s3,554.9 tok/s
RTX 4090, 24 GB VRAMQ3_K91.7 tok/s69.9 tok/s3,179.4 tok/s
16 GiB VRAM + 20 GiB RAM (constrained)Q3_K44.9 tok/s36.8 tok/s335 tok/s
Apple M2 Ultra (Metal)Q3_K57.0 tok/s60.9 tok/s800.3 tok/s

All figures are Edge0’s own measurements from the model card, not independent tests. Two things stand out. First, the Q3_K build is actually faster than NVFP4 on the same card for decoding — the smaller build streams less data per token. Second, the constrained 16 GiB VRAM plus 20 GiB RAM setup still produces a usable 44.9 tokens per second for agent work, which is the configuration that matters most for anyone without a flagship card.

The stated requirements are Linux or Windows with an NVIDIA GPU, CUDA 13, and 16 GB or more of VRAM. macOS is the one point where the sources disagree: the announcement suggested day-one macOS support, while an early read of the repository marked macOS as unsupported for these two models. The model card as it stands now lists Metal on Apple Silicon for the Q3_K build — with a measured Apple M2 Ultra row in the speed table — while NVFP4 stays CUDA-only. Treat macOS support as Q3_K-only and check the card again before downloading, because this detail has already changed once.

How to run Edge0-145B locally

The setup has three parts: the weights, the engine, and the server. Expect a download of roughly 41–48 GiB depending on the build, plus build time for the engine.

1. Pick the build for your machine. If you have a 24 GB NVIDIA card (RTX 4090 or 5090 class), take the NVFP4 build for the best quality-per-token, or Q3_K for the fastest decode. If you have 16 GB of VRAM, take Q3_K — it is the build Edge0 measured on the constrained setup. If you are on Apple Silicon, Q3_K is your only option, and NVFP4 will not run.

2. Download the weights from Hugging Face. The NVFP4 weights live at Edge0/Edge0-145B-NVFP4 and the Q3_K weights at Edge0/Edge0-145B-q3k. Keep the model folder exactly as downloaded: edge0-infer identifies the model from the model.json file inside it, so do not rename or flatten the folder.

3. Build the engine. Clone the Edge0 repository from GitHub, then build the two parts it contains: edge0-strata, the engine, and edge0-infer, the command-line server. Follow the “Build (Linux, NVIDIA)” or “Build (Windows, NVIDIA)” instructions in the repository — the build steps are platform-specific and the repository is the current source of truth for them.

4. Start the server. Point edge0-infer at one model folder, not the folder that holds several of them:

build/cpu/bin/edge0-infer serve --model /path/to/Edge0-145B-NVFP4

When the weights are loaded it prints Listening: http://127.0.0.1:8000 and serves OpenAI- and Anthropic-compatible endpoints, so existing tools that speak either API can point at it with the model name edge0_145b. A quick smoke test:

curl localhost:8000/v1/chat/completions -H 'content-type: application/json' -d '{"model":"edge0_145b","messages":[{"role":"user","content":"hi"}],"max_tokens":64}'

For an interactive session instead, edge0-infer chat --model /path/to/Edge0-145B-NVFP4 opens a terminal chat. There is also an Edge0 Chat desktop app for Windows if you would rather not use the command line.

5. If your machine is short on RAM, use the low-memory options. The model card documents three: --mmap-experts reads uncached experts from SSD on demand instead of pinning them in RAM; --resident-budget-gib 20 caps the host-RAM expert cache at 20 GiB (it needs a static expert profile file); and --resident-experts keeps the whole expert complement resident. Expect slower decoding in exchange — that is the trade the constrained 16 GiB setup in the table above already shows.

One quirk worth knowing before your first prompt: add "reasoning_effort": "none" to a request if you want an answer without a thinking pass first. Without it, the model can spend the token budget on reasoning and return an empty reply.

What does the licence actually allow?

Read this before building anything on the model, because the licence is the real catch in this release. Edge0-145B is open-weight, not open-source. Its weights sit under two licences at once: Edge0’s own contributions are under the Edge0 Community Model License 1.0, and the Qwen materials remain under the Qwen Community License 1.0. Both sets of terms must be followed.

UseAllowed?
Personal use, research, modificationYes
Non-commercial sharing, internal useYes
Commercial use of generated outputs (what the model writes for you)Yes
Selling or commercially bundling the model itselfNo — needs a separate written Edge0 licence
Operating a commercial model service on itNo — needs a separate written Edge0 licence
Offering an independent commercial coding or office assistant using itNo — needs a separate written Edge0 licence

The framework code is a different story: it is Apache 2.0, and the Strata engine it extends is MIT. Only the weights carry the commercial restriction. Edge0 lists commercial licensing enquiries at info@edge0.ai, and notes that its permission does not waive Qwen’s separate authorisation or attribution requirements.

The practical reading is simple. Running the model for yourself, for research, or inside your own company is fine. Shipping a product that is the model, or a service that sells access to it, is not — that needs a written deal with Edge0 first. Anyone planning a product on this model should read the licence file in the repository before writing a line of product code, not after.

Should you run it? Five questions first

The honest answer depends on what you want the model for. Ask these five questions before downloading 48 GiB of weights:

1. Do you have the RAM, not just the GPU? The headline is one consumer GPU, but the NVFP4 build needs about 61.5 GiB of memory in total. If your machine has 24 GB of VRAM and at least 40 GB of system RAM, you are in the measured configuration. If not, the Q3_K build on the constrained setup is the realistic target.

2. Is your job agentic coding, or general chat? Edge0 positions this as an agent model, and its own benchmark leads are on agentic tests — DeepSWE 1.1 at 64.6 versus 58.7 for the parent Qwen model, and Terminal-Bench 2.1 at 86.5 versus 83.3. Those are Edge0’s own figures, with no independent verification yet, but they say where the model is meant to shine. For ordinary chat, a smaller model will be faster and cheaper to run.

3. Are you building a product on it? If yes, stop and read the licence section above. The commercial restriction on selling, bundling, or serving the model is the one thing in this release that can turn a weekend project into a legal problem.

4. Can you live without a technical report? Edge0 has promised a technical report but has not published one yet. Everything about quality rests on the company’s own benchmarks and the model card. That is normal for a fast open-weights release, but it means you should test the model on your own tasks before trusting it with real work.

5. Is 256K context what you actually need? The long context is genuine, but long context on a local machine is where the memory and speed trade-offs bite hardest. If your tasks fit in 32K, a smaller local model may serve you better for less hardware.

If the answers are yes, yes, no, yes, yes — download it. If any answer is no, the model is still worth watching, but not worth 48 GiB of your disk this week.

What Edge0 hasn’t proved yet

Three gaps are worth stating plainly, because they are the difference between a promising release and a proven one. First, every speed and quality figure in this article is Edge0’s own. No independent lab has reproduced the benchmarks, and the six-benchmark average Edge0 quotes — 72.8 against 71.7 for the parent model — is a company average over its own choice of tests. Second, the technical report is still promised, not published, so the training data, the post-training recipe, and the evaluation details are not yet open to inspection. Third, the macOS story has already changed once between the announcement and the current model card, so anyone on Apple Silicon should verify the Q3_K Metal support against the card on the day they download, rather than trusting a week-old write-up — including this one.

None of that makes Edge0-145B a bad release. A 145-billion-parameter agent model that decodes at 98–121 tokens per second on a single consumer card, with a documented low-memory path down to 16 GiB of VRAM, is a genuine step toward capable agents that run locally, without an API bill or a data centre. It arrived in the same week that llama.cpp hit 1.0 with a stable API for local apps and Thinking Machines published the open weights of Inkling 2 Small — the open-weights ecosystem is having a very good week. The gap between “you can download it” and “you can sell with it” is exactly where the Edge0 licence sits, and that is the one thing to get right before anything else.

Frequently asked questions

Can Edge0-145B really run on one consumer GPU?

Yes, with a caveat. Edge0 measures 98.3 tokens per second of agent decode for the NVFP4 build on an RTX 5090 D with 24 GB of VRAM. But the same measurement needs 38.27 GiB of host RAM alongside the 23.25 GiB of VRAM — about 61.5 GiB in total. One GPU, plus a large helping of system RAM.

Which build should I download, Q3_K or NVFP4?

Take NVFP4 if you have a 24 GB NVIDIA card and want the best quality per token. Take Q3_K if you want the fastest decode on the same card, if you have only 16 GB of VRAM, or if you are on Apple Silicon — Q3_K is the only build the model card lists for Metal, and NVFP4 is CUDA-only.

Is Edge0-145B free to use commercially?

Partly. Personal use, research, modification, internal use, and commercial use of the model’s outputs are permitted under the Edge0 Community Model License 1.0. But selling or bundling the model, running a commercial model service on it, or offering a commercial coding or office assistant with it all require a separate written licence from Edge0.

How is Edge0-145B different from Qwen3.8-Flash-Next?

It is derived from Qwen3.8-Flash-Next, but Edge0 rebuilt the expert layer in all 48 layers — replacing the routed mixture-of-experts with a latent version — and then post-trained the model on its own data. Routed expert parameters fall from 120.8 billion to 84.6 billion, and expert parameters per token from 2.36 billion to 1.65 billion. It is not a relabel or a re-quantisation.

Can I use it with my existing tools?

Yes, if they speak the OpenAI or Anthropic API. edge0-infer serves both, on localhost port 8000, with the model name edge0_145b. There is also an Edge0 Chat desktop app for Windows, and a community single-file build for the Cortiq engine whose author reports running it from 12 GB cards — though that build is not made or endorsed by Edge0.

Sources and methodology

Sources: Edge0/Edge0-145B-NVFP4 model card on Hugging Face (read in full on 17 November 2026, including the architecture, speed, memory, licence and troubleshooting sections); FreshSynapse, “Edge0 Releases a 145B Model Built to Run on One Consumer GPU” (14 November 2026); Edge0/Edge0-145B-q3k model card; Edge0 GitHub repository. Internal links: our guide to running Mistral Large 4 locally, our Qwen4-Max Preview explainer, and our 10 November new-AI-tools roundup. Method: all specifications, speeds, memory figures and licence terms were verified against the Hugging Face model card on the day of publication; every benchmark and speed figure is Edge0’s own and is labelled as such, with no independent verification available yet. This article was desk-researched, not hands-on tested — Edge0-145B was not run on our hardware — and that is stated plainly. No X post was embedded because no verified Edge0 announcement post URL was found; none was guessed.

Leave a Reply

Your email address will not be published. Required fields are marked *