Local AI engine
Run Qwen3.8-Flash-Next on your own computer
Strata is an open-source (MIT) inference engine that runs Qwen3.8-Flash-Next on a normal PC — 12–24 GB of graphics memory is enough — and serves an OpenAI/Anthropic-compatible API on localhost, so Claude Code, Codex CLI, OpenCode and Hermes Agent can use it. One-click install for Windows and Linux.
How it works
- The model is 24,576 small experts and each written word needs about ten of them. All of them stay in RAM; the card keeps the ones asked most often and learns which those are while you chat. Your processor computes the rest at the same time as the card, so neither waits for the other.
- Guess, then check: the model’s own draft layer proposes the next words and the full model verifies them in one pass, so one step usually writes two to three words — 1.6–1.8x faster with the same answers, because the big model decides every word.
- Long prompts are read in pieces of up to 8,192 tokens at over 1,000 tokens per second, and the experts of the next layer stream to the card over PCIe while the current layer is being computed.
How much RAM you need
| RAM | Model size | What it means |
|---|---|---|
| 32 GB | Coder (IQ1_M) | Code-focused build, half the experts kept |
| 48 GB | Q2_0 or IQ2_XS | Fits, the larger sizes do not |
| 64 GB | IQ2_XS or IQ3_XXS | The comfortable setup for chat and code |
| 96 GB+ | IQ3_S | Best quality, matches the full model on published tests |
Which of our cards runs Strata
| Card | Memory | Price | Status |
|---|---|---|---|
| NVIDIA GeForce RTX 2080 Ti 22GB | 22 GB GDDR6 (upgraded from 11 GB) | €629.00 | Supported |
| NVIDIA GeForce RTX 3080 20GB | 20 GB (upgraded from 10 GB) | €749.00 | Supported |
| AMD Radeon Pro V620 32GB | 32 GB GDDR6 | €939.00 | Experimental |
| AMD Radeon PRO W7900 48GB | 48 GB GDDR6 with ECC | €4,499.00 | Supported |
MI210, MI50 and V100 are not on Strata’s supported list yet — they run llama.cpp and vLLM with ROCm or CUDA, which our listings state. Cards with more VRAM are faster with Strata because more experts stay on the GPU instead of going to the processor.
Install
Windows: run START-HERE.bat. Linux: ./setup.sh. The installer measures your PC, recommends a size and starts the model; then run strata connect to wire it to your coding tool. It is free and MIT-licensed; the model files carry their own licences.
Also: DeepSeek-V4.1-Flash
The same program runs DeepSeek’s huge mixture-of-experts model through its own engine, strata-dsv41: 40 layers, 384 experts in each, six of them used for every word. The model mostly lives in RAM and on the CPU, the experts that fit go to the graphics card, and the 111 GB n-gram table stays on the SSD — so keep the files on an NVMe drive. The official 2-bit sets are the ones a normal PC runs.
| Set | Disk | Quality |
|---|---|---|
| IQ1_M (1.75 bpw) | 239 GB | Smallest; weaker on prose (+21% perplexity) |
| IQ2_XXS (2.00 bpw) | 257 GB | The balanced pick: +15.5% perplexity, code +8.6% |
| IQ2_XS (2.25 bpw) | 274 GB | Best of the three: +7.3%, code +3.2% |
Measured: an RTX 5070 (12 GB) with 64 GB of RAM answers at about 4.7 tokens/s, an RTX 3060 (12 GB) with 46 GB at about 7.2, and four Radeon AI PRO R9700 at about 31. Below roughly 40 GB of RAM it runs, but very slowly — 64 GB or more is the sensible floor, and that is exactly the system memory we sell. This version has no parallel requests and no image input; thinking is on by default and can be switched off per request.
./setup.sh --family deepseek41 --model IST-DAS-2.00bpw downloads the 2-bit set, the shared Engram table and the drafter (257 GB, resumable, hash-checked). strata launch --model ist-das-2.00bpw --plan shows where it would go on your machine before anything changes.Questions
Which card do I need?
Any NVIDIA RTX 20-50 series card or an AMD card from RDNA 3/4 (RX 7700-7900, 9060/9070, Radeon AI PRO R9700). 12 GB work, 24 GB or more is noticeably faster because more experts stay on the card. RX 6800/6900 are experimental.
Do I need a lot of RAM?
64 GB covers every model size and is the comfortable setup. 32 GB runs the Coder variant, 48 GB runs Q2_0 and IQ2_XS. A low-RAM mode maps the experts from disk for machines with a big card and little memory.
Is it free?
Yes — Strata is MIT-licensed and runs offline. No account, no subscription. The model files come from Qwen, ISTA-DASLab and Unsloth under their own licences.