On this page
- What Strata is and how the engine works
- Strata hardware requirements: NVIDIA, AMD, Intel Arc and Mac
- Strata supported models and context size
- How to install Strata
- Strata local API: OpenAI and Anthropic endpoints
- Strata security: loopback bind and the API key
- Strata performance: published figures and their conditions
- Other projects named Strata
- Bottom line
Strata is a free, open-source local AI inference engine, published as Niko1221/Strata on GitHub, that runs large mixture-of-experts language models on a consumer PC instead of a datacenter GPU. The repository describes it as Qwen3.8-Flash-Next on consumer hardware with an OpenAI/Anthropic API on localhost and optional image input, and the project website presents it as a "Local 125B AI" that works on NVIDIA and AMD, Windows and Linux. It ships as a desktop app and a terminal app, needs no account, and is MIT-licensed.
In practice Strata answers three needs: chatting with a large model on your own machine, pointing coding agents such as Claude Code, Codex CLI or OpenCode at a local endpoint, and keeping prompts and files off third-party servers. The name is also used by unrelated projects: a C++20/CUDA engine by ro99, a pure-Rust engine by Strata Labs, and Strata Manager, a community front end for Niko1221/Strata. For the directory profile, see the Strata listing on iThub.
What Strata is and how the engine works
Strata spreads one large model across the whole PC. According to the project's How it works page, Qwen3.8-Flash-Next is made of 24,576 small experts, and each generated word needs only 10 of them. The graphics card holds the layers used for every token plus a cache of the most-requested experts, system RAM holds all experts, and the processor computes the experts that are not on the card at the same time as the GPU works. A 28.8 GB n-gram lookup table stays on the SSD and is read a few rows per token.
More VRAM therefore matters more than a faster GPU: the project says every extra gigabyte holds about 700 more experts, while RAM remains the gating resource. The same engine is compiled for NVIDIA (CUDA) and AMD (HIP) cards.
Strata also speeds up generation with the model's own multi-token-prediction (MTP) layer, which drafts up to 3 tokens that one pass over all 48 layers then checks, averaging 2.4 to 3.2 tokens per pass. After the first message the engine keeps the conversation and reads only what is new, so follow-up turns start quickly.
On parameter count, the official site's title calls it a 125B model. A September 28, 2026 AlphaSignal article, a third-party source, adds that Qwen3.8-Flash-Next activates about 6 billion parameters per token; the official pages fetched for this guide do not state that active-parameter figure.
Strata hardware requirements: NVIDIA, AMD, Intel Arc and Mac
Strata no longer publishes a single minimum spec. The install guide says memory and disk needs depend on the model, quantization, context and enabled features, and that download size is not the same as RAM use. The supported platforms it lists are:
- NVIDIA Windows 10/11 or Linux x86-64 with an RTX GPU; the README lists GeForce RTX 20, 30, 40 and 50 series. CUDA 13 packages require NVIDIA driver 580 or newer, and older NVIDIA cards use the CUDA 12 engine.
- AMD Radeon RX 7700/7800/7900, RX 9060/9070 and Radeon AI PRO R9700, with RX 6800/6900 marked experimental. The AMD HIP guide rates the RX 7900 XT/XTX, RX 9070/9070 XT and R9700 as maintainer-tested, the RX 9060 XT and RX 7800 XT/7700 XT as validated by their owners, and the RX 6000 package as untested on a real card. The engine package carries the HIP runtime, so only the AMD graphics driver is needed. Mixed AMD/NVIDIA runs are not supported.
- Intel Arc Linux only, through the Terminal; there is no Intel desktop installer.
- Apple Silicon M1 or newer on macOS 15 or later; Intel Macs are not supported.
- AMD Ryzen AI Max (Strix Halo) And NVIDIA DGX Spark (Linux ARM64, Terminal only, no desktop app).
Two or more cards of the same make can share one model through a layer split, described in the multi-GPU guide. Older AMD Instinct MI50/MI60 and Radeon VII cards work only as a community-tested source build.
Plan for disk as well as memory. The three smaller Qwen3.8-Flash-Next packs download 66 to 76 GB, and the first start also fetches the MTP draft layer (about 6 GB, 1 GB more with image input). Use an SSD.
Strata supported models and context size
The model guide covers three model families. The README recommends ISTA-DASLab packs first, then Unsloth and other providers where a compatible model fits; the project notes that this order is a recommendation, not a quality or speed ranking.
Qwen3.8-Flash-Next (the default model)
The four original ISTA-DASLab sizes need the following combined RAM and VRAM for the model weights:
| Size | RAM + VRAM for the model | Speed | Quality |
|---|---|---|---|
| Q2_0 | 37.6 GB | fastest | good |
| IQ2_XS | 39.2 GB | fast | better |
| IQ3_XXS | 47.0 GB | slower | great |
| IQ3_S | 54.8 GB | slowest | best; matches the full model on the project's tests |
The fit rule is simple: RAM must hold the experts plus about 10 GB for the OS and other programs. With 64 GB of RAM every size fits; with 48 GB, Q2_0 and IQ2_XS fit. A low-RAM mode lets a 32 GB PC with a 24 GB GPU run Q2_0, IQ2_XS and the Coder, and a 32 GB PC with a 12 to 16 GB card run the Coder, with most experts read from the SSD and much lower speed.
Variants of the same model are also offered:
- Coder (IQ1_M) ISTA-DASLab keeps 256 of each layer's 512 experts, chosen on code, and reports 91% of the full model's SWE-bench Verified score and 99% of LiveCodeBench.
- Swift 1.5 A UkisAI fine-tune that thinks much shorter before answering, under its own licence.
- Unsloth UD-IQ4_XS to Q8_0 4 to 8 bit files that need 72 to 136 GB of RAM to hold all experts.
- Gyro-S and Gyro-M (agentionai) The smallest options, never the default; Gyro-S needs 27.6 GB of combined memory, at lower quality.
- NVFP4 Three NVIDIA-only versions that need about 80 GB of RAM; RTX 50 cards have FP4 tensor cores, other NVIDIA cards run them at 8-bit precision.
Every Qwen3.8-Flash-Next version in the setup menu reads images. The image reader runs on the CPU when the card has less than 24 GB and on the GPU from 24 GB.
DeepSeek V4.1-Flash-1017
DeepSeek V4.1-Flash uses its own engine, a separate download that the app and strata launch fetch. What does not fit in memory is read from the SSD, so the downloads are large: 239 GB for the 1.75 bpw set, 257 GB for 2.00 bpw, 274 GB for 2.25 bpw and 191 GB for the Coder set. It needs a graphics card with 12 GB or more, at least 46 GB of RAM (64 GB is the tested comfortable size), an NVMe SSD and that disk space. Its hardware list is narrower: NVIDIA RTX 30, 40 and 50 (Windows and Linux), the Radeon AI PRO R9700 (Linux), Strix Halo (Windows) and Apple Silicon Macs with 64 GB or more, text only. A Faster mode uses 5 of the model's 6 experts per word, which the project describes as a small, usually unnoticeable quality change.
Qwen3.6-35B-A3B
The smaller Qwen3.6-35B-A3B has 35 billion parameters, 3 billion active per token, 256 experts and an Apache 2.0 licence. Strata runs Unsloth's UD-Q4_K_XL, UD-Q6_K_XL and Q8_0 files on NVIDIA and AMD cards, planning 32 to 52 GB of RAM; the Intel Arc and Mac engines do not run it.
Context size
For Qwen3.8-Flash-Next, the engine reference says context is chosen in setup between 8K and 262K, and longer requests are refused rather than silently cut. The model was trained on 262,144 positions; longer contexts use experimental rope scaling (yarn), and the strata launch context screen offers 8K to 512K with the extra memory each costs. Context costs RAM: KV streaming uses about 13.7 KB of RAM per context token, or 1.7 GB at 128K. DeepSeek V4.1-Flash is fixed at 8192 tokens, the size it was checked at. By default the server answers one request at a time unless "parallel": N is set.
How to install Strata
A normal install no longer uses a source checkout or a .bat file. The README offers two routes.
- Desktop app. Download the one installer for your system from the latest release (the README lists
Strata-0.2.9-windows-x64-setup.exe, a Linux AppImage and a macOS arm64 DMG). On Windows it installs per user with no administrator rights. On first start the Windows and Linux app downloads the engine for your graphics card; then you choose a model and start chatting. The first launch can take time while large model files are checked, with no fixed startup time. - Terminal. Run the one-line installer, then
strata launch:- Linux/Mac:
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh - Windows PowerShell:
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
- Linux/Mac:
strata launch walks through model, storage folder, context, image input, parallel requests, tuning and access settings. Tuning takes about 20 minutes in quick mode and up to 2 hours in thorough mode. For scripts and agents, --plan prints what would be downloaded without changing anything, and --no-tui leaves the server running in the background.
Updates run through strata update or the app's Settings. If an update fails, the old engine stays in use, and strata components rollback returns to the previous one. The app, the Control component and the engine carry separate version numbers; the Strata website currently announces release 0.2.11. Docker images and source builds are also documented. The repository still contains START-HERE.bat, SETUP.bat, UPDATE.bat, setup.sh and update.sh, but per the source install guide installed apps do not use them.
If you prefer to let an AI coding assistant do the setup, the website suggests pointing it at the repository's docs/AI_SETUP.md. Review each proposed command before it runs, especially driver changes and large downloads.
Strata local API: OpenAI and Anthropic endpoints
The engine's server listens on http://127.0.0.1:8080 by default. The engine reference lists POST /v1/chat/completions (OpenAI Chat Completions), POST /v1/messages (Anthropic Messages) and POST /v1/responses (OpenAI Responses, stateless), each with streaming and tool calls, plus /v1/models and /health. Point an OpenAI-compatible app at http://127.0.0.1:8080/v1, or set ANTHROPIC_BASE_URL=http://127.0.0.1:8080 for Claude Code.
strata connect lists ready-made setups for OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent, OpenClaw and the DeepSeek Harness, and its --check option sends one prompt and one tool call the way that app does. Two behaviours are worth knowing before you test an agent loop: JSON-schema output is schema prompting followed by server validation, not grammar-constrained decoding, and the model field in requests is not matched against the loaded model. Test streaming, tool calls and structured output with the exact client you plan to use.
Strata security: loopback bind and the API key
Out of the box the server is reachable from the local PC only. The security policy explains what changes when you bind to a non-loopback address such as --host 0.0.0.0:
- API key. The
"api_key"setting in the run config, or theSTRATA_API_KEYenvironment variable, is then required on/v1/*, on every endpoint that shows the model's state, requests or answers, and on everyPOST. Several keys can be set, one per client. - Bare engine server. Here a key is recommended, not required: the server starts and warns that anyone who can reach the port can use the model and its tools.
--no-api-keymarks that choice as deliberate. - Terminal.
strata launchcreates a key for you when you pick any access wider than this computer, and the key screen lets you turn it off. - Desktop app. "Share with other devices" and the Cloudflare tunnel will not start until an API key is set, and the shared listener answers only the model and chat endpoints.
- Docker. The container listens on every address inside it, and the Docker guide says Strata does not serve beyond this computer without a key, so a key is always on: you set one of 16 characters or more, or the first start creates one and stores it in the volume.
So if a Strata start stops with a message that a non-loopback bind requires an inference API key, set a key with --api-key, "api_key", STRATA_API_KEY or Docker's API_KEY, or go back to 127.0.0.1. Without a key, the server also applies a host-name check against DNS rebinding and an origin check for browser requests; CORS is off unless you list origins.
Other points from the safety notes: Strata sends no telemetry and has no account, runs without internet once a model is downloaded, and installs updates from a signed feed. MCP tools are opt-in and run with your user's rights. Connected programs such as Claude Code have their own network behaviour, so local inference does not by itself restrict what an agent can do with files or the shell. For the one-line installers, the first install trusts HTTPS and the GitHub release itself. The project states it has not had an outside security audit; vulnerabilities go through GitHub's private reporting, and fixes land only in the latest release.
Strata performance: published figures and their conditions
All figures below are maintainer or third-party measurements, not iThub tests, and each depends on engine version, quantization, context and hardware.
| Setup (as published) | Model | Output | Prompt reading |
|---|---|---|---|
| RTX 5070 12 GB, Ryzen 5 7600, 64 GB RAM, engine 0.1.26 | Qwen3.8 Q2_0 | 87.3 tok/s at 1K, 73.7 at 128K, 60.3 at 262K | 2,171 tok/s at 32K |
| Same PC, history table | Qwen3.8 IQ3_S | 53 tok/s short chat, 46 at 128K | 1,620 tok/s at 32K |
| RX 9070 XT 16 GB, 47 GB RAM, Linux, engine before 0.1.40 | Qwen3.8 Q2_0 / Coder | 60 / 44 tok/s short chat | 1,160 / 1,420 tok/s at 32K |
| RTX 5070 12 GB, 64 GB RAM, Strata 0.2.5 | DeepSeek 2.00 bpw | 4 to 6 tok/s | about 110 tok/s at 4K |
| 4x Radeon AI PRO R9700, Strata 0.2.5 | DeepSeek 2.00 bpw | 74 to 77 tok/s | 400 to 450 tok/s at 8K |
| Radeon AI PRO R9700 32 GB | Qwen3.6 UD-Q4_K_XL | 117.8 tok/s decode | not given |
The project notes that its older tables were not fully re-measured for the current release and that newer engines are faster on some cards. Time to first token is prompt length divided by prompt speed: with Q2_0 on the RTX 5070 that is about 4 seconds at 4K and 4.5 minutes at 262K. The website summarises the Qwen speed as 60+ tokens per second on a good GPU. AlphaSignal reported in September 2026 that an early user cited about 70 tokens per second at 128K on an RTX 3090, and noted that independent results remained limited. To measure your own PC, use --tune quick and record time to first token, generation rate and peak memory with a fixed prompt set.
Other projects named Strata
Several unrelated repositories share the name. Check the GitHub owner before following any install or quantization advice, because one project's recommendations do not transfer to another.
ro99/strata: C++20/CUDA engine for models larger than memory
ro99/strata is a C++20/CUDA inference engine that splits one checkpoint across GPU VRAM, host RAM and read-only NVMe, with no Python or ML framework in the runtime. Its rule is fidelity: precision, routing, expert count and top-k come from the checkpoint, and nothing runs below four bits, not even caches, KV storage or drafts. It supports Gemma 4 31B-IT, DeepSeek-V4-Flash-0731, Laguna S 2.1, Inkling Small, GLM-5.2, GLM-5.3-Flash and Kimi-K3, through strata-chat and an OpenAI-compatible strata-server. A model too large for combined memory is reported as I/O-dependent; GLM-5.2 and Kimi-K3 (1.45 TB, 38.6 s per step) fall in that class on its 64 GiB-VRAM / 251 GiB-RAM test workstation. On that machine Gemma 4 31B-IT, fully resident in VRAM, reaches 668.99 tok/s at 128-token prefill and 29.747 tok/s decode.
Its README also shows why a faster kernel does not always help. An FP4 kernel reached 742.9 and 749.9 GB/s on one RTX 3090, about 88% of that card's measured read ceiling, but on DeepSeek V4, where experts stream from RAM, it measured 0.98x, so the route is opt-in per model. The licence is Apache-2.0.
Strata Labs strata-inference: a pure Rust engine
stratalab/strata-inference is a pure Rust inference engine for transformer models, built with zero C/C++ dependencies. Its stated goals are loading GGUF models in Q8_0, Q4_0, F16 and F32, embedding models (BERT, Gemma) and generation models (LLaMA, Gemma), and GPU acceleration through Metal on macOS and CUDA on NVIDIA. Its architecture has a compute-backend trait with CPU, Metal and CUDA implementations and high-level embed() and generate() APIs. It is under active development and MIT-licensed. It targets Rust developers, not desktop users.
Strata Manager: a desktop front end for Niko1221/Strata
Strata Manager is an independent, MIT-licensed community app that manages Niko1221/Strata installations. Strata itself loads models, runs inference and serves the API; Manager installs and updates engine packages, organises models and configurations, controls services, verifies the API step by step and offers chat, usage statistics, tuning with rollback and benchmarks. Its tuning saves results as configuration versions you can roll back, with no fixed performance gain promised. Installers target Windows x64, Linux x64 and Apple Silicon Macs, where the engine is pinned to 0.1.47 and DeepSeek V4.1-Flash is not available.
How to tell the Strata projects apart
| Project | What it is | Language | Main use | APIs | Licence |
|---|---|---|---|---|---|
| Niko1221/Strata | Local inference engine with desktop and terminal apps | Engine plus Python server and apps | Running Qwen3.8, DeepSeek V4.1 or Qwen3.6 on a PC; feeding coding agents | OpenAI and Anthropic compatible | MIT |
| ro99/strata | Checkpoint-faithful engine for models larger than memory | C++20/CUDA | Large MoE models on workstations without lowering precision | OpenAI compatible | Apache-2.0 |
| stratalab/strata-inference | Embeddable inference library | Rust | Embeddings and generation inside Rust apps | Rust embed() / generate() |
MIT |
| hongfaqiu/strata-manager | Desktop manager for Niko1221/Strata | Desktop app | Setup, monitoring, tuning, benchmarks | Uses Strata's API | MIT |
For other ways to run models locally, browse the local LLM inference category or the GPT4All models and hardware guide.
Bottom line
Choose Niko1221/Strata if you want a large model on one PC and own the memory for it: 48 to 64 GB of RAM and a recent NVIDIA or AMD card cover the Qwen3.8-Flash-Next packs, and 32 GB works only with the Coder or a large GPU. Expect workstation inference for one user at a time, keep the server on loopback unless you set an API key, and test your agent's tool calls before relying on it. Pick ro99/strata only if precision fidelity on very large checkpoints matters more than speed, strata-inference only if you are writing Rust, and add Strata Manager only after the engine already works.
Frequently asked questions
What is Strata in AI?
Strata is a free, MIT-licensed local inference engine from the Niko1221/Strata GitHub project. It runs large mixture-of-experts models such as Qwen3.8-Flash-Next on a consumer PC by sharing work between the graphics card, system RAM, the CPU and an SSD. It comes as a desktop app and a terminal app and serves OpenAI- and Anthropic-compatible APIs on localhost for chat clients and coding agents.
Which models does Strata support?
The current model guide covers three families: Qwen3.8-Flash-Next in ISTA-DASLab, Unsloth, Gyro, NVFP4, Coder and Swift 1.5 variants; DeepSeek V4.1-Flash-1017 with its own engine and 191 to 274 GB downloads; and the smaller Qwen3.6-35B-A3B in Unsloth UD-Q4_K_XL, UD-Q6_K_XL and Q8_0 files. Each model file keeps its own licence.
Does Strata work on AMD GPUs?
Yes. Strata runs on AMD Radeon cards through a HIP build of the same engine. The README lists the RX 7700, 7800, 7900, RX 9060, 9070 and Radeon AI PRO R9700, with RX 6800 and 6900 marked experimental. Only the AMD graphics driver is needed because the engine package carries the HIP runtime. Mixed AMD and NVIDIA setups are not supported.
What context size does Strata support?
For Qwen3.8-Flash-Next, the context is chosen at setup between 8K and 262K tokens, and longer requests are refused rather than cut. The model was trained on 262,144 positions; longer contexts use experimental rope scaling, and the launch screen offers up to 512K. Longer context costs RAM. DeepSeek V4.1-Flash runs at a fixed 8,192-token context.
Why does Strata need an API key when it listens on 0.0.0.0?
Strata listens on 127.0.0.1 by default. Opening it to the network exposes the model and its tools, so the desktop app's sharing will not start without a key, the Docker image always uses one, and strata launch creates one for wider access. Set it with --api-key, the api_key config entry or STRATA_API_KEY, or return the bind to loopback.
Are ro99/strata and strata-inference the same as Strata?
No. ro99/strata is a separate Apache-2.0 C++20/CUDA engine that never runs below four bits and targets models larger than memory. stratalab/strata-inference is a separate MIT-licensed pure Rust engine with embed() and generate() APIs. Strata Manager is an independent desktop app that manages Niko1221/Strata installations rather than running inference itself.
Facts link to their official sources. Product details can change after publication: report an error or read the editorial methodology.



