What is DwarfStar?
DwarfStar, hosted in the ds4 repository, is a native inference engine for a selected set of open-weight language models. It combines model loading, prompt rendering, tool calls, KV state management, an HTTP server and a coding agent. It is deliberately specialized rather than a general-purpose GGUF runner and requires model files produced by the project.
The software is built from source for supported Metal, CUDA or ROCm configurations, with model files downloaded separately. It supports local interactive conversations, coding assistance and multi-session serving. SSD streaming and distributed execution provide options for models that exceed the memory of a single machine.
The project describes the software as beta quality and rapidly changing. Hardware and model support vary by backend, regressions remain possible, and supported models may be removed when replacements are adopted.
Key capabilities
- Selected models DeepSeek, GLM and Qwen models using GGUF files produced by the project.
- Three backends Metal on Macs, NVIDIA CUDA and ROCm on Strix Halo.
- Integrated stack Prompt rendering, tool calls, KV state, HTTP server and coding agent tested together.
- SSD streaming Lets smaller machines run models that do not fit in memory.
- Multi-GPU serving Several CUDA cards can act as a multi-user server.
- Distributed runs Two Macs over RDMA with tensor parallelism.
Where DwarfStar fits
- Local inference Runs DeepSeek, GLM and Qwen models on consumer hardware such as a MacBook, a DGX Spark or a Strix Halo.
- Three backends Metal is the primary target on Macs, NVIDIA CUDA is supported with the DGX Spark as the main goal, and ROCm covers Strix Halo systems such as the Framework Desktop.
- Coding agent and server Ds4-agent runs inference directly without an HTTP server. For Pi, OpenCode, Codex CLI or Claude Code, ds4-server is used instead.
- SSD streaming Smaller machines can stream model weights from SSD, which also lets very large models run on 128 GB systems.
- Multi-user CUDA serving Several CUDA cards, including Ada Lovelace cards, can act as a multi-user LLM server.
- Distributed runs Two 128 GB Macs connected with RDMA can run 4-bit DeepSeek Flash or GLM 5.3 Flash with tensor parallelism.
- Capability checks Ds4-eval runs embedded capability regression tests against a real GGUF.
Reviews of DwarfStar
Be the first to share your experience with DwarfStar.
Write a review
Loading review form…
Alternatives to DwarfStar
FreeTokenFreeToken is an Apache-2.0 serving engine for open-weight MoE models on consumer hardware, with a Windows and Linux desktop app, a CLI and compatible APIs.NNeedleNeedle is an on-device model from Cactus Compute for tool calling, schema-constrained extraction and local embeddings on resource-constrained devices.PMProject MayaProject Maya runs large mixture-of-experts models locally, spreading experts across GPU memory, RAM and SSD, with browser chat and OpenAI-compatible APIs.