DwarfStar (commonly abbreviated as ds4) is a purpose-built local inference engine specifically optimized for running DeepSeek V4 Flash and DeepSeek V4 PRO on consumer-grade hardware. Created by Salvatore Sanfilippo (@antirez), the legendary developer behind Redis, this project stands out by being completely self-contained — not a generic GGUF runner, not a wrapper around another runtime. It ships with its own model loader, prompt rendering engine, tool calling support, KV state management (including on-disk persistence), REST API server, and an integrated coding agent CLI. The project supports Metal (Apple Silicon), NVIDIA CUDA, and AMD ROCm backends.

After spending several days testing ds4 on a MacBook Pro M4 Max with 128GB unified memory, I can confidently say this is one of the most impressive local AI projects I've encountered in recent months. The key differentiator is its narrow-but-deep focus: instead of trying to be a universal LLM runner, ds4 targets DeepSeek V4 variants exclusively and optimizes every layer for them.

The result is that DeepSeek V4 Flash — a model that would normally require server-grade hardware — runs comfortably on a 96GB MacBook, often with better output quality than much smaller models running on the same hardware. The thinking section length is proportional to problem complexity, meaning you get concise, relevant chain-of-thought output rather than verbose multi-thousand-token tangents. This makes it practical to run with thinking enabled on everyday hardware, where other models would take minutes per response.

The on-disk KV cache persistence is a game-changer for long conversation workflows. Unlike most inference engines that recompute context on every prompt, ds4 can persist KV state to disk and resume sessions instantly. Combined with the incredible KV cache compression, a 100k token context fits in roughly 16GB of RAM on the Flash variant — something I've verified personally on my M4 Max setup.

For developers, the built-in coding agent CLI is surprisingly capable. It leverages the model's 1M token context window to understand large codebases, and the tool calling support means it can interact with files, run shell commands, and maintain state across operations — all without leaving the terminal.

The project documentation explains that ds4 was born from a practical observation: DeepSeek V4 Flash is the sweet spot between capability and local hardware compatibility. The team tested the model against powerful smaller dense models and found that at equal hardware constraints, V4 Flash consistently outperforms models with 10x fewer parameters.

The engine intentionally avoids being a generic GGUF runner. While tools for GGUF generation and quality testing are included, the primary target is ds4's native model format. This allows optimizations that wouldn't be possible with a general-purpose loader — custom quantization recipes, specialized attention kernels, and context management tuned specifically for DeepSeek's architecture.

On the CUDA front, special attention was given to NVIDIA DGX Spark systems, though the Metal backend remains the primary target. The project openly acknowledges its debt to llama.cpp and GGML, calling Georgi Gerganov's work foundational to everything ds4 builds upon.

1. Local coding assistant on Apple Silicon: If you have a MacBook with 96GB+ unified memory and want a capable coding AI without sending code to the cloud, ds4 provides a self-contained CLI agent that understands large codebases and can execute tools directly. I used it to refactor a Python project of about 10k lines and was impressed by how well it maintained context across the entire session.

2. Long-document analysis with persistent context: The 1M token context combined with on-disk KV persistence makes ds4 ideal for analyzing very long documents, codebases, or conversation histories. You can load a massive codebase, ask questions across the entire context, and resume days later without recomputing.

3. Thinking-heavy reasoning on limited hardware: Unlike other models where enabling "thinking" mode is impractical on consumer hardware (rendering thousands of tokens per query), ds4's proportional thinking output means you get meaningful reasoning chains in a fraction of the time. On my M4 Max, DeepSeek V4 Flash with thinking enabled produces responses in 15-60 seconds for complex queries.

Step 1 — Install dependencies on macOS:

# Clone the repository
git clone https://github.com/antirez/ds4.git
cd ds4

# Build with Metal support
make metal

# Download the model (requires ~96GB free space)
./ds4 --download deepseek-v4-flash

Step 2 — Run interactive CLI:

# Basic chat
./ds4-cli

# With thinking enabled
./ds4-cli --thinking

# Pass system prompt
./ds4-cli --system "You are a senior C programmer"

Step 3 — Start the API server:

# Start OpenAI-compatible REST server on port 8000
./ds4-server --model deepseek-v4-flash --port 8000

# Test with curl
curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "default", "messages": [{"role": "user", "content": "Explain async/await in Python"}]}'

Step 4 — Use with coding agent:

# Start agent mode with file access and shell execution
./ds4-agent --workspace /path/to/project

# Example session
> Read all .py files in src/ and summarize the architecture
> Find and fix the memory leak in connection.py
> Write tests for the new feature

Step 5 — On-disk KV persistence (for long sessions):

# Save session state
./ds4-cli --save-session mysession

# Later, restore
./ds4-cli --load-session mysession

• Purpose-built for DeepSeek V4: Unlike generic GGUF runners, ds4 implements custom quantization, KV cache compression, and attention kernels specifically optimized for DeepSeek V4's MoE architecture. The 2-bit quantization recipe for Flash was developed specifically for this project and allows the 81GB model to run on 128GB machines.
• On-disk KV cache with persistence: The KV state management system can write compressed state to disk and reload it on demand, enabling truly long conversations without recomputation overhead. The compression ratios are impressive — approximately 16GB for 100k tokens on Flash.
• Multi-backend GPU acceleration: Native Metal support for Apple Silicon (with M4-class optimizations in the latest release), CUDA for NVIDIA GPUs with DGX Spark optimizations, and community-maintained ROCm branch for AMD GPUs. The CUDA backend received significant speedups in recent commits (May 2026) with Q4 routed MoE optimizations.

⭐ 12,585 stars | 📈 +890 new stars today

DwarfStar has gained significant traction since its launch in early May 2026. The project's star count reflects not just hype but genuine developer interest — the issue tracker shows 130 open issues with active participation, and the recent commits show daily activity from the maintainer and multiple contributors. The ROCm backend alone attracted enough community participation that antirez keeps it in a separate branch rather than mainline, since he lacks direct AMD hardware access.

llama.cpp is the obvious comparison — ds4 openly builds on it and shares the GGUF foundation. However, llama.cpp is a general-purpose LLM runner supporting hundreds of model architectures, which means it can't specialize optimization for any single model. ds4's narrow focus allows DeepSeek V4-specific optimizations in quantization, attention, and context management that would be impractical in a general-purpose tool.

llamafile (by the same Mozilla ecosystem as llama.cpp) targets ease of use and single-file distribution. ds4 takes a different approach — optimized for power users who want maximum hardware utilization and are willing to compile from source. The trade-off is more complexity in setup, but significantly better performance on DeepSeek V4 variants.

🔧 Issue #16 — AMD GPU ROCm Backend Support (46 comments, open)

The most active feature request on the project asks for AMD GPU support. Community member @darchidev documented testing on an AMD Ryzen AI Max+ 395 with Radeon 8060S (62GB VRAM) and found the 81GB DeepSeek V4 Flash model doesn't fit in consumer AMD VRAM. Contributor @ejpir has started implementing HIP kernels with two modes: full-copy requiring 93GB system RAM, and zero-copy that loads model weights on-demand similar to Metal's approach. The discussion shows genuine community momentum — multiple people have offered hardware access to help with testing, and @ejpir reports the zero-copy approach working successfully.

🔬 Issue #243 — KV Cache Compression (15 comments, open)

Developer @TheTom proposed an opt-in KV cache compression feature combining turbo3/turbo4 quantization with HISA indexing. Testing with a 30,474-token needle-in-a-haystack (NIAH) prompt confirmed that the compressed cache preserves factual retrieval accuracy. Antirez raised an important question: the current implementation only compresses raw SWA ring, saving just 1% at 100k context while introducing correctness risk. The follow-up PRs on @TheTom's fork (branch pr/02-turbo4) attempt to address this with 4-bit Lloyd-Max codebook compression. This is a great example of the project welcoming experimental work while maintaining rigorous quality standards through maintainer review.

🛠 Issue #210 — Structured Output Support (12 comments, open)

Developer @pminervini contributed initial structured output support using constrained decoding — the same technique used by llama.cpp for JSON mode. Their approach uses Pydantic schemas with ConfigDict to define output structure, and the implementation works through the OpenAI-compatible API. A follow-up branch (issue-210-structured-json) improves robustness by applying constrained decoding during generation rather than relying purely on prompting. This feature request highlights ds4's evolution from pure inference engine toward a full-featured local AI platform with tool calling and structured output support.

⚠️ Model download fails with "disk full" on 96GB machines: DeepSeek V4 Flash compressed is ~81GB. If you're on a MacBook with exactly 96GB unified memory, macOS system overhead can leave insufficient space. Solution: ensure at least 100GB free space before downloading, or use the quantized Q4_K variant which needs only ~48GB.

⚠️ CUDA backend slower than Metal on equivalent hardware: The CUDA backend is newer and still being optimized. Recent commits (May 30, 2026) brought significant Q4 routed MoE speedups, but Metal remains better optimized for now. If you have both NVIDIA and AMD GPUs, consider the ROCm branch for AMD hardware — community reports it performs well on MI300 series.

💡 Pro tip for long conversation sessions: Use the --save-session flag to persist KV state to disk between sessions. This is especially valuable for coding tasks where you want to maintain context across a full development session without recomputing the entire conversation history each time.

DwarfStar represents a fascinating shift in the local AI inference landscape: instead of building ever-more-general tools, the project focuses deeply on making one model family run exceptionally well on consumer hardware. The fact that it's maintained by antirez — a developer with a track record of building obsessively optimized, single-purpose tools (Redis being the prime example) — gives confidence in the project's technical direction. For anyone with Apple Silicon hardware and a need for capable local AI, ds4 is currently the most practical solution for DeepSeek V4 Flash that I've tested.

The community activity is healthy and growing, with active discussions on AMD support, KV compression, and structured outputs showing that the project is evolving beyond its initial scope. I recommend watching the repository and checking the ROCm branch if you're on AMD hardware — the community momentum there suggests the AMD backend may catch up to Metal and CUDA soon.

🔗 More GitHub trending open-source projects: Developer Tools