What is billion-context?
billion-context is a context-compression layer for AI coding agents, distributed as the npm package billion-context with a command-line tool named bili. It sits between an agent and its model API and lets the model fold consumed parts of a long conversation into summaries. The README states that a 100K context window is enough, that one session can run for a month with billions of cumulative tokens and that the approach uses about 5x fewer tokens; these are the project's own claims. It is developed in the ranxianglei/billion-context repository, created in August 2026.
Compression is driven by the model, not by a fixed truncation limit. The proxy injects four tools, compress, decompress, search_context and acp_status, which the model calls as the context grows, prompted by growth nudges about 50K tokens apart and backed by a preflight check when the input alone exceeds the window. Folded ranges stay stored and can be restored on demand. The README states that a healthy session keeps a 96 to 99 percent prefix-cache hit rate, with compression itself costing 2 percent or less. The engine, acp-kernel, is a host-independent TypeScript library in the repository's kernel/ folder.
Agents attach through a native plugin (bili plugin install), a launcher such as bili claude that starts a fresh proxy with the client without editing its config files, or a /bili/ prefix on the client's base URL. A Dockerfile at the repository root builds an image that runs the proxy as a service. Modes and limits per client are listed in the client deep dives.
The latest GitHub release, v0.1.215 of November 15, 2026, lists more than 70 merged pull requests, mostly fixes to proxy handling, stream parsing and SQLite storage performance, plus a -c/--config option that selects a named config file. Version 0.1.214 added a Context Analysis feature and gates, metered by outbound body size, that stop over-window requests from being sent upstream raw; 0.1.213 added the Kiro CLI launcher lane and bun:sqlite storage.
The LICENSE file contains the MIT License text followed by an "Additional Term": user-facing products or services that use the software, acp-kernel included, must state that they use billion-context and link to the repository on a home page, documentation or About page. The npm package and the Claude Code plugin manifest declare MIT, while GitHub detects no standard license. The README links a preprint, Model-Driven Incremental Hierarchical Compression (v0.2, MIT), and summarizes it as a 4.5-month study across three hosts with 174,327 model calls. Its Status section calls the project early. Similar listings on iThub: Headroom and Context Mode.
Key capabilities
- Install with npm on Linux, macOS and Windows The README one-liner for Linux and macOS is
npm install -g billion-context --prefix=~/.local, which placesbiliin~/.local/binwithout sudo. With nvm or Homebrew Node, and on Windows where the default prefix is user-writable, a plainnpm install -g billion-contextis documented. The package declares Node.js 20 or later. A source build withnpm linkis documented; the README warns that master can be broken. - Native plugins for seven hosts
bili plugin installsupports pi, omp, opencode 1.x and 2.x, dsh (deepseek-harness), kimi (Kimi Code 2.0.0 or later), hermes and zcode; the client then starts as usual andbili plugin removeundoes the change. The installer keeps a.bili-bakconfig snapshot. Native mode and the older standalone extensions billion-context-pi and opencode-acp are mutually exclusive because both active means double compression. Host listings on iThub: Pi, OpenCode and Hermes Agent. - Claude Code plugin and launcher The repository doubles as a Claude Code plugin marketplace:
/plugin marketplace add ranxianglei/billion-context,/plugin install billion-context@billion-context, then/billion-context:bili-setup. The native posture writes a managed settings block, a SessionStart hook and an MCP shell, andbili claudeis the launcher alternative. Documented limits: the Code tab of the Claude Desktop app bypasses the proxy, and Remote Control disappears while traffic runs through bili because the session uses a custom base URL. - Codex support
bili codexis the zero-config launcher for Codex CLI.bili plugin install codexonly writes an[mcp_servers.bili]block that exposes the four context tools; it never starts a proxy or routes Codex's own model traffic, so the tools fail to list unless a bili proxy is already running. - Launcher lanes for more than 20 CLIs
bili <client>covers pi, codex, claude, omp, opencode, hermes, dsh, CodeBuddy, Qoder, TRAE, jcode, Kimi, Gemini CLI, iFlow, Qwen Code, Antigravity, MiniMax Code, aider, Copilot CLI, Amp, Goose, Kiro, Crush and Zed. Each run starts a fresh proxy on its own port and reads, but never edits, the client config to find model hosts;--mitm-domainadds more. Gemini CLI, Copilot CLI, Amp and Goose are launcher-only, jcode, aider and MiniMax Code have no native mode, and Goose cannot be certificate-intercepted, so it uses base-URL redirects. - URL prefix for any client with an editable endpoint Running
bilistarts the proxy on port 8787; the client's base URL is then prefixed, for examplehttp://localhost:8787/bili/https://api.openai.com/v1, which embeds the upstream URL so the proxy needs no forwarding config./__bili/healthand/__bili/statsconfirm that it answers. The client guide lists Cline, Roo Code, Continue, OpenHands, Cursor and others as community-verified entry points. Clients that only offer an HTTP proxy setting need their model host added to the MITM whitelist and the bili root CA trusted. - Model-driven compression tools
compressfolds a message range into a detailed summary,decompressrestores it,search_contextsearches summaries and visible messages, andacp_statusreports usage and compressible ranges. The proxy executescompressserver-side so folded ranges stay summarized in history. The nudge step is set bycompress.nudgeGrowthTokens, and the FAQ explains why the first compaction can wait until about 200K tokens when the boot content is large.compress.requireNudgeforbids unprompted compression. - Four wire protocols The ingress adapters and egress mapping handle the Anthropic Messages, OpenAI Chat Completions, OpenAI Responses and Google native wires, as shown in the architecture hub. Requests to unrecognized paths are relayed byte for byte and never compressed, and that outcome is logged and counted in
/__bili/stats. - Prefix-cache reports
/acp-cacheprints a text report inside the client with GRAND LEDGER, FOLD ECONOMICS and LINE ITEMS blocks, plus MODEL SWITCHES and KEY SWITCHES sections that attribute cache invalidation. The linked web page adds a hit-rate line chart with per-break attribution, andGET /__bili/cache-reportreturns the same data. Key switches are detected from a 12-hex SHA-256 fingerprint of the credential headers; the README states that the raw key is never stored. - Local web panel and JSON endpoints
http://localhost:8787serves an overview dashboard, a session list with per-session detail, live logs, a config editor, an upstream connectivity test and a Blog view refreshed from GitHub about hourly. Sessions with 5,000 or more requests carry a slow-session badge. JSON endpoints for scripting include/__bili/stats,/__bili/sessions,/__bili/configand/__bili/status. - Session identity and child sessions Compression state is keyed on the conversation value the client sends, used verbatim and never forwarded upstream. Sources include client headers such as
x-claude-code-session-idand body fields such asprompt_cache_key. Clients without a signal, such as bare pi, get anonymous prefix affinity, which reattaches only on a byte-exact history match. Subagent sessions can read the parent's compressed context through a read-only link with a depth cap of 8. - Storage and maintenance CLI Config lives in
~/.config/billion-context/billion-context.json, session state in~/.local/share/billion-context/(XDG-overridable) and logs inbili.log, rotated at 10 MB. State is kept in a SQLiteindex.dbin WAL mode.bili sessions ls|rm|archive|restoreandbili storage gcmanage the store without a running proxy. Compression shrinks what is sent to the model, not the client's own conversation file. - Configuration file and precedence One JSON document holds
providers,compress(global, provider and model levels) and server blocks such asnetwork,persist,sessions,updateanddiagnostics. Precedence is CLI flag, then environment variable, then config file, then built-in default.-c dsh-testselectsdsh-test.jsonin the config directory. The configuration reference also covers upstream proxies and signed gateways. - Self-update with staggered rollout The proxy checks npm at startup and every 3 minutes and installs newer versions in place; a restart applies them unless
--auto-restart-on-updateis enabled. Each instance waits a random delay between 15 minutes and 48 hours, steered by a public policy file with hold, blocked and pinned controls.ACP_AUTO_UPDATE=0or"autoUpdate": falsedisables it. An advisory channel can force-install a fixed or older version for a known critical defect unless"advisoryCheck": falseis set. - Docker image from the repository
docker build -t billion-context .installs the published npm package on anode:24-slimbase, which uses the built-innode:sqliteengine. Four volumes hold config, session data, cache and logs, and--build-arg BILI_VERSIONpins a release. The container listens on0.0.0.0:8787without authentication and ships a HEALTHCHECK against/__bili/health. No prebuilt image is referenced in the README. - Remote agents and multiple instances The proxy binds
127.0.0.1by default;--host 0.0.0.0serves agents on other machines, with no authentication and with management endpoints kept loopback-only. Remote certificate interception is limited to whitelisted model hosts. Each conversation belongs to one instance, with no shared store or failover between instances.
Where billion-context fits
Running one coding session for weeks
The project's main scenario is a single agent session that keeps going instead of being restarted or compacted into one opaque summary. The model folds content it has already consumed into summary blocks while recent work stays verbatim, and decompress brings back exact details when a task needs them again. The FAQ notes one boundary: the host's own history file keeps growing, and on dsh, where native auto-compaction is disabled, very large histories slow each turn, so the README names starting a new session as the convergence path at that point.
Working inside a 100K context window
The README states that a 100K window is enough for long tasks. Compression triggers on growth intervals rather than on window position: the first soft compaction fires about 50K tokens past the boot content. The configuration guide lists the levers it documents for an earlier first fold: a shorter system prompt, fewer tools and skills, a lower compress.nudgeGrowthTokens value and lean mode.
Watching prefix-cache health and cost
/acp-cache and the web session page show the cache hit rate over a session and attribute cache misses. According to the README, a healthy session sits at 96 to 99 percent, and lower values usually come, in this order, from upstream cache TTL expiry, a model switch, a bili bug or other causes. Model and key switches have their own sections because both re-bill the stable prefix. Per-request [acp-usage] lines in the log carry the raw input, cached and hit-rate figures.
Adding compression to clients without a plugin seam
Closed binaries such as Copilot CLI, Amp and TRAE, and hosts such as Kiro CLI, aider and jcode that expose no plugin or tool-injection seam, are covered by launcher lanes that intercept TLS for whitelisted model hosts only. Any other client with an editable base URL and an API key can use the /bili/ prefix with a running bili start daemon. The client guide explains the cases where nothing is compressed, such as blind CONNECT tunnels and unrecognized endpoint paths, and how each one is reported.
Combining compression with a routing gateway
For setups that already route models through a local gateway such as magpie or opencodex, the README describes placing bili in front: the client points at bili, and bili points at the gateway through an absolute URL in the /bili/ path. Compression then happens before routing, and dummy client keys pass through untouched. The FAQ names the effects to expect, including cache invalidation on account switches visible in the KEY SWITCHES section.
Serving several agents from one host or container
The Docker image and the --host flag let one proxy serve agents on other machines over /bili/ URLs. Because there is no authentication, the README limits this to trusted networks and keeps management endpoints loopback-only. For scaling, it describes many short independent conversations, one proxy per client or lane, as the workload that spreads well, and requires sticky routing on a conversation header rather than client IP when several instances sit behind one service.
Install
npm install -g billion-context --prefix=~/.localReviews of billion-context
Be the first to share your experience with billion-context.
Write a review
Loading review form…
Found billion-context useful? Upvote it todayYour vote helps other IT teams find it in the daily ranking.






