What is Bifrost?

Bifrost is an open-source AI gateway written in Go. It sits between applications and model providers and exposes one OpenAI-compatible HTTP API in front of more than 20 providers, including OpenAI, Anthropic, AWS Bedrock, Google Vertex AI, Azure OpenAI, Mistral, Groq, Ollama and vLLM. On top of that API it adds fallbacks, load balancing across keys, semantic caching, budgets and rate limits through virtual keys, logging, metrics and an MCP gateway. It is built by Maxim AI, whose site names H3 Labs Inc. as the company, and the code lives in the maximhq/bifrost repository. Product pages are on getmaxim.ai/bifrost and the documentation is at docs.getbifrost.ai.

There are three ways to run it. The gateway starts with npx -y @maximhq/bifrost or as the maximhq/bifrost Docker image on port 8080, with a built-in web interface for providers, logs and configuration. A Go SDK is imported with go get github.com/maximhq/bifrost/core for direct embedding. Existing OpenAI, Anthropic and Google GenAI SDKs can also point at the gateway by changing the base URL. The documentation says the unversioned commands suit local evaluation, and that production needs a pinned image version, authentication, TLS and durable storage.

The repository is a monorepo with a core library, provider implementations, framework components for configuration and log stores, the HTTP transport, the web UI and plugins for governance, logging, semantic caching and telemetry. The reviewed release is Bifrost HTTP v2.5.0, published November 6, 2026. It added per-user MCP tool discovery, MCP Apps and multi-round-trip support on the gateway, a base path option, SOCKS5 and TLS-inspecting proxies, Anthropic workload identity federation, caller identity on logs and an 11-language dashboard. The repository also publishes enterprise and plugin releases separately.

The code is under the Apache-2.0 license. An enterprise edition adds features such as guardrails, adaptive routing, clustering, role-based access control, identity-provider integration, audit logs and in-VPC deployment, and is sold through the vendor with a trial offered. The vendor reports about 11 microseconds of added latency at 5,000 requests per second on a t3.xlarge instance.

Inside the application

Bifrost screenshots

Explore the interface. Open a screenshot to view it at full size.

View full size ↗

Bifrost screenshot 1

Official documentation screenshot of the logs view: request counts, success rate, latency, token and cost totals with provider and model columns.

View full size ↗

Provider key configuration

Official documentation screenshot of editing a provider key, including weight, allowed and blocked models.

View full size ↗

Virtual MCP servers

Official documentation screenshot of the virtual MCP list in the MCP gateway.

View full size ↗

Code Mode sequence diagram

Official figure from the documentation showing how Code Mode runs a multi-step tool workflow in a sandbox.

View full size ↗

Guardrail providers

Official documentation screenshot of the enterprise guardrails provider screen.

Source: official Bifrost project page.

Key capabilities

Run with NPX npx -y @maximhq/bifrost starts the gateway locally, with an option to install a specific transport version. It listens on localhost port 8080 by default, and flags change the port, host, log level and log style. Use it for evaluation rather than production.
Run with Docker docker run -p 8080:8080 maximhq/bifrost starts the same gateway. Mounting a data directory at /app/data keeps configuration across restarts, and tagged images exist for amd64 and arm64. Pin a version for anything beyond a trial.
Go SDK The core module embeds the gateway logic directly in a Go program, with native APIs and custom middleware. Use it when an extra network hop is not acceptable or when you need the library without the HTTP server.
One OpenAI-compatible API Requests use the OpenAI chat completions format with model names such as openai/gpt-4o-mini, and Bifrost translates them for each provider. The provider matrix shows which features each provider supports, which varies.
Provider coverage Documented providers include OpenAI, Anthropic, AWS Bedrock, Google Vertex AI, Azure OpenAI, Gemini, Groq, Mistral, Cohere, Cerebras, Databricks, DeepSeek, Ollama, Hugging Face, OpenRouter, Perplexity, ElevenLabs, xAI, Replicate, SGLang and vLLM. Multimodal requests, streaming, embeddings, speech and image calls depend on the provider.
Drop-in SDK replacement Change the base URL of an OpenAI, Anthropic, GenAI or Bedrock SDK to the matching gateway path, such as /openai or /anthropic, and use a Bifrost virtual key as the API key. The docs also describe integrations for LiteLLM, LangChain and PydanticAI.
Automatic fallbacks and retries When a primary provider or model fails, Bifrost can retry and move to configured backups. You define the fallback chain per request or through routing rules, and logs record what happened.
Load balancing across keys Weighted distribution across several API keys per provider, with model-specific filtering. The key selection is described as taking about 10 nanoseconds, which is a vendor figure.
Virtual keys Virtual keys are the main governance object. They carry model and provider filters, budgets, rate limits, key restrictions and an active flag, and each belongs to at most one team or customer. Several auth headers are accepted, including x-bf-vk and Authorization.
Budgets and rate limits Hierarchical cost control at virtual key, team and customer levels, with token and request throttling at the key level. Budget overrides add temporary capacity without editing the base budget.
Semantic caching A cache plugin offers exact-match replay and embedding-based similarity lookup, with direct-only mode needing no embedding provider. The docs distinguish it from provider prompt caching, which is billed differently.
MCP gateway Bifrost acts as an MCP client and server, connecting to external tool servers and exposing them to clients such as Claude Desktop or Cursor. It supports tool filtering per virtual key, agent mode with configurable auto-approval and several auth types including per-user OAuth.
Code Mode for MCP Instead of exposing hundreds of tool definitions, Code Mode exposes a few generic tools and lets the model write Python-like code in a sandbox. The docs report up to 92.8 percent fewer input tokens in a controlled benchmark with 508 tools; treat these as the vendor's own measurements.
Web UI and logs The dashboard lists requests with status, provider, model, latency, tokens and cost, with filters and search. Release 2.5.0 adds a command palette, caller identity on logs and dashboard translations.
Observability integrations Native Prometheus metrics and OpenTelemetry tracing work with Grafana and similar tools, and logging stores are configurable. The enterprise edition adds a Datadog connector and log exports.
Plugins Go and WASM plugins add custom middleware, and a mocker plugin returns simulated provider responses for tests. Governance, logging, caching and telemetry are themselves plugins in the repository.
Configuration options Settings can come from the web UI, an API or a config.json file. The docs describe secrets by environment variable references, a base path option and proxy settings.
Enterprise features Guardrails with several provider integrations, adaptive routing, clustering, role-based access, identity providers such as Okta and Entra, audit logs and in-VPC deployments are documented as enterprise features and need a conversation with the vendor.

Where Bifrost fits

Putting one API in front of several model providers

Add your provider keys in the web interface or config.json, then point applications at the gateway with an OpenAI-style request. Check which provider features your use needs, such as tools, streaming or vision, in the provider matrix before you rely on a fallback target that behaves differently.

Setting spending limits for teams

Create a virtual key per team or product, attach a budget and rate limit, and give out the virtual key instead of provider credentials. Verify that requests are rejected when the budget is spent, and keep the provider keys restricted to the gateway host.

Improving availability with fallbacks

Define a fallback chain across providers or regions, and test it by revoking a key in a staging environment. Look at the logs to confirm which provider served each request, since latency and answers can differ between providers.

Reducing cost with semantic caching

Enable the cache plugin, start with exact-match mode, and add similarity matching only when repeated questions are common. Choose the similarity threshold carefully and exclude prompts where a slightly different request needs a different answer.

Running a shared MCP gateway

Connect tool servers once in Bifrost, filter which tools each virtual key can call, and point clients such as Claude Desktop or Cursor at the gateway. Review auth types for each upstream server, and test per-user discovery before giving tool access to a team.

Observing and auditing AI traffic

Use the dashboard for requests, latency and cost, and export metrics to Prometheus or OpenTelemetry. Decide how long logs are retained, whether to capture client IP and caller identity, and who may read prompts that can contain sensitive data.

Embedding the gateway in a Go service

Import the core module when the application is already written in Go and a separate process is unwanted. Confirm which provider and plugin features the SDK exposes compared with the HTTP gateway, and keep configuration in code or files under version control.