Helm API

English · Chinese

One control plane for LLM traffic: text, images, subscriptions, fallback, and memory.

Open-source · self-hosted · MIT

LLM apps tend to accumulate routing code in all the wrong places: fallback lists inside clients, one-off patches for provider quirks, hard-coded model names, ad hoc cost controls, and no clean way to answer "why did this request go there?"

Helm API puts that work in one place: an open-source, self-hosted LLM routing gateway — nginx for the LLM world. Your app sends normal OpenAI, Anthropic, Gemini, or image-generation requests. Helm classifies the request, picks a lane, chooses a provider account, falls back when an upstream breaks, translates protocols when needed, and records the whole decision trail. Clients usually change only base_url and the API key.

Manage traffic as configuration, not as code.

# Your app: the same OpenAI client, just a new base_url and key.
client = OpenAI(base_url="http://localhost:8080/v1", api_key="<helm-key>")
client.chat.completions.create(model="auto", messages=[...])   # Helm classifies and routes

Change the model behind a lane? Edit one YAML line — or click in the dashboard. Your apps never notice.

The dashboard — live traffic, token usage by model, spend, and the most recent routing decisions.

Quickstart

Prerequisites: Docker, or Node ≥ 22 + pnpm 10 to build from source.

# 1. Clone and create your env file
git clone https://github.com/EasyMetaAu/helm-api.git && cd helm-api
cp .env.example .env
#    In .env, set HELM_ADMIN_PASSWORD and at least DEEPSEEK_API_KEY

# 2. Start it
docker compose up -d

# 3. Copy the root API key — generated and printed once on first boot
docker compose logs helm | grep -i "root API key"

What	Where
Gateway	http://localhost:8080 (status landing page at /)
Dashboard	http://localhost:8080/admin — HELM_ADMIN_USER / HELM_ADMIN_PASSWORD
API docs	GET /docs (Swagger UI) · GET /openapi.json (OpenAPI 3.1, generated from the same Zod schemas the gateway validates with)
Health / version	GET /healthz · GET /version

docker-compose.yml mounts ./config and ./data — config and database survive restarts. Credentials enter via environment variables only, never the image.

What you get

	Feature	Detail
🔀	Multi-protocol text routing	OpenAI Chat, Anthropic Messages, OpenAI Responses, and Google Gemini — streaming + non-streaming. Text requests share one routing core, with native passthrough when the inbound protocol already matches the selected upstream.
🖼️	Image generation with failover	OpenAI Images (/v1/images/generations), Gemini image models on generateContent, and Gemini Interactions (/v1beta/interactions). Image requests can name an image model or image lane and fail over across providers without text classification.
🧭	Three-layer classification	Deterministic rules (pure, zero-network, unit-tested — always on) → optional small-model eval (temperature: 0, cached, off by default — needs a configured eval model) → balanced lane as the fail-open sink.
🛣️	Lanes + policies	Requests route through lanes (economy, balanced, premium, plus task lanes like coding, json, vision, tool_use), never raw provider names. First-match policies can force a lane, restrict allowed lanes in config, and override reasoning effort. Each lane is a primary model plus an ordered fallback chain. Opt-in Agentic Signals can promote weak ranked lanes without overriding explicit pins or key caps.
🪪	Drop-in for fixed-model clients	A client that hard-codes a vendor model id (Claude Code's claude-opus-4-8, an SDK locked to gpt-5.5) just works — no 400 unknown model. A standard key classifies it like auto; a custom-model key can map each vendor family onto a lane via model-aliases.yaml (cap-bounded).
🛡️	Resilient execution	Circuit breaker (OPEN/HALF_OPEN + single probe), capability filter with explicit skip reasons, -tier 429 skipping, per-key concurrency queueing. Client disconnects are never counted as provider faults.

Roadmap: Account/customer billing is intentionally out of scope. See 09 Roadmap.

Inside the dashboard

The gateway ships a SvelteKit console at /admin (HTTP Basic, five languages) when admin is enabled. Everything here is live: route rules rebind on the next request, runtime settings apply without a restart, provider-pool edits rebuild the next request's pool, and key changes take effect immediately.

Every request, fully explained. Open any request to follow the whole trail: which layer classified it, the policy that applied, the lane's full candidate chain, each provider actually tried, and the cost split down to cached tokens.

A payload inspector built for debugging. With verbatim capture on, the same page loads the full request and response bodies as a collapsible tree (or Formatted / Raw):

• Read anything. Pop any oversized field — a giant system prompt, a tool schema, a continued-session summary — into a fullscreen, copyable reader instead of scrolling a wrapped cell.
• See the multimedia. A media overview at the top collects every image sent (request) and generated (response) as clickable thumbnails — no tree-digging — and inline base64 or remote images still render in place, with zoom, fit-to-window, and open-in-new-tab.
• Edit and replay. Hit Retry, tweak the body, and re-send it in its original protocol (OpenAI Chat, Anthropic, Responses, or Gemini) as an isolated, newly-traced debug run.

Pool your subscriptions. Route Claude Pro/Max, ChatGPT Codex, and GitHub Copilot logins as backends — several accounts per provider, each with its own model curation, egress proxy, priority, live quota, reset-credit controls, and a global account-usage strategy.

Routing is just config. Each lane is a primary model plus an ordered fallback chain. Reorder or swap lane candidates in the UI or YAML; policy and key caps keep clients inside the lanes you allow.

Dashboard — traffic, spend, token usage, recent decisions	Requests — the filterable request log
Request trail — the full per-request decision trail	Lanes — primary + ordered fallback chain per lane
Classifier — eval toggle, threshold, rule weights	Providers — pooled OAuth subscription accounts
Memory — facts & reflections by scope or key	Policies — first-match rules that force lanes or reasoning effort
API Keys — per-key caps, limits, budgets, memory mode	Settings — payload capture, rate limits, queue, DB maintenance

Each screen is annotated in 11 · Admin UI.

Two failure disciplines

This is the design rule everything else hangs off:

• Config and credentials are fail-closed. Invalid YAML, a missing required key, an unknown store driver — the gateway refuses to start. It never runs half