Unified Gateway

Unified Gateway is a backend-only, provider-agnostic AI gateway. It exposes stable OpenAI-shaped public endpoints while routing requests to provider-specific upstream APIs through adapters.

Unified Gateway is developed by Boelabs as shared infrastructure for the products and services we run or plan to run.

It is built for teams that want one public model catalog, one auth/rate-limit/logging layer, and one predictable contract across OpenAI, Anthropic, Google, Azure, OpenAI-compatible providers, and custom models.

Highlights

• Exact OpenAI-compatible public contracts for Chat Completions, Responses, Images, Embeddings, Audio Transcriptions, and Models.
• Anthropic-compatible /v1/messages rendering over the same canonical core.
• Provider adapters for OpenAI, OpenAI-compatible APIs, Google AI Studio, Anthropic, Azure OpenAI, Azure Foundry, DeepSeek, MiniMax, Moonshot, and ZAI.
• Public model aliases, weighted pools, cooldowns, retries, dedicated fallbacks, and per-operation transports.
• Master key plus virtual keys with model scopes, RPM/TPM limits, budgets, and rate-limit headers.
• Opt-in response cache, request logs, cost accounting, OpenTelemetry metrics/traces, daily Postgres partitions, and graceful shutdown.
• Admin API and OpenAPI spec for operating deployments, keys, router settings, logs, usage, and fallbacks.
• Runtime extensions stored in Postgres and managed through the Admin API for request/response/stream/image hooks without forking the gateway.

Status

Unified Gateway is pre-1.0. The main contracts are covered by unit and integration tests, but breaking changes are still possible while the adapter surface and admin API settle.

Monorepo layout

This repository is a Turborepo managed with Bun workspaces.

.
├── apps/
│   ├── gateway/        # the Unified Gateway service (runs on Bun; see apps/gateway)
│   └── docs/           # documentation site (Next.js, Fumadocs, MDX)
├── packages/
│   └── tsconfig/       # shared TypeScript configuration (@boelabs/tsconfig)
├── turbo.json          # task pipeline
└── package.json        # workspace root (bun workspaces + turbo)

Dependencies, tasks, and the gateway runtime are all Bun (with Turbo orchestrating the workspace); Bun runs TypeScript directly, so there is no build step.

Quickstart

Requirements: Bun 1.3+, Postgres 18+, Redis 8+ (Docker optional, for local dependencies).

bun install
docker compose -f docker-compose.yml -f compose.local.yaml up -d postgres redis
cp apps/gateway/.env.example apps/gateway/.env
bun run --filter @boelabs/unified-gateway db:migrate
bun run --filter @boelabs/unified-gateway dev

In .env, set at least MASTER_KEY (any long secret) and CREDENTIALS_ENCRYPTION_KEY (openssl rand -hex 32). Everything else ships with production-ready defaults — the full environment reference lives in Setup.

curl http://localhost:4000/health/live    # liveness (no dependencies)
curl http://localhost:4000/health/ready   # readiness (Postgres + Redis + extensions)

From here, the Quickstart guide walks from clone to a first completion: create a deployment through the Admin API, then call /v1/chat/completions with your key.

Common workspace commands, from the repo root:

bun run dev          # run all dev tasks via turbo (gateway + docs)
bun run typecheck    # typecheck every package
bun run test         # unit tests across packages
bun run check        # Biome format + lint (whole repo)

To work on a single package, use --filter, e.g. bun run --filter @boelabs/unified-gateway dev.

Running in production

Both apps ship container images built from the repo root (apps/gateway/Dockerfile, apps/docs/Dockerfile). docker-compose.yml is the production/PaaS base: Postgres, Redis, a one-off migration job, the gateway, and the docs site, without publishing host ports — Coolify, Dokploy, and similar platforms deploy that file directly and expose services through their proxy. For local or single-host use, merge compose.local.yaml, which publishes ports on loopback and provides development-only secrets:

MASTER_KEY=$(openssl rand -base64 48) \
CREDENTIALS_ENCRYPTION_KEY=$(openssl rand -hex 32) \
docker compose -f docker-compose.yml -f compose.local.yaml up -d

The production base leaves both secrets empty and the gateway refuses to start until they are configured. Per-platform guides and the compose naming rationale: Deployment.

Documentation

Everything beyond this page — API contracts, provider setup, admin operations — lives in the docs, authored in MDX under apps/docs/content/docs and rendered as a Fumadocs site (bun run --filter @boelabs/docs dev).

• Overview — what Unified Gateway is, and the full documentation map
• Quickstart — clone to first response, end to end
• Architecture — the request pipeline, traced in precise order
• Setup — requirements, environment, secrets, and database choices
• Deployment — Docker Compose and per-platform guides (Coolify, Portainer, Dokploy, Linux)
• Creating deployments — provider setup and custom model examples
• Routing — balancing strategies, cooldowns, and retries
• Virtual keys — client keys, scopes, budgets, and limits
• Fallbacks — fallback semantics and lifecycle
• Model catalog — catalog schema and capability profiles
• Providers — every built-in adapter's credentials and quirks
• Runtime extensions — uploading extensions, hooks, and versioning
• Operations — production runbook
• Security — auth model, credential encryption, and redaction
• API overview — auth, error shape, and OpenAPI import notes
• Troubleshooting — error shapes, status codes, and known issues with exact fixes

The machine-readable API spec is apps/gateway/openapi.yaml — see the API guide for import instructions.

Common errors

• Bun's TLS rejects self-signed Postgres/Redis certificates (e.g. databases exposed by a raw Coolify/Dokploy port). Connect over a private network without TLS, or use a managed provider with a public-CA certificate.
• The gateway refuses to start until MASTER_KEY and CREDENTIALS_ENCRYPTION_KEY are set — the production compose base deliberately ships them empty.

Exact symptoms, fixes, and every error code: Troubleshooting.

Contributing

Contributions are welcome. Please read CONTRIBUTING.md for the development setup, conventions, and the checks CI runs. All code, comments, and documentation are written in English. AI coding agents should start with AGENTS.md.

Security

To report a vulnerability, follow SECURITY.md. Please do not open public issues for security problems.

License

Released under the MIT License.