AI Gateway

What It Does

AI Bridge gives TypeScript code one format for every provider. AI Gateway puts that format behind HTTP, so services in any language, scripts and shell pipelines can share it. Credentials stay on the server, callers refer to accounts and models by name, and switching the model behind an alias is a config change, not a code change.

The gateway ships as two packages that share one config format and one HTTP contract.

The CLI never calls providers directly. It sends the same HTTP requests to the API, either in-process with no socket opened, or to a running server with --url.

One Endpoint, Any Response Shape

POST /{model} runs a completion. The body is always AI Bridge's canonical input, covering messages, media, tools, reasoning, caching and structured output. Query parameters pick the provider, the named account, and the response dialect.

curl 'http://127.0.0.1:8787/llama3.2:latest?provider=ollama&dialect=anthropic' \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Hello"}],"maxOutputTokens":256}'

The response keeps the same envelope whatever dialect you ask for. output is in the requested shape, while usage stays canonical, so token accounting looks the same across providers.

{
  "provider": "ollama",
  "key": "ollama",
  "model": "llama3.2:latest",
  "dialect": "anthropic",
  "output": { "type": "message", "role": "assistant", "content": [...] },
  "usage": { "inputTokens": 8, "outputTokens": 3, "totalTokens": 11 },
  "raw": { ... },
  "meta": { "startedAt": "...", "endedAt": "...", "durationMs": 100 }
}

Named Accounts and Model Aliases

A JSON config declares accounts under keys and public model names under models. Two Anthropic accounts, a work one and a personal one, can sit side by side, and callers choose between them by name.

{
  "defaultKey": "claude-work",
  "keys": {
    "claude-work":     { "provider": "anthropic", "apiKeyEnv": "ANTHROPIC_WORK_API_KEY" },
    "claude-personal": { "provider": "anthropic", "apiKeyEnv": "ANTHROPIC_PERSONAL_API_KEY" },
    "cloud":           { "provider": "ollama-cloud", "apiKeyEnv": "OLLAMA_CLOUD_API_KEY" }
  },
  "models": {
    "writer": { "key": "claude-work", "model": "your-claude-model-id" },
    "coder":  { "key": "cloud", "model": "your-cloud-model-id" }
  }
}

POST /writer resolves to the model and account behind that alias. Native model IDs still work directly with ?key=claude-work. Keys are read from the environment variables you name, and the accounts in the file are the complete registry: nothing else in the environment is added automatically.

From the Terminal

The ai-gateway command reads the same config. Prompts come from arguments or stdin, so it fits into shell pipelines.

ai-gateway complete writer "Summarize this repo in one line"
git diff | ai-gateway complete coder -s "Review this diff"
ai-gateway complete llama3.2 hi --provider ollama --dialect openai
ai-gateway models --key cloud        # provider catalog, one model ID per line
ai-gateway models --configured       # alias, key, provider, native model
ai-gateway keys                      # account, provider, base URL
ai-gateway serve --config ./gateway.config.json

With the bridge dialect it prints the reply text. Other dialects print their JSON, and --json prints the full envelope. Exit codes separate provider errors from usage errors, so scripts can tell them apart.

Mount It or Run It Standalone

The API is a Hono app. Mount it inside a server you already run, and it shares that server's listener and middleware.

import { Hono } from "hono";
import { createDefaultGateway } from "@productivehub/ai-gateway-api";

const server = new Hono();
server.route("/api/ai", createDefaultGateway());
// Pass server.fetch to your existing HTTP listener.

Or run it on its own with ai-gateway serve, which uses native Deno.serve on Deno 2+ and @hono/node-server on Node 22+. It binds to 127.0.0.1:8787 by default. For custom registries, createGateway({ bridge }) accepts any AI Bridge instance you build.

Endpoints

The gateway does not include authentication. Attach your existing Hono middleware before mounting it, or keep the standalone listener on localhost. Errors return a short JSON message with a meaningful status code, and SDK payloads and stack traces stay out of HTTP responses.

Status and Contributing

AI Gateway is at version 0.1.0 and is not yet published to npm. It builds from source alongside AI Bridge. Runtime smoke tests drive all four built-in providers through a real listener against mocked upstream HTTP, so they need no API keys or paid requests.

The project is MIT-licensed and maintained by Segev Shmueli as part of ProductiveHub. Read the API and CLI docs and the contributing guide, and open an issue on GitHub for bugs or feature requests.