Shim AI Gateway

Introduction

Shim is an SDK-native governance gateway. It validates each request, resolves the authenticated tenant, applies privacy and admission policy, calls the selected provider, then settles usage and audit records. OpenAI and Anthropic clients can switch by changing the base URL and API key. Gemini clients keep their native routes. Compatibility is scoped to the documented endpoints, not every SDK resource.

Authentication

Authenticate requests using your Shim API Key. You can generate keys in Gateway keys.

Both methods are equivalent. The Anthropic SDK may instead send the Shim key as x-api-key on its native Messages and Models routes. That header is gateway authentication, never Anthropic BYOK.

Quick Start

The Python examples use the official OpenAI and Anthropic SDKs. Gemini requests use its native endpoints and payloads. Shim does not translate, route, or fail over requests between providers.

Set optional OpenAI and Gemini output limits so quota and spend reservation match the response size you intend. Omitting one reserves the platform ceiling, and Anthropic always requires max_tokens.

# Tested with openai==2.53.0
from openai import OpenAI

client = OpenAI(
    base_url="https://api.getshim.tech/v1",
    api_key="sk-shim-..."
)

response = client.responses.create(
    model="gpt-5.6-luna",
    input="Hello world",
    max_output_tokens=128
)

chat = client.chat.completions.create(
    model="gpt-5.6-luna",
    messages=[{"role": "user", "content": "Hello world"}]
)

models = client.models.list()

Coding agents

The gateway above covers the traffic your applications send. Your developers also run coding agents all day, and those calls never pass through it. Shim CLI covers that side. It runs on the developer machine, with no account, API key or network call to Shim, and it works with Claude Code, Codex and GitHub Copilot CLI.

uv tool install --python 3.12 --compile-bytecode shim
# or
pipx install --python python3.12 shim

shim install claude --dry-run
shim install claude
shim doctor claude

shim watch -- claude
printf '%s' 'Mail alice@example.com' | shim redact

Native Inference APIs

Each documented route preserves provider-owned request fields and native success JSON or SSE. Compatibility is endpoint-scoped rather than a promise that every SDK resource works, and Shim never silently moves a request to another provider. OpenAI Responses background execution is excluded because Shim does not expose its retrieval lifecycle.

Shim applies controls before an upstream call, then finalizes usage and audit state exactly once.

Shim records daily requests, model usage, input/output tokens, and settled cost. Budgets can target an organization, team, or request tag with USD or token limits and alert thresholds.

When enabled by tenant policy, Shim scans prompts for PII (Personally Identifiable Information) at the gateway before sending them to the selected provider.

Randomized placeholders are request-local and repeated values reuse the same placeholder within a request. The value is restored only in the client-facing response. OpenAI Responses continuations additionally use encrypted, tenant-bound, time-limited mappings. Other provider routes preserve their native conversation semantics. While scrubbing is enabled, opaque image, audio, and file inputs are rejected because Shim cannot inspect them safely.

Detected PII Categories

Shim applies configured request-per-minute and token-per-minute burst limits, durable daily/monthly quotas, and exact-repeat loop protection before the selected provider is called.

When a configured limit is exceeded, Shim returns 429 Too Many Requests. Admission errors may also name the rejected dimension.

Streaming

Shim preserves each provider's streaming response format: set stream: true for OpenAI or Anthropic, and call Gemini :streamGenerateContent?alt=sse.

Audit & Compliance Evidence

Shim keeps tenant-scoped request lifecycle and audit records and exposes audit-chain verification, evidence reports, findings, and oversight workflows through the management and compliance control planes.

Compliance reports map observed evidence to controls. They are not legal advice, certification, or attestation.

Health Check

Use the health endpoint for monitoring and uptime checks.

{
  "status": "ok",
  "version": "0.1.0",
  "database": "connected",
  "redis": "connected"
}

Dependency failures return a "degraded" status. A database failure also makes the health endpoint return HTTP 503. Redis health is reported separately.

Supported Models Reference

Chat Completions and Anthropic Messages require an approved model. OpenAI Responses may omit model and input to use the provider default, but an explicit model must be approved. GET /v1/models is provider-aware: the OpenAI and Anthropic SDKs receive native responses for their matching enabled catalogs. Gemini model admission remains on its native routes.

Provider failures preserve the upstream status and safe request ID but sanitize their bodies into the calling SDK's native envelope: OpenAI {"error": {...}} or Anthropic {"type": "error", "error": {...}, "request_id": "..."}.

Response Headers

Completed inference responses include the Shim lifecycle identifier. Provider responses and sanitized provider errors include a safe upstream identifier when one is available.

Provider Credentials

Store an OpenAI, Anthropic, or Google key for the tenant in the dashboard, or supply one for a single request. Shim consumes an invocation-scoped key as the provider credential and excludes it from durable request metadata. Authenticate Shim separately with Authorization, x-shim-key, or the Anthropic SDK's gateway x-api-key. Only documented Shim and provider-credential headers are interpreted. Arbitrary provider-specific caller headers are not forwarded upstream.

x-provider-key is also a generic fallback for OpenAI and Google. The Anthropic SDK's x-api-key contains the Shim key and authenticates the gateway, and it is never BYOK. Every invocation-scoped Anthropic provider key uses x-provider-key.