Migrating from Cloudflare AI Gateway
Switch from Cloudflare AI Gateway to Shim
Cloudflare AI Gateway is fast, close to your users and cheap to start. It is an observability and caching layer. It is not a data-protection layer, and that is the gap this page is about.
Side by side
How the two differ
| Dimension | Cloudflare AI Gateway | Shim |
|---|---|---|
| What it optimises | Latency, caching, request analytics at the edge | What leaves your network, and the record proving it |
| PII handling | None, the prompt reaches the provider as written | Detected and tokenized before the request leaves |
| Caching | Exact-match caching at the edge | Exact and semantic caching, opt-in and default-off, and PII-bearing prompts bypass the cache |
| Audit record | Request analytics and logs | Hash-chained audit log with an evidence export |
| Network position | Cloudflare's global edge | Regional deployment, with self-hosting on the Enterprise plan |
| Cost of entry | Free tier, bundled with an account you probably already have | Paid from the point you need governance |
The code change
The diff, in full
Your provider key moves into Shim (encrypted, per organization) and your application carries a Shim key instead. That is the one meaningful change: the provider credential stops living in your application config.
Before· today
from openai import OpenAI
client = OpenAI(
api_key="sk-...",
base_url=(
"https://gateway.ai.cloudflare.com/v1/"
"<account>/<gateway>/openai"
)
)After· through Shim
from openai import OpenAI
client = OpenAI(
api_key="sk-shim-...",
base_url="https://api.getshim.tech/v1"
)What it takes
Under an hour to route, longer to decide what you keep on Cloudflare.
The routing change is one base URL. The design question is whether Shim replaces the Cloudflare gateway or sits behind it. Both work. Running Shim alone is simpler to reason about. Keeping Cloudflare in front preserves edge caching for traffic that carries no personal data, at the cost of two hops to debug.
- Prompts are scrubbed before they cross a provider boundary, rather than logged after they already did.
- A record of which entity types were masked and how often, in a form an auditor accepts.
- Policy that can block a request (model allowlists, region pinning) instead of only observing it.
What Shim does not do
- You give up the edge. Shim runs in a region, not in 300 cities. If your latency budget was built around Cloudflare's proximity, measure before you commit.
- No free tier at the governance level.
- Shim does not replace your CDN, WAF, or anything else you run on Cloudflare: it sits in the AI request path only.
Stay where you are when
Cloudflare AI Gateway is the better answer
- Latency is your binding constraint and the traffic carries no personal data. Nothing here beats an edge cache on a public-content prompt.
- You want zero additional spend and your compliance obligations are already satisfied elsewhere.
- Your AI traffic is one internal tool with no regulatory exposure: a governance layer is overhead you do not need yet.
We would rather you read this and stay than migrate and find out in month two. If one of these describes you, Shim is not the tool you need yet.
Ready to move your first service?
Start free on the hosted gateway, or tell us about your setup and we will plan the switch with you, on our cloud or on your own servers.