AI Governance Framework for Engineering Teams

What to define:

• Approved model list. Which providers and model versions are sanctioned for production? Which are for dev and testing only?
• Risk classification per use case. A customer-facing chatbot handling financial data has different requirements than an internal code review assistant.
• Data residency constraints. Your model selection policy needs to account for where each provider processes and stores data.
• Cost guardrails. Per-team or per-project token budgets. Without limits, a single runaway agent loop can burn thousands of dollars in hours.

How to enforce it:

Route all API calls through a central gateway that validates the model against your approved list before forwarding the request. Policy violations get blocked at the gateway, not caught in a code review three weeks later.

What to define:

• Data classification tiers. At minimum: names, emails, phone numbers, national IDs, financial info, health records, and login credentials. Also indirect identifiers: IP addresses, device IDs, and location data.
• Redaction requirements by tier. Which categories must be stripped before leaving your perimeter? Which can pass through with encryption?
• Provider data policies. Some providers may store prompts for up to 30 days. Without enterprise agreements, your data could be used to train future models.

How to enforce it:

Automatic PII redaction at the gateway level. Every prompt passes through detection and redaction before reaching any external API. The external provider never sees raw data. This is not optional for teams operating under GDPR, where fines reach up to 4% of global revenue under Article 44 for improper cross-border data transfers.

What to define:

• Role-based access. Not every engineer needs every model. Junior developers might get Haiku-tier. Senior engineers get Sonnet. Only specific projects get Opus-level capabilities.
• API key management. Shared API keys are a governance anti-pattern. Every team or service should have its own credentials, scoped to specific models and rate limits.
• Tenant isolation. Every request must be scoped by organization or team ID. An authentication bug should never expose another team's keys or data.

How to enforce it:

shim's security architecture stores production provider keys in a managed secret backend, never returns stored secrets in plaintext, and keeps requests tenant-scoped. Fernet is used only by the local development backend.

What to define:

• Log retention policy. Align with your regulatory requirements and incident response needs.
• What to capture per request. At minimum: timestamp, user/service identity, model called, token count, latency, HTTP status, and whether PII was detected and redacted.
• Anomaly detection thresholds. Alerts for unusual patterns: token usage spikes, unapproved model requests, or PII detection rates that suggest a data pipeline is leaking unredacted information.

How to enforce it:

A centralized gateway gives you a single audit trail across every AI interaction. Instead of scraping logs from 15 different microservices, you get one unified view: every request, every response, every redaction event.

What to define:

• Incident classification. PII leakage, an unapproved model in production, a prompt injection, or a cost overrun signaling an uncontrolled agent loop.
• Response playbook. Who gets notified? What gets shut down? How do you assess blast radius? Define these before the incident.
• Review cadence. Quarterly review of approved models and data policies. Monthly review of cost and usage trends. Immediate review whenever a provider changes their data policies.

How to enforce it:

Automated monitoring with alerts based on your defined thresholds. The gateway layer catches policy violations in real time rather than in a post-mortem.