Enterprise AI Governance with Virtual Keys

Enterprise AI governance is the practice of controlling which teams and applications can reach which models, tools, and budgets, and recording that access for audit. As organizations route production traffic to multiple LLM providers and, increasingly, to external tools through the Model Context Protocol (MCP), that control has to span two different kinds of traffic at once. Managing both through scattered provider API keys leaves no central place to set limits, revoke access, or prove who did what. Bifrost, the open-source AI gateway built in Go by Maxim AI, makes the virtual key the single control point for enterprise AI governance across LLM and MCP traffic. This post explains how virtual keys govern model access, tool access, budgets, and rate limits from one place.

What Is Enterprise AI Governance

Enterprise AI governance is the set of controls that determine who can use AI resources, how much they can spend, which models and tools they can invoke, and how that usage is recorded. It applies to both the language model calls an application makes and the tool calls an agent makes through MCP. The goal is uniform enforcement: one policy layer that every request passes through, rather than per-application logic that drifts out of sync.

Standards bodies now treat this as a core requirement. The NIST AI Risk Management Framework names Govern as one of its four functions, covering the policies, accountability, and oversight that surround an AI system across its lifecycle. Access control, cost limits, and audit trails are the operational expression of that function.

Why Governing LLM and MCP Traffic Is Hard for Enterprises

Provider API keys were designed for a single application talking to a single provider. Enterprise AI traffic broke that model in three ways:

• Multiple providers, no shared limits. A raw OpenAI key and a raw Anthropic key have independent, unrelated quotas. There is no combined budget, no shared rate limit, and no single switch to cut off a compromised consumer.
• Agents now call tools, not just models. With MCP, an agent can invoke external tools that read data, write to systems, or trigger actions. The OWASP Top 10 for LLM Applications flags excessive agency, where an agent holds more tool access than it needs, as a distinct risk. Model-level controls do nothing to limit which tools an agent can reach.
• No unit of accountability. When usage is keyed to shared provider credentials, cost and access cannot be attributed to a team, a customer, or a project. That makes chargeback, revocation, and audit reporting manual and error-prone.

Governing this well requires a single entity that carries access rules, spending limits, and tool permissions together, and that every request authenticates against. In Bifrost, that entity is the virtual key.

How Virtual Keys Govern LLM and MCP Traffic

Virtual keys are the primary governance entity in Bifrost. Applications authenticate with a virtual key instead of a raw provider credential, and the key carries the access permissions, budgets, and rate limits that apply to every request made with it. Because the same key governs both model calls and tool calls, LLM and MCP traffic are controlled through one object rather than two disconnected systems. Virtual keys can be activated or deactivated instantly, so revoking access is a single action.

Requests present a virtual key through standard headers, including the OpenAI-style Authorization: Bearer, the Anthropic-style x-api-key, and the native x-bf-vk header, which keeps the governance layer compatible with existing SDKs.

Controlling LLM Provider and Model Access

Each virtual key defines which providers and models it can use. A key scoped to a specific set of models cannot call anything outside that list, and key restrictions can limit a virtual key to specific provider API credentials. A typical configuration allocates providers and allowed models per key:

curl -X POST <http://localhost:8080/api/governance/virtual-keys> \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Engineering Team API",
    "provider_configs": [
      { "provider": "openai", "weight": 0.5, "allowed_models": ["gpt-4o-mini"] },
      { "provider": "anthropic", "weight": 0.5, "allowed_models": ["claude-3-sonnet-20240229"] }
    ],
    "team_id": "team-eng-001",
    "is_active": true
  }'

This means one gateway policy governs access across every connected provider, so adding a model or rotating a provider credential does not require touching application code.

Controlling MCP Tool Access

For agentic traffic, MCP tool filtering is enforced on the same virtual key. The model is deny-by-default: a virtual key with no MCP configuration has access to no tools, and administrators explicitly add the MCP clients and tools each key may use. Within a client, you can allow specific tools, permit all with a wildcard, or leave the list empty to block them entirely.

Bifrost enforces the allow-list at two points: when the model is deciding which tools to call, and again at tool execution time. Inactive or expired virtual keys are rejected at execution with a 403, regardless of their tool configuration. This gives enterprises a way to grant an agent exactly the tools its task requires and nothing more, which directly addresses the excessive-agency risk. The MCP gateway and its tool connections run through the same control plane as model traffic.

Budgets and Rate Limits Across the Hierarchy

Budgets and rate limits attach directly to virtual keys and roll up through a hierarchy. A customer can hold a budget, a team beneath it can hold its own, and each virtual key beneath the team has an independent budget and rate limits, with provider-level limits inside the key. Bifrost checks all applicable levels cumulatively, so a team cannot exceed its allocation even if an individual key still has room.

• Budgets set a dollar limit per reset period (1m, 1h, 1d, 1w, 1M, 1Y), with optional calendar-aligned resets, and Bifrost calculates cost from real-time provider pricing and token usage.
• Rate limits throttle both tokens and requests per period at the virtual key and provider-config levels.

Because cost is tracked per key, per team, and per customer, spend is attributable for chargeback and audit rather than pooled under shared credentials.

Scaling Governance Across the Organization

Virtual keys give enterprises the enforcement primitive, and Bifrost adds the layers a large organization needs to operate this governance model at scale. Role-based access control assigns fine-grained permissions with custom roles, so administrators, team leads, and engineers see and change only what their role allows. Access profiles bundle reusable provider, model, budget, rate-limit, and MCP policies that auto-allocate virtual keys, which removes manual per-key setup when onboarding many teams.

Identity is handled through your existing provider. User provisioning connects Okta, Microsoft Entra, Keycloak, Zitadel, or Google Workspace over OAuth 2.0 and OIDC, maps IdP groups to Bifrost teams, and supports inbound SCIM 2.0 so users inherit the right roles automatically. Audit logs then produce immutable trails suited to SOC 2, GDPR, HIPAA, and ISO 27001 evidence, recording access and policy events across all that traffic.

For organizations with strict data-handling requirements, in-VPC deployment keeps the gateway and its governance data inside private infrastructure with no public egress. These capabilities are why Bifrost is positioned for enterprises and regulated industries that have to audit AI access rather than assume it.

Getting Started with Bifrost

Enterprise AI governance stops being a per-application problem once every request authenticates against a virtual key. With Bifrost, one key governs which models an application calls, which MCP tools an agent can execute, how much each team can spend, and how fast requests can flow, all recorded for audit and all managed from a single control plane. Explore the governance resources to see how virtual keys, budgets, and MCP tool filtering fit together.

To see how virtual keys can bring your LLM and MCP traffic under one governance layer, book a demo with the Bifrost team.