Govern Enterprise LLM and MCP Usage with Bifrost Edge and Gateway

TL;DR

• Enterprise AI governance has two gaps: shallow controls on provisioned LLM and MCP traffic, and no coverage at all for the AI tools employees run on their own machines.
• Bifrost, the AI gateway, closes the first gap with virtual keys, guardrails, MCP authentication, tool filtering, request logs, and audit logs.
• Bifrost Edge, currently in alpha, closes the second by routing traffic from supported desktop apps, browser AI, and coding agents through the same gateway and enforcing app and MCP server approvals on each device.
• Together, AI Gateway + Bifrost Edge apply one set of policies to both centrally configured clients and endpoint AI usage.

Enterprise LLM and MCP usage has outgrown the access patterns most governance frameworks were designed for. Teams route model requests through approved API clients, security teams configure guardrails for those clients, and logs capture the traffic those clients generate.

Then developers install Claude Code on their laptops, wire up a dozen MCP servers with file system and database access, and route requests directly to model providers. That usage is invisible to the governance layer: no virtual key, no guardrails, no audit record.

Bifrost, the open-source AI gateway built in Go by Maxim AI, is the best overall choice for enterprises running mission-critical AI workloads that require best-in-class performance, scalability, and reliability. Together, the Bifrost AI gateway and Bifrost Edge address both sides of the problem: the LLM and MCP usage that flows through provisioned infrastructure, and the usage that does not.

The Two Governance Gaps in Enterprise AI

Enterprise AI governance has two distinct gaps that require different technical solutions: controls that are too shallow for the LLM and MCP traffic already flowing through provisioned clients, and no coverage at all for the AI tools employees run locally. A complete LLM governance framework has to close both.

The first gap is governance depth for provisioned AI traffic. Teams that have adopted an LLM API client or AI gateway often lack the granular controls that enterprise governance requires: per-consumer access control with scoped model permissions, content guardrails that apply on every request, MCP server authentication and tool filtering, and audit logs that support SOC 2, GDPR, HIPAA, and ISO 27001 programs.

The second gap is governance coverage for endpoint AI traffic. Employees use AI tools that were never configured to route through the governance layer: coding agents, desktop AI applications, browser AI. This traffic carries real organizational data and generates no organizational record. Microsoft WorkLab research, compiled in Programs.com's shadow AI statistics, found that 78% of AI users bring their own AI tools to work, which is the shadow AI problem in its most common form.

Bifrost addresses the first gap as the AI gateway and control plane. Bifrost Edge addresses the second by routing endpoint AI traffic through the same gateway.

Governing LLM Usage at the Gateway

Bifrost governs LLM usage at the gateway by attaching every request to a virtual key that defines which models the consumer can reach, what it can spend, and how often it can call. Guardrails inspect each prompt and response, and request logs record the traffic, so policy lives in one place instead of in each application.

Bifrost provides the access control and policy enforcement layer for LLM traffic that flows through provisioned API clients.

Per-Consumer Access Control with Virtual Keys

Virtual keys are the primary governance entity in Bifrost. Each virtual key represents a specific consumer: an application, a service, a team, or an individual user. Virtual keys carry a complete permission scope: which model providers are accessible, which specific models are permitted, budget limits that cap spend per period, and token- and request-based rate limits that control request frequency.

Every LLM request authenticated against a virtual key inherits these permissions. A request from a customer-facing AI application virtual key can only access the models explicitly permitted for that key. A request that exceeds the configured budget limit is blocked before it reaches the provider. Budget and rate enforcement is per-consumer, not global: one team hitting their limit does not affect other consumers.

At enterprise scale, access profiles define reusable permission templates that bundle provider access, model permissions, budget limits, and rate limits into a named profile. Profiles attach to users and teams through SSO/OIDC directory sync, so new users receive the correct AI permissions automatically on provisioning and lose them automatically on deprovisioning.

Guardrails on Every LLM Request

Guardrails in Bifrost apply content policies to every request and response that flows through the gateway. Guardrail checks execute in the request path, before the prompt reaches a model provider and before the response returns to the calling application.

Available guardrail integrations include:

• Secrets detection: Gitleaks-backed pattern matching for API keys, credentials, and tokens. Prompts containing credentials are blocked or redacted before they reach an external model, depending on the configured action.
• Custom regex: organization-specific PII detection, internal identifier patterns, and project-sensitive content. The PII detection template covers common personally identifiable information patterns.
• External guardrail providers: including Microsoft Presidio, Azure AI Language PII, AWS Bedrock Guardrails, Azure Content Safety, Google Model Armor, CrowdStrike AIDR, Gray Swan Cygnal, and Patronus AI.

Guardrail profiles are configured once and applied through rules written in CEL, which can target specific virtual keys, teams, customers, or users. A rule scoped to a healthcare application's virtual key can apply the profile configured for PHI protection, while a developer tooling key gets a profile suited to that context. Guardrails do not require application-level implementation.

Request and Audit Logging for LLM Traffic

Bifrost request logs record every LLM request through the gateway, including the model and provider, inputs and outputs, token usage, and cost, with selected request headers captured as metadata. Bifrost Enterprise audit logs separately record administrative activity, such as changes to virtual keys, budgets, and guardrail profiles, with HMAC signing for verification.

Together, these records support SOC 2, GDPR, HIPAA, and ISO 27001 programs. Log exports offload request and response payloads to S3 or GCS, audit logs export as JSON, JSON Lines, or Syslog for SIEM ingestion, and the Datadog integration sends APM traces, LLM Observability data, and metrics to Datadog.

Role-based access control governs which users can review audit logs, modify virtual keys, adjust budgets, and change guardrail profiles. The governance layer itself is governed.

Governing MCP Usage at the Gateway

Bifrost governs MCP usage by acting as the single MCP endpoint agents connect to, holding upstream credentials, filtering which tools each virtual key can call, and logging every tool execution. The approach is covered in more depth in why your AI stack needs an MCP gateway.

The MCP gateway capability in Bifrost addresses the governance problem created by AI agents that connect to external tools and data sources. Without a governance layer, individual agents configure their own MCP server connections, with their own credentials, with no organizational visibility or control over which tools they access or what data they expose.

Centralized MCP Authentication

Bifrost as an MCP gateway centralizes authentication for all MCP server connections. Rather than each agent managing credentials for each MCP server directly, agents connect to Bifrost, which holds and manages the authentication to each downstream MCP server. Bifrost supports six MCP authentication types: none, static headers, OAuth 2.0 with automatic token refresh and PKCE, per-user OAuth, per-user headers, and Token Exchange.

Token Exchange, an enterprise option, extends this to internal MCP servers that trust your identity provider: each caller's identity token is exchanged on the way through, so no per-user credential is stored and offboarding at the identity provider takes effect within at most five minutes.

Tool Filtering and Virtual MCPs

Tool filtering in Bifrost controls which MCP tools each virtual key can access. A virtual key assigned to a customer support agent might have access only to a specific set of approved read-only tools. A virtual key assigned to a developer's coding agent might have access to a broader set including code execution and repository tools. Tool access is enforced at the gateway, not at the application level.

Virtual MCPs, previously called MCP tool groups, define named bundles of MCP tools from one or more servers, each served at its own endpoint and attached to the virtual keys that should reach it. Bifrost Enterprise can also grant Virtual MCPs through access profiles and restrict projects to their assigned Virtual MCPs. Virtual MCPs implement least-privilege access at the tool level: an AI agent is exposed only to the tools its bundles permit, and a Virtual MCP can be disabled centrally without deleting it.

MCP Tool Call Logging

Every tool call routed through Bifrost as an MCP gateway is recorded in Bifrost request logs as an MCP log entry, with the tool's arguments and results and any configured request headers captured as metadata. When an AI agent executes a file operation, queries a database, or calls an internal API through an MCP tool, that action has an execution record tied to the request that made it.

This logging capability addresses a significant compliance gap in many enterprise MCP deployments: AI agent actions through MCP tools have typically carried no organizational record and no attribution.

Governing Endpoint LLM and MCP Usage with Bifrost Edge

Bifrost Edge extends the gateway's governance to employee machines, where desktop apps, browser AI, and coding agents otherwise bypass it. Edge is currently in alpha, and teams register to be onboarded.

The governance capabilities at the gateway cover traffic that flows through provisioned API clients. They do not cover the LLM and MCP traffic that employees generate from their own machines through desktop applications, browser AI, and coding agents.

Bifrost Edge is the endpoint layer that closes this gap. Deployed fleet-wide through MDM platforms including Jamf, Microsoft Intune, Kandji, Omnissa Workspace ONE, and JumpCloud, Edge routes AI traffic from supported applications on each machine through the organization's Bifrost. The virtual keys, guardrails, and request logging configured at the gateway apply to all routed endpoint traffic automatically.

Fleet-Wide LLM Application Governance

App governance gives administrators control over which AI applications are permitted across the fleet. Bifrost Edge inventories AI applications running on each managed device and surfaces them in the Approvals dashboard. Administrators approve or deny each application, and the decision is enforced at the device level. Currently governed applications include Claude Desktop, ChatGPT desktop, Cursor, Codex desktop, Claude Code, Codex CLI, OpenCode, ChatGPT web, and Claude web, with coverage expanding.

The devices dashboard shows which apps and MCP servers are installed on each machine.

Fleet-Wide MCP Server Governance

MCP governance at the endpoint inventories the MCP servers configured inside each supported AI application across every managed device. This fleet-wide catalog, covering which MCP servers exist, in which applications, and across how many machines, gives security teams the visibility they need to make governance decisions about tool access that was previously invisible to them. Administrators approve or deny each discovered MCP server, and denials are enforced on the device. The article on shadow MCP and ungoverned AI tools covers this workflow in detail.

Guardrails and Logs at the Endpoint

Because Bifrost Edge routes endpoint AI traffic through Bifrost, the guardrails configured at the gateway apply to desktop app requests, browser AI requests, and coding agent requests automatically. Routed endpoint traffic lands in the same request logs as gateway API client traffic, providing a unified compliance record across all governed AI usage. From AI gateway to the endpoint explains why this last mile matters.

Putting It Together: A Unified LLM and MCP Governance Architecture

A complete governance architecture using Bifrost and Bifrost Edge puts one policy engine behind three enforcement layers, each applying the same virtual keys, guardrails, and logging:

• Gateway layer: Bifrost as the AI gateway and control plane. Virtual keys provide per-consumer identity for provisioned API clients. Guardrails inspect every LLM request and response. Request logs record all governed traffic, and audit logs record administrative changes. RBAC and SSO govern administrative access.
• MCP gateway layer: Bifrost as the MCP server that AI agents connect to. Centralized authentication for downstream MCP servers. Tool filtering and Virtual MCPs enforce least-privilege tool access. MCP tool calls are logged with their arguments and results.
• Endpoint layer: Bifrost Edge on every machine. LLM and MCP traffic from supported desktop apps, browser AI, and coding agents routes through Bifrost. Fleet-wide AI app and MCP server inventory. Endpoint enforcement of app and MCP server approval decisions.

LLM and MCP Governance Controls by Layer

Each governance layer answers a different question: the gateway decides what provisioned applications may do, the MCP gateway decides which tools agents may call, and Bifrost Edge decides which AI apps and MCP servers may run on a company machine. The table below summarizes the controls each layer enforces.

The full scope of governance capabilities across these three layers is documented on the Bifrost governance resource page and the Bifrost Enterprise page for organizations in regulated industries. Security teams can pair this with the LLM gateway security controls for prompt injection, PII, and audit compliance.

Frequently Asked Questions

What does an AI governance platform do?

An AI governance platform enforces an organization's rules for how AI is used: which models and tools each person or application can reach, what content is allowed in prompts and responses, how much can be spent, and what gets recorded for audit. Frameworks such as the NIST AI Risk Management Framework describe these as ongoing controls rather than one-time policies. Bifrost does this at the gateway for provisioned traffic, and Bifrost Edge extends it to employee machines.

What is shadow AI?

Shadow AI is the use of AI apps, models, and tools without approval from IT or security, such as personal chat accounts, desktop assistants, coding agents, and the MCP servers connected to them. The guide to what shadow AI is covers its risks; endpoint governance is how organizations bring it under policy.

How is governing LLM traffic different from governing MCP traffic?

LLM governance controls which models a consumer can call, what content passes through, and what it costs. MCP governance controls which tools an agent can execute, which credentials reach upstream servers, and what actions get recorded. Bifrost applies both through the same virtual keys, so one identity carries model permissions and tool permissions together.

Does Bifrost Edge replace the AI gateway?

No. Bifrost Edge extends the AI gateway rather than replacing it. The gateway remains the control plane where virtual keys, guardrails, budgets, and logging are configured; Edge routes endpoint AI traffic into that gateway and enforces app and MCP server approvals on each device.

How is Bifrost Edge deployed across a fleet?

Bifrost Edge, currently in alpha, deploys through existing device management platforms, including Jamf, Microsoft Intune, Kandji, Omnissa Workspace ONE, and JumpCloud, on macOS, Windows, and Linux. A managed configuration points each machine at the organization's Bifrost without placing secrets on the device, and users sign in once through single sign-on.

What is an AI usage policy, and how is it enforced?

An AI usage policy defines which AI tools employees may use, what data may be shared with them, and how usage is monitored. Written policies alone are rarely enforced; a gateway and endpoint layer turn them into technical controls by blocking unapproved apps and MCP servers, applying guardrails to prompts, and logging usage.

To see how this architecture applies to your organization's LLM and MCP governance requirements, book a demo with the Bifrost team.