What Is an MCP Gateway? How It Governs Agent Tool Access
An MCP gateway is a control layer that gives AI agents one endpoint for every Model Context Protocol server while centralizing authentication, tool access, guardrails, and audit logging. This guide explains how an MCP gateway works and how it compares to MCP servers, proxies, and LLM gateways.
TL;DR
- An MCP gateway is a control layer between AI agents and Model Context Protocol (MCP) servers that centralizes tool discovery, authentication, access control, and audit logging.
- An MCP server exposes tools; an MCP gateway sits in front of many MCP servers and decides which agent can call which tool with which credentials.
- MCP gateways also cut token cost: Bifrost Code Mode reduced input tokens by 92.8% in a benchmark with 508 tools across 16 MCP servers.
- Bifrost runs as an MCP gateway and an LLM gateway behind one set of virtual keys, so model calls and tool calls share one identity, budget, and audit trail.
An MCP gateway is a control layer between AI agents and Model Context Protocol (MCP) servers that gives every agent one endpoint for its tools while centralizing authentication, tool-level access control, guardrails, and logging. Teams need an MCP gateway once agents connect to more than a handful of MCP servers, because each new server otherwise adds its own credentials, its own tool list in the model's context, and no shared audit trail. Bifrost, the open-source AI gateway and MCP gateway built in Go by Maxim AI, governs tool traffic and model traffic with the same virtual keys. This guide explains what an MCP gateway does, how a tool call moves through one, and how it differs from MCP servers, proxies, and LLM gateways.
What Is an MCP Gateway?
An MCP gateway is a server that aggregates many MCP servers behind a single MCP endpoint and enforces policy on every tool call that passes through it. Agents connect to it as if it were one MCP server, while the gateway holds the upstream connections, credentials, and access rules.
The Model Context Protocol specification defines how hosts, clients, and servers exchange tools and resources over JSON-RPC, and states that MCP cannot enforce its security principles at the protocol level, leaving access controls to implementors. An MCP gateway is where most organizations build those controls once instead of in every agent.
Figure 1: Clients configure one endpoint and one credential; the gateway holds the connections and credentials for every tool server.
In Bifrost, connected MCP servers are exposed through a single /mcp endpoint over HTTP and Server-Sent Events. Desktop hosts, IDEs, and coding agents point at that MCP gateway endpoint and see a tool registry filtered to what their credential allows.
Why Agents Need an MCP Gateway
Agents need an MCP gateway because direct MCP connections scale poorly on four fronts: credentials, access control, context cost, and visibility. Each new MCP server adds a secret to manage, tools the agent may misuse, definitions that consume tokens, and activity no central log records.
Gartner predicts that 40% of enterprise applications will be integrated with task-specific AI agents by the end of 2026, up from less than 5% in 2025.
Without a gateway, the common failure modes are:
- Credential sprawl: tokens for each tool server copied into agent configs on every machine.
- Overbroad access: agents that can call every tool a server exposes, including destructive ones.
- Context bloat: hundreds of tool definitions loaded into the model context on every turn.
- No audit trail: no record of which agent called which tool, with which arguments.
Unapproved MCP servers configured inside desktop AI apps are a form of shadow IT, a risk examined in ungoverned MCP servers as the new shadow IT.
How an MCP Gateway Works
An MCP gateway works by authenticating the agent, filtering the tool list to what that agent may use, inspecting tool arguments against policy, attaching the right upstream credential, forwarding the call to the MCP server, and logging the result. The agent never sees tools it is not allowed to call and never holds upstream credentials.
Figure 2: The agent never holds tool credentials, and a tool outside its allow-list is never visible to it.
Each stage in Figure 2 maps to a control:
- Identify the caller. In Bifrost, the agent presents a virtual key, the same credential used for model calls.
- Filter tools. Only tools on the caller's allow-list appear in
tools/listresponses. Bifrost applies tool filtering at three stacked levels: client configuration, request headers, and virtual key. - Inspect arguments. Guardrails can block or redact a call before it executes and check its result.
- Attach credentials. The gateway adds the upstream header, OAuth token, or per-user token.
- Forward and log. The call reaches the server, and the gateway records it for audit.
By default, Bifrost does not execute model-suggested tool calls automatically; autonomous loops are an opt-in through Agent Mode.
MCP Gateway vs MCP Server vs MCP Proxy
An MCP server exposes tools, an MCP proxy forwards MCP traffic between a client and a server, and an MCP gateway governs traffic across many servers. The difference between an MCP server and an MCP gateway is ownership: the server implements tools, while the gateway decides who may call them and with what credentials.
| Dimension | MCP server | MCP proxy | MCP gateway |
|---|---|---|---|
| Primary role | Implements and exposes tools | Relays MCP messages, often across transports | Governs access to many MCP servers |
| Number of upstream servers | Itself | Usually one | Many, aggregated into one endpoint |
| Authentication | Its own scheme | Pass-through | Central identity plus upstream credential management |
| Tool-level access control | Rarely | No | Per agent, per team, per key |
| Audit logging | Server-specific | Minimal | Every call, in one place |
| Typical owner | Tool or service team | Developer | Platform or security team |
If a component exists because there are many agents and many tool servers, it is a gateway. The full breakdown is in MCP gateway vs MCP proxy vs MCP server.
MCP Gateway vs API Gateway vs LLM Gateway
An API gateway governs generic HTTP APIs, an LLM gateway governs model inference calls, and an MCP gateway governs agent tool calls over the Model Context Protocol. The key difference between an API gateway and an MCP gateway is that an MCP gateway understands MCP sessions, tool lists, and tool arguments, not only HTTP routes.
| Dimension | API gateway | LLM gateway | MCP gateway |
|---|---|---|---|
| Traffic | REST, gRPC, GraphQL | Model requests | MCP JSON-RPC tool calls |
| Unit of control | Route and method | Model, tokens, cost | Server, tool, arguments |
| Typical policy | Rate limits, auth | Budgets, fallbacks, routing | Tool allow-lists, per-user credentials |
An agent uses an LLM gateway to reason and an MCP gateway to act. Running both in one AI gateway that governs model and tool traffic lets one virtual key carry the agent's model budget and its tool permissions together.
MCP Authentication and Security at the Gateway
MCP security at the gateway rests on two separate authentication steps: authenticating the agent to the gateway, and authenticating the gateway to each upstream MCP server. Separating the two means agents never receive raw tool credentials, and revoking an agent's access is a single change at the gateway.
Upstream MCP authentication varies by server, so a gateway needs several modes. Bifrost supports six MCP authentication types:
| Auth type | Who authenticates | Typical use |
|---|---|---|
| None | No one | Public servers and local STDIO tools |
| Headers | Admin, once | Shared API keys or bearer tokens |
| Per-user headers | Each user, on first use | Personal API keys and signed tokens |
| OAuth 2.0 | Admin, once | Shared third-party services |
| Per-user OAuth | Each user, on first use | Per-user services such as code hosting and docs tools |
| Token exchange (Enterprise) | Each caller, every call | Internal servers that trust your identity provider |
Per-user OAuth preserves who acted on the upstream service, where a shared token gives every agent the same identity. Beyond identity, the OWASP Top 10 for LLM Applications lists excessive agency and prompt injection among its top risks, and both are amplified when agents hold tools. Bifrost Enterprise guardrails can run before a tool call executes, after its result returns, or both. Regulated teams can review a control-by-control approach in this MCP gateway guide for regulated industries.
Reducing Token Costs for Large MCP Tool Catalogs
Large MCP tool catalogs are expensive because classic MCP sends every tool definition to the model on every turn. An MCP gateway reduces that cost by curating which tools each agent sees and by changing how tools are presented to the model, without reducing capability.
Bifrost addresses this in two ways. Virtual MCPs bundle a curated subset of tools from one or more servers behind a stable /mcp/<slug> endpoint, so each agent loads only what its job requires. Code Mode replaces the full tool list with four meta-tools and lets the model write Python against on-demand tool stubs. Across three benchmark rounds, Code Mode savings grew with catalog size:
| MCP footprint | Input tokens, classic MCP | Input tokens, Code Mode | Change |
|---|---|---|---|
| 96 tools, 6 servers | 19.9M | 8.3M | -58.2% |
| 251 tools, 11 servers | 35.7M | 5.5M | -84.5% |
| 508 tools, 16 servers | 75.1M | 5.4M | -92.8% |
Pass rates held at or above classic MCP in every round; methodology is in the Bifrost MCP gateway benchmark writeup, and a shorter explanation is in what Code Mode does in the Bifrost MCP gateway.
What to Look for in an MCP Gateway
Look for an MCP gateway that covers transports, per-user authentication, tool-level access control, guardrails on tool calls, audit logging, and self-hosted deployment. The best MCP gateway for an enterprise usually shares identity and policy with its LLM traffic, because agents generate both.
- Transports: STDIO, HTTP, and SSE upstream connections with health monitoring.
- Authentication: server-level and per-user credentials, including OAuth with token refresh.
- Access control: allow-lists per agent, team, or key, enforced at
tools/listandtools/call. - Safety: guardrails that inspect tool arguments and results, not only model prompts.
- Visibility: a log of every tool call with caller, tool, arguments, and latency.
- Deployment: an open source MCP gateway that runs inside your own network.
Tool-call logging is the evidence a security review asks for first; the specifics are covered in MCP gateway observability for every AI tool call. Teams comparing products can start with MCP gateway tools for governing MCP server access.
How Bifrost Works as an MCP Gateway
Bifrost acts as both an MCP client and an MCP server: it connects to upstream MCP servers over STDIO, HTTP, or SSE, and exposes their tools to external MCP clients through one endpoint. The same virtual keys, budgets, and logs that govern Bifrost model traffic govern its tool traffic.
Figure 3: Each team sees a stable endpoint with only its approved tools, while servers are connected once at the gateway.
The components in Figure 3 correspond to these Bifrost capabilities:
- Server connections: connect to MCP servers with automatic retry and exponential backoff.
- Per-key tool access: MCP tool filtering per virtual key creates strict allow-lists of clients and tools.
- Curated endpoints: Virtual MCPs serve tool bundles at stable slugs, attached to the virtual keys that should reach them.
- Coding agents: integrations for Claude Code and other CLI agents, with a setup walkthrough in connecting Claude Code to an MCP gateway.
The Bifrost MCP gateway applies one governance model to every connected server. For MCP servers that users configure on their own machines, AI Gateway + Bifrost Edge extends the same gateway policy to endpoints: Bifrost Edge governs MCP servers by inventorying those configured inside AI apps across the fleet and enforcing allow or deny decisions on each device. Edge is currently in alpha. For enterprise MCP deployments, Bifrost adds access profiles, RBAC, and in-VPC deployment.
Frequently Asked Questions
What is the difference between an MCP server and an MCP gateway?
An MCP server implements and exposes tools, such as a code hosting or database integration. An MCP gateway sits in front of many MCP servers, aggregates their tools into one endpoint, and enforces authentication, tool-level access control, guardrails, and logging. Agents connect to the gateway instead of to each server directly, and never hold upstream credentials.
Do we need an MCP gateway?
You need an MCP gateway when agents connect to more than a few MCP servers, when several teams or users share tool servers, or when security needs a record of agent tool calls. A single developer using one local MCP server does not. Most organizations reach the threshold when MCP moves from experiments into shared agents.
What is the difference between an LLM gateway and an MCP gateway?
An LLM gateway governs model calls, including routing, failover, token budgets, and cost tracking. An MCP gateway governs tool calls, including tool discovery, access control, and upstream credentials. Agents use both: one to reason, one to act. Bifrost runs both behind one set of virtual keys, so an agent's model spend and tool permissions are managed together.
What are the best MCP gateways?
The best MCP gateways support multiple transports, per-user authentication, tool-level allow-lists, guardrails on tool calls, full audit logging, and self-hosted deployment. Bifrost is a strong choice for enterprises because it is open source, combines MCP and LLM gateway functions, and offers Code Mode, which cut input tokens by 92.8% at 508 tools in benchmarks.
Is there an open source MCP gateway?
Yes. Bifrost is an open source MCP gateway available on GitHub. The open-source build includes the /mcp gateway endpoint, tool filtering, virtual MCPs, Agent Mode, and Code Mode; Bifrost Enterprise adds guardrails, token exchange auth, access profiles, RBAC, and audit logs. Options are compared in this guide to the best open source MCP gateways.
Getting Started with Bifrost
An MCP gateway gives platform and security teams one place to decide which agents can call which tools, with a record of every call. Bifrost provides that MCP gateway as open-source infrastructure built for enterprise agent workloads, governed by the same virtual keys as model traffic. Book a demo to see Bifrost govern MCP tool access for your agents.
