Best Zero-Trust AI Gateways for LLM and MCP Access

TL;DR

• A zero trust AI gateway authenticates every caller, grants each user, team, or agent only the models and MCP tools it needs, inspects traffic, and logs every call.
• The hardest zero trust gaps sit on the MCP side: tools that are allowed until someone writes a deny rule, shared admin credentials to upstream servers, and no inspection of tool arguments.
• Bifrost is an open-source AI gateway that runs in your own VPC, applies deny-by-default MCP tool access, exchanges each caller's identity token for a scoped upstream credential, and runs guardrails on LLM and MCP traffic.
• Cloudflare, Kong AI Gateway, Azure API Management, Agent Router, and Google Apigee each cover parts of the model; the reviews below list what each one's own documentation says it does and does not do.
• Deployment narrows the list fastest: self-hosted and air-gapped requirements rule out hosted-only gateways before any feature comparison.

Zero trust architecture removes implicit trust based on network location, and AI traffic needs the same treatment: every LLM request and every MCP tool call should be authenticated, authorized against least-privilege grants, inspected, and logged. A zero trust AI gateway is the enforcement point that makes this possible. Bifrost, the open-source AI gateway built in Go by Maxim AI, is the best choice for enterprises running mission-critical AI workloads that require top-tier performance, scalability, and reliability. This guide compares six gateways on caller identity, tool-level access, upstream credentials, inspection, and deployment.

What Is a Zero-Trust AI Gateway?

A zero trust AI gateway is a control point between AI clients and the models and tools they use that verifies identity on every request, grants only the access each caller needs, inspects prompts and tool calls, and records the result. Nothing is trusted because it comes from inside the network or carries a shared key.

Figure 1: One enforcement point for model calls and tool calls, with identity coming from the directory you already run.

The idea comes from NIST SP 800-207, which describes zero trust as granting no implicit trust based on network location or asset ownership, with authentication and authorization performed before a session to a resource is established. For AI traffic, the resources are model endpoints and MCP servers, and the subjects are people, applications, and agents. The explainer on applying zero trust principles to enterprise AI tools covers the concept in depth.

Zero Trust Principles for LLM and MCP Traffic

Zero trust for AI applies four checks to every request: verify the caller, authorize the specific model or tool, inspect the content, and call upstream with a credential scoped to that caller. Anything that fails a check is denied, and every decision is logged.

Figure 2: Every request is checked; nothing is trusted because it came from inside the network.

MCP raises the stakes because tools take actions. The MCP security best practices call token passthrough, where a server forwards a client's token to a downstream API without checking it was issued for that server, an anti-pattern. A zero trust gateway avoids it by issuing or exchanging a credential for each upstream call rather than passing one through. The overview of MCP authentication with OAuth, API keys, and token management covers the credential patterns, and the MCP security topic page lists the threats they address.

Key Criteria for Evaluating Zero-Trust AI Gateways

The criteria that separate zero trust AI gateways are caller identity, least-privilege grants for models and tools, how upstream credentials are handled, inspection of LLM and MCP payloads, audit logging, and where the gateway runs. Defaults matter as much as features: a control that allows everything until configured is not zero trust by default.

The AI governance resource page maps these criteria to keys, teams, and budgets, and the comparison of enterprise AI gateway security options covers adjacent controls such as rate limits and budgets.

Zero-Trust AI Gateways Compared at a Glance

The six gateways split into a self-hosted AI gateway built for both traffic types (Bifrost), a hosted zero trust platform (Cloudflare), API management platforms that added AI and MCP features (Kong, Azure API Management, Apigee), and an open-source Envoy-based control plane (Agent Router). The table reflects each vendor's documentation as of November 2026.

The Bifrost MCP gateway page shows its tool controls in one place, and each other vendor's gaps are covered in its review below.

The Best Zero-Trust AI Gateways, Reviewed

The best zero trust AI gateways verify identity on every call, deny tools that were never granted, and avoid shared credentials to upstream servers. They differ most in deployment and in how far their controls extend from LLM traffic to MCP tool calls. Each review covers strengths, fit, and the trade-offs the vendor's own documentation describes.

1. Bifrost

The Bifrost AI gateway is an open-source (Apache 2.0) gateway written in Go that governs model calls, MCP tool calls, and agent traffic under one set of identity-based rules. It runs entirely inside your infrastructure, and no prompts, responses, or files leave it.

Best for: Bifrost is built for enterprises running mission-critical AI workloads that require top-tier performance, scalability, and reliability. It serves as a centralized AI gateway to route, govern, and secure all AI traffic across models and environments with ultra low latency. Bifrost unifies LLM gateway, MCP gateway, and Agents gateway capabilities into a single platform. Designed for regulated industries and strict enterprise requirements, it supports air-gapped deployments, VPC isolation, and on-prem infrastructure. It provides full control over data, access, and execution, along with strong security, policy enforcement, and governance capabilities.

Figure 3: Identity comes from your directory, and every model and tool call is authorized, inspected, and logged in your own network.

• Verified identity. User provisioning with OIDC and SCIM connects Okta, Microsoft Entra, Keycloak, or Google Workspace, and Bifrost can sit behind an identity-aware proxy such as Cloudflare Access or AWS ALB that signs every request.
• No anonymous inference. With auth enforcement on, a virtual key or other valid credential is mandatory on every request, and each key is scoped to specific providers, models, budgets, and rate limits.
• Deny-by-default tools. MCP tool filtering exposes no tools to a key without an MCP configuration, unless a client is marked Allow by Default, and request headers can narrow a key's allow-list but never widen it.
• No shared tool credentials. Token exchange swaps each caller's identity-provider token for a short-lived token scoped to the upstream MCP server on every call, with no stored credential and no service-account fallback; per-user OAuth covers SaaS servers.
• Inspection on both traffic types. Guardrails check LLM prompts and responses and MCP tool arguments and results, with secrets detection, PII redaction, and external providers.
• Least-privilege administration. Role-based access control and data access control decide what each administrator can do and which records they can see, and secret management keeps provider keys in your vault.

Audit logs record administrative changes with secrets redacted, and Bifrost is SOC 2 Type II, ISO 27001, HIPAA, and GDPR compliant, with in-VPC and air-gapped options on the Bifrost Enterprise page. AI Gateway + Bifrost Edge extends the same policies to AI apps on employee machines.

2. Cloudflare AI Gateway and MCP Server Portals

Cloudflare pairs AI Gateway for model traffic with MCP server portals in Cloudflare One, where Cloudflare Access signs users in through your identity provider before they reach any MCP server. Both run on Cloudflare's network.

Best for: Organizations already running Cloudflare Zero Trust that want MCP access behind the same Access policies as their other applications.

• Identity and per-server policy. MCP clients authenticate through Access with OAuth, machines use service tokens, and Access policies decide which users see each server.
• Per-user upstream auth. With "Require user auth" on, the default, each user completes their own OAuth with the upstream MCP server.
• Inspection. Portal traffic routed through Cloudflare Gateway gets DLP on tool requests and responses, and AI Gateway adds guardrails, DLP scanning, and OpenTelemetry trace export for model calls.

Trade-offs: Tools are available to portal users by default; an allowlist mode is configured through the API. Cloudflare notes that blocked users can still reach a server through its direct URL unless Access is that server's OAuth provider, and that independent MFA is not enforced for servers authorized through a portal. Service tokens use a shared admin credential upstream, AI Gateway tokens are account-scoped, MCP portal log export is Enterprise-only, and both products are hosted only. The list of Cloudflare AI Gateway alternatives covers self-hosted options.

3. Kong AI Gateway

Kong AI Gateway governs LLM, MCP, and agent traffic through AI entities and plugins on Kong's API platform. AI Gateway 2.x runs hybrid, with a Konnect-managed control plane and data planes you operate.

Best for: Organizations that already run Kong for API management and want AI traffic under the same platform.

• Fail-closed authentication. Callers authenticate with API keys or OIDC tokens from Okta, Azure AD, Google, or another IdP, and failed authentication returns 401 by default.
• Per-tool ACLs. MCP access lists match a consumer, a consumer group, or an OAuth token claim, with CEL rules at the server level, and every allow or deny decision is written to an audit log.
• Scoped upstream credentials. Token Vault gives each caller its own short-lived credential for MCP servers, and LLM providers can use Entra managed identity, AWS IAM, or GCP service accounts.

Trade-offs: Kong's MCP access lists allow a request when no list exists, so tools are not denied by default. Kong lists guardrails on MCP requests and responses as not supported; several LLM guardrail plugins, such as AWS Guardrails and Semantic Prompt Guard, are a paid add-on on Plus. The AI MCP Proxy, AI MCP OAuth2, and AI Proxy Advanced plugins are enterprise-tier, virtual keys cannot be rotated, and fully self-hosted deployment is part of Kong Gateway Enterprise. The roundup of Kong AI Gateway alternatives compares self-hosted options.

4. Azure API Management

Azure API Management adds AI gateway policies and MCP server support to its API gateway. It can expose REST APIs as MCP servers and front existing MCP servers, with Microsoft Entra ID as the identity layer.

Best for: Azure-standardized enterprises that already run API Management and Entra ID.

• Identity. Callers present Entra ID tokens, other JWTs, subscription keys, or client certificates.
• Keyless upstream access. LLM backends can use managed identity, and the credential manager can grant OAuth connections on behalf of a specific Entra user or group.
• Content safety on tools. The LLM content safety policy checks prompts and completions, including prompt attacks, and can also check requests and responses for MCP tools and A2A agent APIs.

Trade-offs: Microsoft's documentation states that policies apply to all tools exposed by an MCP server, with no per-tool scoping described. MCP is not available on the Consumption tier or workspace gateways, and the self-hosted gateway is limited to Developer and Premium and needs outbound connectivity to Azure. OpenTelemetry export is available only on the self-hosted gateway, and no dedicated PII policy is documented. The overview of Azure AI gateway alternatives for multi-cloud traffic compares cloud-neutral options.

5. Agent Router (formerly Envoy AI Gateway)

Agent Router, formerly Envoy AI Gateway and now an Agentic AI Foundation project, is an Apache 2.0 control plane for AI and MCP traffic built on Envoy. It runs on Kubernetes with Envoy Gateway or standalone.

Best for: Platform teams running Kubernetes and Envoy that want an open-source, configuration-driven gateway.

• Strong caller authentication. Envoy Gateway policies provide mTLS, JWT validation, OIDC login, external authorization, and IP allow and deny lists.
• Per-tool authorization. MCP authorization rules match JWT scopes and claims per backend and per tool with CEL, the default action is deny once rules are configured, and a CEL rule that errors at runtime blocks the call.
• Short-lived LLM credentials. Upstream model calls can use AWS STS, Entra ID, or GCP workload identity federation instead of long-lived keys, and MCP sessions are bound to the user who started them.

Trade-offs: Agent Router documents no built-in guardrails, PII detection, or prompt injection checks, so inspection depends on external extensions. Upstream MCP authentication is a static key or a forwarded client header, with no token exchange documented. Without a tool selector, all tools from an MCP server are exposed, and the APIs are still v1beta1 and v1alpha1. The comparison of Envoy AI Gateway alternatives covers gateways with built-in guardrails.

6. Google Apigee

Google Apigee adds AI gateway policies and MCP endpoints to its API management platform. Apigee can host MCP endpoints that turn OpenAPI specs into tools or proxy your own MCP servers.

Best for: Enterprises on Google Cloud that already manage APIs in Apigee.

• Identity. Apigee-hosted MCP endpoints support OAuth 2.1 and OpenID Connect, and clients can also use API keys.
• Tool allowlists. Access is granted per API product: a tool not listed in the client's product returns 401, and the tool list is filtered to match.
• Model Armor. Sanitize policies call Model Armor on LLM prompts and responses for prompt injection, jailbreaks, and sensitive data, and Apigee hybrid keeps MCP traffic inside your cluster.

Trade-offs: Model Armor policies are Extensible policies with added cost and need a higher environment tier, priced at $1,460 or $3,431 per month per region on pay-as-you-go. Apigee limits an organization to 1,000 MCP tools, accepts OpenAPI 3.0.x only, and does not support SSE; the newest stateless MCP protocol is not available on hybrid. Per-user token exchange to upstream MCP servers is not documented. The roundup of Apigee AI gateway alternatives covers lighter-weight options.

How to Choose a Zero-Trust AI Gateway

Choosing a zero trust AI gateway starts with where it must run and which identity stack you already own. If prompts, keys, and logs must stay inside your network, only self-hosted gateways qualify; if Cloudflare, Kong, Azure, or Google Cloud already holds your access policies, extending that platform reduces new infrastructure.

Figure 4: Where the gateway must run and which identity stack you already own narrow the list fastest.

Whichever way the flow points, test three behaviors before committing: whether a new MCP tool is blocked until someone grants it, whether a tool call reaches the upstream server under the caller's identity or a shared credential, and whether guardrails see tool arguments and results as well as prompts. The enterprise deployment guide covers in-VPC and air-gapped installs, the AI guardrails resource page covers inspection, and the write-up on shadow MCP servers on employee machines covers traffic that never reaches a central gateway.

Frequently Asked Questions

What is zero trust for AI?

Zero trust for AI applies the zero trust model to model and tool traffic: no request is trusted because of its network location or a shared key. Every LLM call and MCP tool call is authenticated, authorized against least-privilege grants, inspected, and logged, usually at a gateway that sits between AI clients and the models and tools they use.

What makes an AI gateway zero trust?

An AI gateway is zero trust when it verifies the caller's identity on every request, grants only specific models and tools, denies anything not granted, avoids shared credentials to upstream servers, inspects prompts and tool calls, and logs each decision. A gateway that allows tools until someone writes a deny rule only partly meets that bar.

How do you secure MCP access in the enterprise?

Route all MCP traffic through a gateway, authenticate callers with your identity provider, grant tools per team or key with deny by default, and reach upstream servers with per-user OAuth or token exchange instead of shared keys. Add guardrails on tool arguments and results, and log every call. The MCP gateway overview explains the pattern.

Is an identity-aware proxy enough for AI traffic?

No. An identity-aware proxy authenticates users before a request reaches an application, but it does not know which models or MCP tools a user should reach, cannot inspect prompts or tool arguments, and does not manage upstream credentials. Many teams place an identity-aware proxy in front of an AI gateway, which then applies model, tool, and content policy.

Can one gateway handle both LLM and MCP traffic under zero trust?

Yes, but coverage varies. Some gateways authenticate and log both traffic types but run guardrails only on LLM requests, or govern MCP servers in a separate product. Check that identity, tool grants, inspection, and audit logging apply to model calls and tool calls alike, so one policy covers an agent's full workflow.

Does zero trust for AI require self-hosting?

Not always, but regulated teams often need it. Hosted gateways process prompts, tool arguments, and credentials on the vendor's infrastructure, while a self-hosted, in-VPC, or air-gapped gateway keeps them inside your network. The guide to AI agent security platforms compares deployment models across adjacent tools.

Apply Zero Trust to AI Traffic with Bifrost

Of the zero trust AI gateways in this guide, Bifrost combines identity from your directory, deny-by-default MCP tool access, per-call token exchange, and guardrails on LLM and MCP traffic in one self-hosted deployment. Explore the Bifrost MCP gateway resources and the AI governance controls, or book a demo to see Bifrost enforce zero trust for AI in your environment.