Top 5 AI Gateways for Controlling Shadow AI

TL;DR

• Shadow AI is AI usage that bypasses approved infrastructure: desktop chat apps, browser AI, coding agents, and MCP servers running on employee machines with no policy layer in between.
• An AI gateway governs only the traffic routed through it, so endpoint coverage is the deciding criterion when comparing gateways for this problem.
• Bifrost is the only gateway in this list that pairs a self-hosted AI gateway (the policy engine) with an endpoint agent, Bifrost Edge, which routes AI traffic from every company machine through that gateway.
• Kong AI Gateway, Cloudflare AI Gateway, Azure API Management, and Gravitee govern traffic that applications send to them, but none publishes endpoint discovery or on-device enforcement for unsanctioned AI apps.
• Bifrost Edge is currently in alpha; the gateway controls it enforces are available today, with virtual keys and budgets in open-source Bifrost and guardrails and audit logs in Bifrost Enterprise.

One in five organizations reported a breach caused by shadow AI in IBM's 2025 Cost of a Data Breach Report, and organizations with high levels of it paid about $670,000 more per breach. Bifrost, the open-source AI gateway built by Maxim AI, is the best choice for enterprises running mission-critical AI workloads that require best-in-class performance, scalability, and reliability, and the one gateway here that extends its policy to employee machines. This guide covers how gateways close that gap, the criteria that matter, and how five options compare.

What Is Shadow AI?

Shadow AI is the use of AI applications, models, and agent tools inside an organization without the approval or visibility of security and IT teams. It includes personal ChatGPT and Claude accounts, desktop AI clients, coding agents in the terminal, and MCP servers wired into those tools, all sending company data to model providers with no policy, budget, or audit trail.

It moves faster than shadow IT: an MCP server takes one line in a config file, and a coding agent takes one install command. Prompts also carry source code, customer records, and credentials directly to a third-party model, the exposure OWASP lists as sensitive information disclosure in its LLM Top 10.

Figure 1: Gateway policy is complete only when the endpoint traffic in the lower lane is routed into it as well.

As Figure 1 shows, a gateway governs only the traffic configured to reach it. For the business exposure, see this overview of shadow AI risks, governance, and security in enterprises; for discovery, this guide to identifying unapproved AI tools across the organization.

How to Detect Shadow AI with an AI Gateway

An AI gateway detects and controls shadow AI by becoming the single path between users and model providers. Once traffic flows through it, every request carries an identity, is checked against a budget and an access policy, passes through guardrails, and is logged. Detection becomes a query over gateway data instead of a survey of employees.

In the Bifrost AI gateway, each step on that path maps to a specific control:

• Identity and access: Virtual keys are the primary governance entity. Each key carries model and provider filtering, so a key issued to a team can reach only the models that team is approved to use.
• Spend and volume: Budgets and rate limits apply hierarchically across virtual keys, teams, and customers.
• Content policy: Guardrails inspect inputs before they reach the provider and outputs before they return, blocking or redacting PII, secrets, and policy violations.
• Evidence: Built-in request logging records inputs, outputs, tokens, cost, and latency for each request.

Figure 2: Every control that answers an audit question sits on this path, so traffic off the path is invisible to all of them.

The table maps common risks to the gateway control for each.

The gap is coverage. Sanctioned applications point their base URL at the gateway; unsanctioned tools, by definition, do not. That is why the governance controls in Bifrost are paired with an endpoint layer, covered below.

Key Criteria for Evaluating AI Gateways for Shadow AI

The most important criterion is whether a gateway can reach traffic that was never configured to use it. After coverage, evaluate identity-bound policy, MCP server visibility, guardrail options, and whether the audit trail separates request data from administrative changes.

MCP visibility deserves separate scrutiny, because a tool wired into an AI client can act on data the user never sees. This article on shadow MCP servers and control at the gateway covers that surface.

AI Gateways Compared at a Glance

Bifrost is the only gateway below that publishes an endpoint agent for governing AI traffic on employee machines. The other four govern traffic that applications send to them; "Not published" means the capability did not appear on the vendor's own pages reviewed for this article.

For a formal evaluation, pair this table with the LLM gateway buyer's guide.

1. Bifrost (AI Gateway + Bifrost Edge)

Bifrost controls shadow AI in two layers. The Bifrost AI gateway is the control plane and policy engine, where virtual keys, budgets, rate limits, guardrails, and audit logs are configured and enforced. Bifrost Edge extends that same governance to every company machine, routing desktop apps, browser AI, coding agents, and MCP servers through the gateway without per-app configuration.

Best for: Bifrost is built for enterprises running mission-critical AI workloads that require best-in-class performance, scalability, and reliability. It serves as a centralized AI gateway to route, govern, and secure all AI traffic across models and environments with ultra low latency. Bifrost unifies LLM gateway, MCP gateway, and Agents gateway capabilities into a single platform. Designed for regulated industries and strict enterprise requirements, it supports air-gapped deployments, VPC isolation, and on-prem infrastructure. It provides full control over data, access, and execution, along with robust security, policy enforcement, and governance capabilities.

The AI gateway: where policy is defined

The Bifrost AI gateway exposes one OpenAI-compatible API across 25+ providers and 10,000+ models, so approving a model is a configuration change. Bifrost adds 11 microseconds of overhead per request at 5,000 RPS. Policy is configured once, in the gateway, with identity, guardrails, and audit logs provided by Bifrost Enterprise:

• Identity: User provisioning connects the identity provider through OIDC SSO and SCIM 2.0, and role-based access control limits who can change policy.
• Guardrails: Bifrost-managed providers (Prompt Guardrails, Custom Regex, Secrets Detection) run alongside external providers such as AWS Bedrock Guardrails, Azure Content Safety, Google Model Armor, and CrowdStrike AIDR, on LLM calls or MCP tool executions.
• MCP tools: MCP tool filtering creates a strict allow-list of MCP clients and tools per virtual key.
• Audit: Audit logs record administrative activity with HMAC signing and JSON, JSON Lines, or Syslog export; request logs capture the traffic.

Bifrost Edge: extending gateway policy to every machine

Bifrost Edge runs natively on macOS, Windows, and Linux and routes AI traffic at the machine level, with no base URLs to change and no SDKs to swap. Users sign in once through the browser with SSO, which links the machine to the user and syncs their virtual keys and policies; no API keys live on the device. Bifrost Edge is currently in alpha.

Figure 3: Policy is defined once in the gateway; Edge carries it to the endpoint and reports apps and MCP servers back for approval.

As Figure 3 shows, Bifrost Edge enforces the gateway's policies rather than a separate policy set:

• App governance: Administrators decide which AI apps are allowed. Allowed apps run normally with traffic governed through Bifrost; disallowed apps are blocked before any data leaves the machine.
• MCP governance: Edge inventories the MCP servers configured inside each AI app and enforces per-server allow or deny decisions on the device, including for apps that had a server configured before the policy existed.
• Guardrails everywhere: Every configured guardrail applies to endpoint AI, so a prompt typed into ChatGPT on the web is evaluated before it leaves the machine.

Supported applications today include Claude Desktop, ChatGPT desktop, Cursor, Codex, Claude Code, OpenCode, and ChatGPT and Claude on the web. Edge deploys silently through MDM platforms such as Jamf, Microsoft Intune, Kandji, Omnissa Workspace ONE, and JumpCloud.

For regulated environments, the gateway runs inside your VPC or on-prem as part of Bifrost Enterprise.

2. Kong AI Gateway

Kong AI Gateway extends Kong's API gateway with AI plugins for routing, prompt security, and cost control. It suits organizations already running Kong for API management, though its published materials do not cover endpoint discovery of unsanctioned AI apps.

Best for: Platform teams already standardized on Kong Gateway who want AI routing and prompt policies as plugins on existing API infrastructure.

Kong's pages list:

• Routing: AI Proxy and AI Proxy Advanced plugins across multiple providers.
• Prompt security: AI Sanitizer redacts PII; AI Prompt Guard and AI Semantic Prompt Guard block disallowed topics and prompt injection.
• Cost control: AI Rate Limiting Advanced for spend limits and AI Consumer Groups for token budgets.
• MCP and deployment: An AI MCP Server entity exposes APIs as tools; management runs through Konnect with self-hosted, cloud, or Kubernetes data planes.

Kong governs AI traffic that applications are configured to send to it. For dedicated discovery tooling, see this roundup of tools for shadow AI detection and governance.

3. Cloudflare AI Gateway

Cloudflare AI Gateway is a hosted gateway on Cloudflare's network between applications and model providers. It offers analytics, caching, spend limits, and DLP for routed traffic, and is available on all Cloudflare plans.

Best for: Teams already on Cloudflare that want a hosted gateway for application AI traffic without operating infrastructure.

Cloudflare's documentation lists:

• Traffic controls: Caching, Rate Limiting, Spend Limits, Dynamic Routing, Auto Router, and fallbacks.
• Security: Guardrails, Data Loss Prevention (DLP), Authentication, and Bring Your Own Keys.
• Visibility: Analytics on requests, tokens, and cost, plus Logging with Logpush and Custom Costs.

Cloudflare's AI Gateway pages do not describe discovering AI apps on employee devices, and the hosted model matters where prompts must stay inside your network. For the self-hosted pattern, see centralizing every LLM call into one auditable control plane.

4. Azure API Management

Azure API Management delivers AI gateway capabilities as policies on Microsoft's API management service. It fits organizations whose AI workloads run mainly on Azure OpenAI and Microsoft Foundry.

Best for: Azure-centric enterprises governing Azure OpenAI and Foundry workloads through existing API Management instances.

Microsoft's documentation describes:

• Token governance: The llm-token-limit policy sets token quotas per counter key; llm-emit-token-metric sends metrics to Azure Monitor.
• Content safety: Prompt moderation through Azure AI Content Safety.
• Resilience and caching: Load-balanced LLM backends with circuit breakers, plus semantic caching.
• MCP: Exposing REST APIs as MCP servers and passing through to existing ones.

Foundry's AI gateway inventories agents and MCP tools registered with it; unregistered usage on employee machines is outside that published scope. See how Bifrost Edge and the Bifrost gateway deploy together to cover both layers.

5. Gravitee AI Gateway

Gravitee's AI gateway, part of Gravitee Agent Management, governs LLM, MCP, and agent-to-agent traffic with one policy model, as SaaS, self-hosted, or hybrid.

Best for: Organizations building internal agent platforms that want LLM, MCP, and A2A traffic under one API management product.

Gravitee's page lists:

• LLM Proxy: Traffic management for Anthropic, Bedrock, OpenAI, Gemini, and Vertex, with PII filtering, prompt guardrails, semantic caching, token rate limiting, and model routing.
• MCP Proxy: JSON-RPC 2.0 tool discovery and invocation, with MCP server mediation and method-level access control.
• Identity: Shared OAuth 2.1 authentication across LLM, MCP, and A2A traffic.
• Observability: OpenTelemetry traces with policy decisions and cost attribution by team or agent.

Gravitee's page does not describe endpoint discovery of AI apps or MCP servers on user machines. For the MCP side, see how Bifrost as an MCP gateway handles access control and cost governance.

How to Roll Out Shadow AI Controls Without Blocking Work

The rollout that works is visibility first, policy second, enforcement third. Blocking every unknown AI tool on day one pushes usage to personal devices; discovering what is in use, approving what meets policy, and denying the rest keeps employees on governed paths.

With AI Gateway + Bifrost Edge, that sequence maps to concrete steps:

1. Stand up the gateway policy first. Configure providers, virtual keys, budgets, and guardrails in Bifrost, and connect the identity provider so keys map to people and teams.
2. Deploy Edge and discover. Push Edge through MDM. New apps and MCP servers enter a pending state, and admins configure whether pending items are allowed or blocked.
3. Review the inventory. The Devices dashboard lists every machine with its owner, installed AI apps, and configured MCP servers.
4. Decide once per tool. The Approvals dashboard deduplicates apps and MCP servers across the fleet, so one decision applies everywhere at each device's next check-in.

Figure 4: Discovery feeds one deduplicated catalog, so a single approve or deny decision applies across the fleet at the next check-in.

Approved tools run under gateway policy, which makes approval safe to grant. This breakdown of seven exposure categories and the control for each helps sequence which policies to configure first.

Frequently Asked Questions

What is a shadow AI tool?

A shadow AI tool is any AI application, model API, coding agent, or MCP server used for work without approval from security or IT. Examples include personal ChatGPT or Claude accounts, desktop AI clients installed by individuals, coding agents running on personal API keys, and MCP servers added to an AI client's configuration. The tool may be reputable; the problem is that its data flows have no policy, budget, or audit trail.

Why is shadow AI a problem?

Shadow AI sends company data to third-party models with no inspection or record. Prompts routinely contain source code, customer data, and credentials, and without a gateway there is no guardrail to redact them and no log to investigate afterward. The exposure extends to uncontrolled spend and to MCP servers acting on internal systems without review, as this guide to enterprise AI risk and governance details.

How to block shadow AI?

Route all AI traffic through a governed path and deny what does not meet policy. An AI gateway enforces identity, budgets, and guardrails for routed traffic, while an endpoint layer such as Bifrost Edge app governance blocks denied apps and MCP servers on the device. Network-level blocking alone tends to push usage to personal devices.

What are the statistics on shadow AI?

IBM's 2025 breach research found that one in five organizations reported a breach due to shadow AI, with about $670,000 in added cost where usage levels were high. It also found that 63% of breached organizations lacked an AI governance policy or were still developing one, and that 97% of organizations with AI-related breaches lacked proper AI access controls.

What is shadow IT in cyber security?

Shadow IT is any software, hardware, or cloud service used without the knowledge or approval of the IT or security team. Shadow AI is a subset of shadow IT focused on AI tools, and it differs in speed and data exposure: AI tools can be adopted with a single install or configuration line, and every prompt can carry sensitive data directly to an external model provider.

Can an AI gateway alone stop shadow AI?

An AI gateway alone governs only the traffic configured to reach it, so it cannot see AI apps that employees install and point directly at providers. Closing the gap requires a gateway as the policy engine plus a way to route endpoint traffic into it. Bifrost combines both: the Bifrost AI gateway defines policy, and Bifrost Edge enforces it on each machine.

Is Bifrost Edge generally available?

No. Bifrost Edge is currently in alpha, and organizations request access through the Bifrost Edge overview page. The controls Edge enforces, including virtual keys, budgets, rate limits, guardrails, and audit logs, are configured in the Bifrost AI gateway, so teams can establish policy in the gateway now and extend it to endpoints as Edge onboarding proceeds.

Try Bifrost Today

Shadow AI grows wherever AI traffic can reach a model without passing through policy. Bifrost closes that path in two layers: the AI gateway defines identity, budgets, guardrails, and audit, and Bifrost Edge carries those controls to the desktop apps, browser AI, coding agents, and MCP servers on every machine. To see AI Gateway + Bifrost Edge control shadow AI across your fleet, book a demo with the Bifrost team.