MCP Gateway Observability: Audit Every AI Tool Call
TL;DR
- An MCP gateway is the single point in an agent stack where every tool call from every model and every client can be recorded consistently.
- Bifrost captures input messages, output messages, tool calls, function results, token counts, cost, and latency for every request, and the logging plugin runs asynchronously with no added request latency.
- Audit logs and request logs are two separate records in Bifrost Enterprise: audit logs cover administrative changes and can be HMAC-signed for verification, while request logs and OpenTelemetry traces cover runtime tool execution.
- MCP tool filtering in Bifrost is deny-by-default, so a virtual key with no MCP configuration is granted no tools at all.
- Bifrost adds 11 microseconds of overhead per request at 5,000 requests per second, so recording at the gateway does not become the bottleneck.
An MCP gateway is a control layer that sits between AI agents and every MCP server they can reach, brokering tool discovery, authentication, and execution through one endpoint. Without that layer, tool calls are recorded (if at all) inside each agent, each SDK, and each MCP server separately, which means no team can answer who invoked which tool, with which arguments, on whose behalf. Bifrost, the open-source MCP gateway built in Go by Maxim AI, is designed for enterprises running mission-critical AI workloads that need that record without paying for it in latency. This post covers what an MCP gateway should capture on every tool call, how request logging and audit logging differ, and how to configure both in Bifrost.
What Is an MCP Gateway?
An MCP gateway is a control layer that centralizes authentication, tool discovery, access policy, and execution logging for every MCP server an AI agent can reach. It terminates the agent's connection on one side and holds credentials for upstream tool servers on the other, so policy and telemetry are enforced once rather than per client.
The Model Context Protocol defines how a client and a server exchange tool definitions and invocations. It does not define who may call which tool, how long the record is kept, or where the trace goes. Those are deployment concerns, and the gateway is where they belong.
Bifrost operates as both an MCP client and an MCP server. It connects to external MCP servers over STDIO, HTTP, or SSE, aggregates their tools into one registry, and can expose that registry as a single MCP endpoint to clients such as Claude Desktop or Cursor. For a broader treatment of the pattern, see how an MCP gateway centralizes tool access across an agent fleet.
Why Tool Calls Are the Hardest Part of AI Observability
Tool calls are harder to observe than model calls because they cross a trust boundary. A model call produces text; a tool call writes to a ticketing system, queries a customer database, or pushes a commit. The blast radius is different, and so is the evidence a security team needs.
Three structural problems make ad-hoc logging insufficient:
- The record is fragmented. Each MCP server logs its own invocations in its own format, with no shared request identifier tying them to the conversation that caused them.
- The identity is lost. By the time a request reaches an upstream tool server, it usually carries a shared service credential rather than the end user's identity.
- The arguments are the interesting part. Knowing that
execute_sqlran is close to useless. Knowing which statement ran, against which database, on whose behalf, is the actual audit record.
A gateway resolves all three because it is the one component that sees the model request, the returned tool calls, the execution, and the result in the same context. The distinction between an MCP gateway, an MCP proxy, and an MCP server matters here: only the gateway holds enough context to produce a complete record.
MCP Observability: What to Capture on Every Tool Call
MCP observability means capturing the full lifecycle of a tool call: the model request that produced it, the tool name and arguments, the execution result, the identity that authorized it, and the latency and cost attached to the whole exchange. The Bifrost AI gateway records all of it by default, with no changes to application code.
Built-in observability in Bifrost captures every AI request and response that flows through the gateway, including complete conversation history, model parameters, the provider and model that handled the request, output messages, tool calls, function results, latency, token usage, and success or error status. The logging plugin operates asynchronously, so capture does not sit in the request path.
Three signal types cover different questions:
| Signal | What it answers | Where it goes in Bifrost |
|---|---|---|
| Request logs | What exactly was called, with what arguments, and what came back | Built-in log store (SQLite, Postgres, or ClickHouse) |
| Traces | How a tool call fits into the wider agent workflow and where the time went | OpenTelemetry OTLP export |
| Metrics | How the fleet is behaving in aggregate: throughput, error rate, cost | Prometheus scraping or Push Gateway |
Bifrost exports traces in the genai_extension format, following OpenTelemetry GenAI semantic conventions, so LLM and tool spans correlate with an existing application trace instead of sitting in a separate silo. Requests sharing an x-bf-session-id header group into one trace with group_traces_by_session, which turns a scattered sequence of tool calls into a readable agent run.
Where content must not leave the gateway, disable_content_logging drops message content from exported spans while keeping the structural record, and disable_root_span_content drops it from the root span only.
MCP Governance: Controlling Which Tools Can Run
MCP governance is the set of controls that decide which tools a given caller may see and execute, evaluated before the model is ever offered the tool. Bifrost enforces this through virtual keys, tool filtering, and tool groups, so an unauthorized tool never appears in the model's tool list.
Virtual keys are the primary governance entity. Each one carries model and provider access rules, independent budgets, token and request rate limits, and an optional team or customer association. Callers authenticate with a virtual key using the x-bf-vk, Authorization, x-api-key, x-goog-api-key, or api-key header, which means an existing OpenAI-style or Anthropic-style client works unchanged.
MCP tool filtering is deny-by-default. A virtual key with no MCP configuration is granted no MCP tools at all, and tools become available only when a client and its permitted tool list are configured explicitly. Bifrost generates an x-bf-mcp-include-tools allow-list header from that configuration, unless the caller already sent the header or disable_auto_tool_inject is set. Either way the virtual key's restrictions are enforced twice: once at inference time and again at tool execution time.
At larger scale, MCP tool groups in Bifrost Enterprise replace per-key configuration with reusable bundles. A group is a named collection of tools drawn from one or more MCP servers, attachable across six dimensions: virtual keys, teams, customers, users, LLM providers, and API keys. When several dimensions match a request, the tools merge, and matching happens against an in-process index rather than a database lookup, so it adds no request latency. Further patterns for structuring these controls are covered in the Bifrost governance resource page.
Audit Logs and Request Logs Are Not the Same Record
Audit logs and request logs answer different questions and are stored separately in Bifrost Enterprise. Audit logs record administrative activity: who changed a configuration, when, and which resource was affected. Request logs record runtime traffic, including every tool call and its result. Compliance programs generally need both.
Conflating the two leaves gaps in an evidence package. A SOC 2 auditor asking "who granted this team access to the production database tool" needs an audit log entry. An incident reviewer asking "what did the agent send to that tool at 14:07" needs a request log.
| Audit logs | Request and tool-call logs | |
|---|---|---|
| Records | Administrative actions on Bifrost resources | Model requests, tool calls, arguments, results |
| Integrity | HMAC-signed entries, verifiable with a configured key | Structured records in the log store |
| Retention | retention_days in the database, configurable |
client_config.log_retention_days |
| Long-term storage | Time-windowed JSONL archives in S3 or GCS | Payload offload to S3 or GCS with metadata retained in the database |
| Export formats | JSON, JSON Lines, or Syslog | Object storage keyed per request, queryable from a data lake |
Audit log entries can be signed with an HMAC key of at least 32 bytes, filtered in the dashboard by search text, action, outcome, and date range, and archived periodically to object storage in size-bounded JSONL objects for compliance-grade retention. Request payloads take a different path: log exports stream large fields such as request bodies, response bodies, streamed chunks, and tool call arguments to a bucket while the database keeps searchable metadata and a pointer, which keeps queries fast without shortening retention.
Who can read those records is governed separately. Data access control scopes result sets by role, with own-data, team-data, and all-data scopes applied consistently to dashboard sessions and to inference requests authenticated with a virtual key. Teams working under HIPAA, GDPR, or similar regimes will find the control mapping in the Bifrost Enterprise documentation and in this guide to running an MCP gateway in regulated industries.
MCP Gateway Architecture for High-Volume Deployments
Recording every tool call only works if the recording layer does not become the constraint. Bifrost is written in Go and adds 11 microseconds of overhead per request at 5,000 requests per second in sustained benchmarks, with the logging plugin operating asynchronously outside the request path.
Three architectural properties matter for a gateway that is in the path of every tool call:
- Asynchronous capture. Logging that blocks the request turns an observability requirement into a latency budget problem. Bifrost writes logs off the hot path.
- Horizontal scale with shared state. Clustering provides high availability with automatic service discovery and zero-downtime deployments, so budgets and rate limits stay consistent across nodes.
- Storage that matches retention. The log store supports SQLite, Postgres, and ClickHouse, so a team retaining a year of tool-call records is not forced into the same backend as a team retaining a week.
For deployments where traffic cannot leave a private network, Bifrost supports in-VPC deployment with complete isolation inside the customer's VPC and no external network dependencies, which keeps both the tool traffic and its record inside their own boundary.
MCP Security: Authenticating Every Tool Server Correctly
MCP security starts with the credential each tool server sees. Bifrost supports six MCP authentication types, so the identity attached to a tool call can be the end user rather than a shared service account, which is what makes the resulting log entry attributable.
The available auth types cover the practical cases:
nonefor public servers and local STDIO tools that need no credentialheadersfor a shared API key or bearer token configured once by an administratoroauthfor an administrator-configured OAuth 2.0 token shared by the whole teamper_user_headersandper_user_oauthfor credentials keyed to an individual person, stored against their virtual key, signed-in user, or asserted session IDtoken_exchangefor first-party servers that trust the organization's identity provider, exchanging an identity-provider token on every call without storing it
Execution control is separate from authentication. By default, Bifrost does not automatically execute the tool calls a model returns: they are treated as suggestions, and running one requires an explicit call to the tool execution API. Agent Mode relaxes that for named tools through tools_to_auto_execute, which must be a subset of tools_to_execute, and no tool auto-executes unless it is listed. That default is the right one for the risks catalogued in the OWASP Top 10 for LLM Applications, where excessive agency and insecure output handling both depend on unreviewed tool execution.
Content-level controls sit alongside these. Guardrails integrate providers including AWS Bedrock Guardrails, Azure Content Safety, Gray Swan Cygnal, and Patronus AI, with secrets detection catching API keys and credentials in prompts and completions before they reach a tool server or a log.
Open Source MCP Gateway Options for Self-Hosted Teams
An open source MCP gateway lets a team inspect exactly how tool calls are brokered and recorded, which matters when the gateway is the system of record for an audit. The full source is in the Bifrost repository under an open-source license, with Bifrost Enterprise as a strict superset that adds audit logs, RBAC, clustering, and tool groups.
Self-hosting changes the evaluation criteria. What matters is whether payload storage can point at infrastructure the team already runs, and whether the gateway survives a node failure without losing budget state. Bifrost answers both through the pluggable log store, S3 and GCS payload offload, and gossip-based cluster synchronization.
Getting a local instance running takes one command, and the gateway setup guide covers configuration from there. Teams comparing self-hosted options can review this roundup of open-source MCP gateways and their trade-offs, and the MCP gateway resource page covers the access-control and cost-governance side of the same deployment.
How to Configure MCP Gateway Observability in Bifrost
Configuring MCP gateway observability in Bifrost takes four steps: connect the MCP servers, restrict tools with a virtual key, enable the telemetry destinations, and turn on audit logging if the deployment is Enterprise. Each step is independent, so an existing installation can adopt them in order.
- Connect the tool servers. Register each MCP server over STDIO, HTTP, or SSE, and pick the auth type that preserves end-user identity where the server supports it.
- Create a virtual key with an explicit tool allow-list. Because tool filtering is deny-by-default, list the clients and tools the key may use, and set budgets and rate limits at the same time.
- Point traces and metrics at existing infrastructure. Configure the OTel plugin with a
collector_url,trace_typeofgenai_extension, andprotocolofhttporgrpc; the/metricsendpoint is exposed automatically when the telemetry plugin is enabled. - Enable audit logs and set retention. Set an
hmac_keyof at least 32 bytes and aretention_daysvalue, and configureobject_storagewhen archives need to outlive the database.
One detail is worth checking afterward. The archive_* fields are normalized at runtime rather than rejected at startup, so a malformed duration such as "6 hours" silently falls back to the 24-hour default and archive_interval: "10s" is clamped up to the five-minute floor. Verify the values took effect rather than assuming a bad string would fail loudly.
Coding agents are a common first workload here, generating high tool-call volume against sensitive repositories. Bifrost integrates with Claude Code, Codex CLI, Cursor, and Gemini CLI, and this practical guide to running an MCP gateway with Claude Code walks through the configuration end to end.
Frequently Asked Questions
What is an MCP gateway used for?
An MCP gateway centralizes how AI agents reach MCP servers. It holds credentials for upstream tool servers, enforces which tools each caller may use, executes tool calls under policy, and records every invocation in one place. Without it, authentication, access control, and logging have to be implemented separately in every agent and every tool server.
Does an MCP gateway log every tool call?
Bifrost captures every request and response passing through it, including tool calls, tool arguments, and function results, along with token counts, cost, and latency. The logging plugin runs asynchronously, so capture adds no latency to the request itself.
Can an MCP gateway restrict which tools an agent sees?
Yes. In Bifrost, MCP tool filtering is deny-by-default: a virtual key with no MCP configuration receives no tools. Administrators grant specific clients and named tools per key, and Bifrost enforces the allow-list twice, at inference time and again at tool execution. MCP tool groups apply the same model at fleet scale.
How much latency does an MCP gateway add?
Bifrost adds 11 microseconds of overhead per request at 5,000 requests per second in sustained benchmarks. Logging runs asynchronously and tool-group matching uses an in-process index rather than a database lookup, so neither observability nor governance appears in the request path. Published figures and reproduction steps are in the benchmarking documentation.
Can tool-call logs be exported to an existing data lake?
Yes. Bifrost Enterprise streams large payload fields, including tool call arguments, to S3 or GCS while keeping searchable metadata in the log store, so archived traffic can be queried from a data lake. Audit events archive separately as time-windowed JSONL objects and export as JSON, JSON Lines, or Syslog.
Is Bifrost open source?
Bifrost is open source, written in Go and deployable as a gateway or a Go SDK. Bifrost Enterprise is a strict superset: every open-source provider, plugin, and SDK works identically, with audit logs, RBAC, data access control, clustering, and MCP tool groups added on top.
Get Started with Bifrost
Tool calls are the part of an agent stack with real-world consequences, and an MCP gateway is the only layer positioned to record all of them consistently, attribute them to a person, and stop the ones that should not run. Bifrost provides that layer with deny-by-default tool filtering, asynchronous request logging, OpenTelemetry and Prometheus export, and HMAC-signed audit trails, at 11 microseconds of overhead per request.
To see how Bifrost fits an existing agent and MCP deployment, book a demo with the Bifrost team, or start from the docs overview and run it locally first.
