Top 5 Tools to Stop API Keys and Secrets Leaking Into LLM Prompts

TL;DR

• API keys and secrets reach LLM prompts through pasted code, coding agents reading config files, debugging logs, and MCP tool results, usually without anyone intending to share them.
• Secrets detection for LLM traffic scans prompts and responses for credentials and blocks or redacts them before they reach a model provider.
• Bifrost runs secrets detection in the request path with 222 Gitleaks rules, so one guardrail covers every provider and app routed through it, including coding agents.
• Platform-specific filters such as AWS Bedrock Guardrails, Google Model Armor, and Cloudflare AI Gateway DLP cover traffic sent through those services.
• Gitleaks catches secrets in source code before they are pasted into a prompt, and works best alongside a runtime guardrail.

Secrets leak into LLM prompts every day because developers paste code, configs, and stack traces into chat tools, and coding agents read whatever files sit in a repository. Bifrost, the open-source AI gateway built by Maxim AI, is the best choice for enterprises running mission-critical AI workloads that require best-in-class performance, scalability, and reliability, with secrets detection applied to every request before it leaves the network. This guide compares five tools that stop API keys and secrets leaking into LLM prompts and explains which leak path each one closes.

Why Secrets Leak Into LLM Prompts

Secrets leak into LLM prompts because credentials are already scattered across code, configs, and logs, and AI tools now read those sources directly. Once a key is in a prompt, it is transmitted to a third party and may be stored in provider logs and in your own request logs.

The scale of the underlying problem is large. GitGuardian's State of Secrets Sprawl 2025 report counted 23.8 million new secrets on public GitHub in 2024, and found that 70% of secrets leaked in 2022 were still active. The same sprawl now flows into AI tools through four common paths.

Figure 1: Most secrets reach a model as a side effect of normal work, not through deliberate sharing.

• Pasted code and config: developers paste files into chat apps to ask for help, including connection strings and keys.
• Coding agents: agents such as Claude Code read repository files, including .env files, and send their contents as context.
• Logs and stack traces: debugging prompts often include headers, tokens, and URLs with embedded credentials.
• MCP tool results: tools that read files or call APIs can return secrets that the agent then passes to the model.

Our guide to securing MCP servers and stopping secret exfiltration covers the tool-result path in depth.

What to Look for in Secrets Detection for LLM Traffic

Effective secrets detection for LLM traffic scans every prompt and response in the request path, recognizes a broad set of credential formats, and can block or redact without breaking the workflow. It should also keep secrets out of logs, because a log full of keys is a second leak. These criteria follow the same model as PII filtering at the AI gateway layer, and fit within a broader AI governance program from policy to runtime enforcement.

Secrets Detection Tools Compared at a Glance

The table below compares the five tools by where they run, what credential coverage they publish, and what actions they take. Entries for third-party tools reflect each vendor's published documentation. For the wider security picture, see our guide to LLM gateway security for prompt injection, PII, and audit.

1. Bifrost

The Bifrost AI gateway is an open-source AI gateway that applies secrets detection to LLM requests and responses in the request path, before any provider receives them. Its Secrets Detection guardrail runs entirely inside Bifrost using the 222 default rules from Gitleaks v8.30.1, with no external service to call.

Best for: Bifrost is built for enterprises running mission-critical AI workloads that require best-in-class performance, scalability, and reliability. It serves as a centralized AI gateway to route, govern, and secure all AI traffic across models and environments with ultra low latency. Bifrost unifies LLM gateway, MCP gateway, and Agents gateway capabilities into a single platform. Designed for regulated industries and strict enterprise requirements, it supports air-gapped deployments, VPC isolation, and on-prem infrastructure. It provides full control over data, access, and execution, along with robust security, policy enforcement, and governance capabilities.

How Bifrost secrets detection works

Secrets Detection is a Bifrost Enterprise guardrail provider that scans text in the input, output, or both phases of a request:

• Coverage: cloud provider credentials, GitHub and GitLab tokens, package registry tokens, LLM provider keys (OpenAI, Anthropic, Cohere, Hugging Face), payment keys such as Stripe, private keys, JWTs, and generic API key patterns.
• Actions: detect_only, block (the default), or redact, with replace, mask, or hash redaction strategies.
• Log protection: redaction modes can redact the live request and its logs (runtime), the logs only (logs_only), or both with reversible placeholders (runtime_reversible).
• False positives: an ignored_secret_keywords allowlist suppresses known test values.

Figure 2: Scanning at the gateway catches secrets from every app before any provider receives them.

A configuration that redacts secrets before the provider sees them looks like this (logs_only mode would instead keep raw text in the live request and redact only stored logs):

curl -X POST http://localhost:8080/api/guardrails/secrets \
  -H "Content-Type: application/json" \
  -d '{
    "name": "block-leaked-credentials",
    "enabled": true,
    "config": {
      "ignored_secret_keywords": ["example", "dummy", "sample-token"],
      "action": "redact",
      "redaction_strategy": "replace",
      "redaction_mode": "runtime"
    }
  }'

The configuration is then attached to a guardrail rule that decides which traffic it applies to. Because guardrail rules can also target MCP tool calls, the same detection applies to tool arguments and results through Bifrost as an MCP gateway.

Coverage for coding agents and desktop AI

Coding agents are a fast-growing leak path, and they often run on developer laptops rather than through application code. Bifrost Edge, currently in alpha, extends Bifrost's guardrails to AI traffic on employee machines, including Claude Code, Cursor, ChatGPT, and Claude on the web. With Bifrost Edge security and guardrails, a key pasted into a browser chat is caught before it leaves the machine. Our article on connecting Claude Code to internal MCP servers without exposing credentials covers the agent side.

2. AWS Bedrock Guardrails

AWS Bedrock Guardrails provides sensitive information filters that detect credentials and personal data in prompts and model responses for models on Amazon Bedrock. The filters work across natural language and code, including hardcoded credentials in code snippets.

AWS's published capabilities include:

• Built-in entity types including AWS_ACCESS_KEY, AWS_SECRET_KEY, and PASSWORD
• Custom regex patterns for other credential formats, such as internal token prefixes
• Block or mask actions, set separately for inputs and outputs
• Detection in code syntax, comments, and string literals

AWS notes two limits worth planning for: the filter evaluates text content but not tool call arguments or tool results, and Bedrock model invocation logs retain the original unmasked input. Bifrost also supports AWS Bedrock Guardrails as a guardrail provider, so teams can apply the same Bedrock policies to traffic for other providers.

Best for: AWS-native teams whose LLM traffic runs on Bedrock.

3. Google Model Armor

Google Model Armor screens prompts and responses for prompt injection, malicious URLs, harmful content, and sensitive data through its Sensitive Data Protection integration. Its basic configuration covers a fixed set of credential and financial infoTypes, and advanced configuration uses custom inspection templates.

Google's published capabilities include:

• Basic sensitive data screening for GCP_CREDENTIALS, GCP_API_KEY, PASSWORD, and financial identifiers
• Advanced screening with Sensitive Data Protection inspect and de-identify templates
• Prompt injection and jailbreak detection, malicious URI filtering, and responsible AI filters

Basic mode covers Google Cloud credentials and passwords but not most third-party API keys, so broader credential coverage requires advanced templates. Bifrost Enterprise supports Google Model Armor as a guardrail provider, which applies existing Model Armor templates to traffic for any provider.

Best for: Google Cloud teams that already manage Sensitive Data Protection templates.

4. Cloudflare AI Gateway DLP

Cloudflare AI Gateway is a managed edge gateway, and its Data Loss Prevention feature scans prompts and responses routed through Cloudflare using the same detection profiles as Cloudflare One DLP. It is listed as a beta feature in Cloudflare's AI Gateway documentation.

Cloudflare's published capabilities include:

• A predefined Credentials and Secrets profile covering AWS keys, Azure API keys, Google Cloud keys, SSH keys, and Cloudflare API keys and tokens
• Custom profiles for organization-specific patterns
• Block actions that replace the provider response with an error, or flag actions that log findings
• Scanning of request and response text, with full buffering of streamed responses when response scanning is on

The Credentials and Secrets profile does not list GitHub tokens or general private keys, so teams with broader needs add custom profiles. See our comparison of Cloudflare AI Gateway alternatives for self-hosted options.

Best for: teams already routing AI traffic through Cloudflare.

5. Gitleaks

Gitleaks is an open-source secret scanner, released under the MIT license, that finds credentials in source code and git history. It does not inspect LLM prompts, but it removes secrets from code before developers or coding agents paste that code into a prompt.

Gitleaks fits the workflow in three places:

• Pre-commit hooks that stop a secret from being committed
• CI pipelines that fail a build when a secret appears
• Repository audits that find secrets already in git history

Gitleaks and Bifrost share a rule set: the Bifrost Gitleaks-based Secrets Detection guardrail uses the Gitleaks default rules, so a credential format that Gitleaks catches in a commit is the same one Bifrost catches in a prompt.

Best for: engineering teams reducing the number of secrets available to leak in the first place.

Building a Layered Defense Against Secrets in Prompts

The most reliable defense combines secret scanning in source code, a runtime guardrail in the request path, and endpoint coverage for desktop and coding-agent traffic. Each layer closes a different leak path, and a self-hosted gateway is the only layer that sees every provider and app your teams use, including coding agents on laptops through Bifrost Edge.

Figure 3: Each layer catches a different leak path; the gateway layer sees every provider and app routed through it.

A practical rollout:

• Scan code with Gitleaks in pre-commit and CI to reduce secrets in repositories.
• Route LLM traffic through Bifrost with a drop-in SDK base URL change and a Secrets Detection rule on inputs, using redact for chat workflows and block for automated pipelines.
• Extend coverage to laptops with Bifrost Edge so desktop apps and coding agents pass through the same guardrail.
• Keep platform filters such as Bedrock Guardrails or Model Armor where a team already relies on them.
• Redact logs so detected secrets never persist in request logs or exported traces.

Our guide to stopping AI from leaking PII or unsafe output covers the related PII controls, and the Bifrost governance overview shows how guardrails fit with virtual keys and budgets.

How to Choose a Secrets Detection Tool

Choose a secrets detection tool by the path your LLM traffic takes. If traffic spans several providers or comes from many apps and agents, a gateway guardrail covers it all in one place; if traffic stays on one cloud platform, that platform's filter may be enough for that traffic.

Figure 4: Platform filters cover traffic sent through their service; a self-hosted gateway covers every provider your teams call.

For regulated environments, Bifrost Enterprise supports in-VPC deployment so prompts, detections, and logs stay inside your network. Our list of enterprise AI guardrails for company policy and our explainer on MCP guardrails for tool calls cover other policies to run alongside secrets detection.

Frequently Asked Questions

What is secrets detection for LLMs?

Secrets detection for LLMs scans prompts, responses, and tool calls for credentials such as API keys, access tokens, and private keys, then blocks or redacts them before they reach a model provider. It applies the same pattern-matching approach used in source code scanning to AI traffic in real time.

How do API keys end up in ChatGPT and other AI tools?

API keys reach ChatGPT and other AI tools when people paste code, configuration files, or logs into a chat, and when coding agents read repository files such as .env files as context. MCP tools that read files or call APIs can also return secrets that are then sent to the model.

Should secrets be blocked or redacted?

Redact secrets in interactive chat and coding workflows, so the user's request still succeeds without the credential. Block requests in automated pipelines where a secret indicates a misconfiguration that should fail loudly. Bifrost supports both, plus a detect-only mode for measuring exposure before enforcing.

What is Gitleaks?

Gitleaks is an open-source tool, released under the MIT license, that scans source code and git history for secrets such as API keys and tokens. It typically runs in pre-commit hooks and CI. Bifrost's Secrets Detection guardrail uses the Gitleaks default rule set to scan LLM prompts and responses at runtime.

Do provider-side guardrails stop secrets in prompts?

Provider-side guardrails stop secrets only for traffic on that provider's platform, and their built-in credential coverage is narrower than a dedicated secret rule set. AWS Bedrock Guardrails, for example, ships AWS key and password entity types and relies on custom regex for other formats. A gateway guardrail covers every provider at once.

Does a secrets detection guardrail add latency?

Pattern-based secrets detection runs in-process and scans text, so its cost is small compared with model inference. Bifrost runs Secrets Detection inside the gateway with no external service call, and Bifrost itself adds 11 microseconds of overhead per request at 5,000 RPS (benchmarks).

Stop Secrets at the Gateway with Bifrost

Secrets detection works best where every prompt passes: the AI gateway. Bifrost scans LLM requests and responses against 222 Gitleaks rules, redacts or blocks credentials before any provider sees them, and extends the same guardrail to coding agents and desktop AI through Bifrost Edge. To see secrets detection on your own traffic, book a demo with the Bifrost team.