No Gateway Yet? How to Give Your AI Agents One Door
This summer at Heron I worked on checking what AI agents actually do. The companies I talked to showed me the question came too early. Agents run on personal Claude Code and Codex subscriptions, and every person wires up their own access: an OAuth grant to an MCP server here, a database password in a config there. There is no single place their actions pass through.
In August I built that place on my own laptop, and this guide is what I learned, updated for what has changed since. The parts are free to run, and a platform engineer can stand up the simple version in days.
What an AI gateway is
One door with two sides. On one side, your agents' tool calls: every read and write to Slack, GitHub, your database or your CRM goes through one box, an MCP gateway. On the other, their model calls to Claude, GPT or whatever your team runs. With both in one door, one place controls access and shows who used what.
You don't need both sides on day one.
Step 1: tools first, and let the keys do the enforcing
The usual mistake is starting with a mandate: "all AI traffic must go through the proxy." Nobody complies, and you get a privacy argument about personal ChatGPT use.
The tool side needs none of that. Move the credentials (the Slack token, the GitHub app, the database password) out of people's dotfiles and into the gateway. Agents get a URL instead of keys. The company's tools now work only through the door, because the keys live there. An agent that skips the door can't reach the data, and personal AI use on personal accounts is untouched.
Access is per person: everyone connects with their own gateway key. When I denied one GitHub tool to one key, that key's catalog showed 43 tools instead of 44: the agent never saw the missing one.
It also helps adoption. For a lot of people, "connect the database yourself" is where the agent experiment ends. With a door, tools are set up once for everyone, including next month's hire.
Step 2: the model side, as strict as you need
Model traffic comes second. Three modes:
Personal stays personal. Do nothing. Only tool calls go through the door.
Per work repo. For Claude Code, commit .mcp.json for the tools and an env block in .claude/settings.json with ANTHROPIC_BASE_URL pointing at the gateway. Commit only the URL; each person's key stays in their own uncommitted settings. One committed file routes every work session through the door, onboards future hires and separates work from personal by folder. It's the right default for most teams. Codex changed here: a project .codex/config.toml can still add MCP servers but now ignores openai_base_url, so Codex model routing lives in each person's ~/.codex/config.toml or managed config.
Machine-wide via MDM. Managed settings pushed to company devices; in Claude Code a managed ANTHROPIC_BASE_URL can't be overridden. Codex has managed config too (requirements.toml); I haven't checked whether it can pin the model endpoint. Only the strict regime needs this, and SOC 2 brings MDM anyway.
Plain claude.ai subscriptions work with both routed modes. Set only ANTHROPIC_BASE_URL, with no gateway credential, and traffic goes through the gateway while the subscription stays the login and keeps paying. The gateway holds no Anthropic key and still logs model and token counts. That's how I ran it.
Picking the door: who owns it and how it keeps up
In my view two things kill a choice within a year. The owner: Portkey, the best-known independent AI gateway, was bought by Palo Alto Networks this spring. And protocol churn: MCP has had four spec revisions since November 2024, and a door that trails the spec breaks your clients. So I choose on these:
- owned by a foundation, not one vendor;
- a permissive license with no seat limit;
- follows the current MCP spec closely;
- carries tool calls and model calls in one process.
My default is agentgateway: Apache 2.0, in the Linux Foundation's Agentic AI Foundation, one process for MCP, A2A and LLM traffic, run as a single binary or on Kubernetes. I tested v1.4.1; v1.7.0 is current. If your platform team already runs Envoy, look at Agent Router, the new name of Envoy AI Gateway since it joined the same foundation in September.
Your clients' configs hold only a URL, so swapping doors later is days of work.
Where the keys live
The door is the same either way; what differs is who holds your SaaS credentials.
Fast build. The gateway plus a cloud credential provider such as Composio. The trade: OAuth tokens for your tools sit with a third party.
Own build. The gateway on your own machine or VM, plus self-hosted Nango for SaaS OAuth (tokens in your own Postgres), plus a secrets store for the rest. I ran a version of this: agentgateway and Nango in Docker, no Kubernetes, no Vault, GitHub's tools behind a token held in the door, Gmail through Nango. The test that mattered: a Gmail call after the one-hour Google token expired still worked, because Nango refreshed it on the call.
License notes: free self-hosted Nango is Elastic-licensed and covers auth and proxy only, which is what the gateway needs. Vault is under the Business Source License; OpenBao is its open-source fork.
Moving from the fast build to the own build swaps the credential layer and never touches the door, so your agents' configs don't change.
What a gateway will not give you
"We installed a gateway, so we have traces" is false on both sides. Model-side gateways see the prompt but are built not to keep it: Anthropic's own gateway logs usage metrics, not prompt or completion content. Tool-side gateways log the call but never had the prompt: MCP has no field for the user's original request.
My install found more gaps. A GitHub permission error came back inside a successful MCP response, so the gateway logged status 200. Errors from APIs behind an OpenAPI target came back marked as not errors. Token usage showed up in logs but in no metric.
So I treat the gateway as plumbing: the place where access is given and taken away. If you later need a record of what agents did and why, the door is where the two streams can be joined, but that is separate work. To see what your current traces can answer, my skill agent-evidence-check is free and sends nothing anywhere.
Where I'd start this week
One gateway binary, the few tools everyone connects by hand with their keys moved into it, a gateway key per colleague, and a committed config in the main work repo. The model side can wait.
An earlier version was published on the Heron blog as No Gateway Yet? How to Give Your AI Agents One Door.