Salesforce AI Gateway: Complete 2026 Guide | SF Dictionary

Your org has eight agents running in production. Three of them call GPT-5. Two call Claude. One calls Atlas, one calls Gemini, and one talks to a private Bedrock deployment that the data science team stood up last quarter. Nobody knows how many tokens each team consumed last month, and finance is asking. Worse, you found out on Friday that the renewals agent had been pasting full account records into an external model with zero masking, because the team that built it never wired up the Einstein Trust Layer.

This is the mess AI Gateway was built to clean up. It went GA in May 2026 as part of the Agent Fabric expansion, and it does one job well: it is the single point that every LLM call passes through on its way out of your Salesforce estate. Atlas, Claude, GPT-5, Gemini, a private model on Bedrock, it does not matter. If the call leaves your estate, it goes through the gateway first.

This guide is for the admins, architects, and developers who now own a fleet of agents and are realizing that "build the agent" was the easy part. Governing what those agents do with customer data, and proving it to an auditor, is the hard part. Let us walk through what AI Gateway actually is, what it controls, how it fits with the rest of Agent Fabric, and what you should do about it this quarter.

Why This Became Urgent in 2026

For about two years, the standard pattern for getting a Salesforce process to talk to an external model was depressingly direct. A developer grabbed an API key, dropped it in a named credential or, worse, a custom setting, and called the model from Apex or Flow. Customer data went straight out the door. No central log. No masking unless someone remembered to add it. No idea what it cost.

That was survivable when one team ran one model for one use case. The blast radius was small. If something went wrong, you knew where to look, because there was only one place to look. It stopped being survivable the moment Summer '26 Multi-Agent Orchestration went live. Suddenly you did not have one agent calling one model. You had a primary agent routing to specialist agents, each grounded on different data, each potentially calling a different provider. The number of distinct paths customer data could take out of your org went from a handful to a combinatorial mess.

Salesforce's own research puts the average enterprise at roughly a dozen agents in production, and that figure climbs fast. Multiply a dozen agents by four or five available models, sprinkle in a few external MCP servers, and the old "each team manages its own keys" approach collapses. You cannot govern what you cannot see, and nobody could see anything.

What AI Gateway Actually Does

Strip away the marketing and AI Gateway is a reverse proxy with opinions. Every model call routes through it, and on the way through, it applies a consistent set of controls that used to be optional and inconsistent. Four of those controls matter most.

Standardized token accounting. Every provider counts tokens differently and bills differently. The gateway normalizes this. One call to Atlas and one call to GPT-5 land in the same ledger, measured the same way, so you can actually compare them. No more reconciling four vendor invoices against four different definitions of a token.

Per-team quotas. You can cap how many tokens the service team burns in a month, separately from the sales team. When a team approaches its limit, the gateway throttles before the overspend happens, not after the invoice arrives. This is the difference between a budget and a hope.

PII redaction before egress. This is the big one. The gateway can strip or mask personally identifiable information out of a prompt before it ever reaches an external model. The renewals agent pasting full account records into GPT-5 is no longer a thing that can quietly happen, because the masking is enforced at the chokepoint, not left to whichever developer remembered.

A single audit log. One log, one schema, every call. When legal asks "show me every time customer data was sent to a third-party model in Q1," you have one place to answer that, instead of stitching together debug logs from six teams.

The key design decision is that none of this is opt-in at the agent level. The agent does not get to choose whether to follow policy. The gateway sits in the path, so the policy applies whether the developer thought about it or not. That is the entire point.

A small detail with big consequences: the gateway also abstracts the provider. Your agent does not need to hardcode an endpoint, an API key, or a vendor-specific request shape. It calls the gateway, and the gateway decides where the request goes. That sounds like a convenience, and it is, but it is also a governance lever. When a model gets deprecated, or a cheaper one appears, or a provider has an outage, you reroute at the gateway. The agents do not change. You will appreciate this the first time a model version is retired with two weeks notice and you do not have to touch forty Flows to handle it.

LLM Governance: Numbers Finance Can Actually Use

Inside AI Gateway sits a capability called LLM Governance, and it is the piece that turns a security tool into a budgeting tool. Because every call carries a Trusted Agent Identity, the gateway knows which team, which agent, and which business unit generated each token. That means cost can be attributed, not estimated.

If you have ever tried to answer "what does our AI actually cost the service org versus the sales org," you know the current answer is usually a shrug and a spreadsheet. LLM Governance replaces the shrug with a real number. Finance gets cost per business unit, broken out by model, updated continuously. The CFO stops asking IT to guess.

This matters more than it sounds. The first wave of agent deployments was funded as experiments, on innovation budgets nobody scrutinized. The second wave has to justify itself line by line. An org that can show "the service agent costs us eleven cents per resolved case and deflects forty percent of tier-one tickets" wins the budget fight. An org that can only say "AI cost us something, we think around fifty grand" does not. LLM Governance is how you become the first kind of org.

How It Works With the Einstein Trust Layer

A fair question at this point: does AI Gateway replace the Einstein Trust Layer? It does not. They stack.

Every call that goes through AI Gateway also passes through the Trust Layer. The Trust Layer handles the data-side guarantees you already know: PII masking, toxicity scoring on responses, zero data retention agreements with model providers, and its own audit logging. AI Gateway handles the operational layer on top: which team, which quota, which model, what did it cost, is this team even allowed to use this provider.

Think of it as two checkpoints on one road. The Trust Layer asks "is this data safe to send and is this response safe to return." The gateway asks "is this team allowed to make this call, do they have budget left, and is it logged for cost attribution." A prompt gets masked on the way out and the response gets scored on the way back, and the whole round trip is recorded once.

Where AI Gateway Sits in Agent Fabric

AI Gateway is not a standalone product. It is one of four pillars in Agent Fabric, the control plane Salesforce built to keep a fleet of agents in line. The InfoWorld coverage of the launch lays out the full set, and it is worth knowing how the pieces relate.

• AI Gateway (GA): the model-call control plane this guide is about.
• MCP Bridge (GA): turns your existing APIs into tools agents can call through the Model Context Protocol, without rewriting them. If you want the full picture, see our Salesforce MCP guide.
• Trusted Agent Identity (GA): gives every agent a verifiable identity so its actions can be attributed and permissioned. This is what makes per-team cost attribution possible in the first place.
• Agent Broker (beta): the intelligent router that decides which agent or model handles a given request.

AI Gateway also plugs into MuleSoft through MuleSoft Agent Fabric, which extends the same governance reach to agents and APIs living outside the core Salesforce platform. The point of bundling these is that governance only works if it is comprehensive. A control plane with gaps is not a control plane, it is a suggestion. The Salesforce Engineering team makes a related argument in its writeup on guided determinism and the Agent Graph: predictable agent behavior depends on controlling the paths agents can take, not just hoping they behave.

The Competitive Angle: Why It Ships Free

Here is the part worth being blunt about. AI Gateway is positioned directly against standalone LLM gateway vendors like Portkey and Helicone, and against Databricks Unity AI Gateway. Those are real products that do real work, and some of them have a head start on raw features.

Salesforce's move is to ship AI Gateway as part of the platform rather than as a paid add-on. That is the competitive moat, and it is a sharp one. If you are already on Salesforce and you already run Agentforce, the gateway is sitting right there, already wired into your data, your identities, and your Trust Layer. Adopting a third-party gateway means standing up another vendor, another contract, another integration, another thing to secure. Adopting the native one means flipping it on.

The Futurum Group analysis makes the smart caveat here, and I agree with it: bundling buys adoption, but the real test is enforcement. A gateway that is easy to turn on but easy to bypass solves nothing. The value lives entirely in whether the policies are actually mandatory, whether teams can route around them, and whether the determinism holds under load. That is the bar to hold Salesforce to, not the feature checklist. Free is a strong opening move, but governance is judged on whether it actually governs.

My honest take: for shops already deep in the Salesforce ecosystem, the native gateway wins on integration alone, even if a specialist vendor edges it on a feature or two. The reason is friction. The thing about a free, already-installed control plane is that people actually use it, and a control plane only matters to the extent it is used. A best-in-class vendor gateway that half your teams route around is worse than a decent native one that all of them go through. For shops with a heavily multi-cloud, multi-platform AI footprint where Salesforce is one of many systems, a vendor-neutral gateway may still make more sense, because it can govern the parts of your estate that have nothing to do with Salesforce. Know which shop you are before you decide.

What to Do This Quarter

Enough theory. Here is the concrete next step. Before you do anything else, run an inventory: find every place in your org where an external model gets called today. Search your named credentials, your Apex callouts, your Flow HTTP actions, and any MCP server registrations. Build a list of every model, every team that owns a call, and whether masking is currently applied.

That list will be uglier than you expect. There will be a call nobody remembers writing, and at least one that sends data it should not. That ugly list is your business case. Take it to whoever owns your Agentforce rollout, route every one of those calls through AI Gateway, set a per-team quota on each, and confirm PII redaction is on for every external provider. Then pull the LLM Governance cost-per-business-unit report and send it to finance before they ask. Being the person who already has the number is a good position to be in.

About the Author

Dipojjal Chakrabarti is a B2C Solution Architect with 29 Salesforce certifications and over 13 years in the Salesforce ecosystem. He runs salesforcedictionary.com to help admins, developers, architects, and cert/interview candidates sharpen their fundamentals. More about Dipojjal.