AI gateways compared: Unity Gateway, Cortex AI Gateway, AgentCore Gateway and Foundry
Databricks, Snowflake, AWS and Microsoft each sell an AI gateway: one endpoint in front of your models and tools that controls who calls what, how much, and what happens when a provider fails. What each one actually does, from its own documentation, and how to choose.
Tecyfy · Published · 9 min read

An AI gateway is one endpoint between your applications or agents and the models and tools they call. It decides who may call which model, caps how much each caller uses, retries or reroutes when a provider fails, and records every call so cost and behaviour can be traced to a person. Databricks, Snowflake, AWS and Microsoft each sell one: Databricks has Unity Gateway, Snowflake has Cortex AI Gateway, AWS has AgentCore Gateway, and Microsoft Foundry has an AI Gateway built on Azure API Management.
They share a name and very little else. Each grew out of the platform underneath it, and that decides more than any feature list.
The short version
- Databricks Unity Gateway covers the most of these jobs for models: external providers, per-user and per-group rate limits, traffic splitting, fallbacks, guardrails and budgets that block, all as Unity Catalog objects.
- Snowflake Cortex AI Gateway is the simplest: one gateway per account, created for you, that puts every request to a Snowflake-served model under the same access, traces and cost reporting. It does not route to providers outside Snowflake.
- Amazon AgentCore Gateway starts from tools: it turns APIs and Lambda functions into MCP tools for agents, and also routes model calls across providers with fallback chains.
- The Foundry AI Gateway is Azure API Management in front of a Foundry resource: token limits and quotas per project, with the rest of API Management's AI policies (load balancing, semantic caching, content safety) a step away.
Pick what yours must do below. Every answer comes from the maker's own pages; "Not in its docs" means we did not find the feature, not that it can never be built.
Databricks Unity Gateway
Databricks · 3 of 3
- YesModels from other providers: OpenAI, Azure OpenAI, Anthropic, Bedrock, Foundry, Gemini Enterprise, custom
- YesAutomatic fallback: Ordered fallbacks on 429 or 5xx, split across up to 5 models
- YesBlock a user at a spend limit: Per-user monthly budgets that block, near real time
Snowflake Cortex AI Gateway
Snowflake · 1 of 3
- NoModels from other providers: Models Snowflake serves (OpenAI, Claude, Llama, Mistral and others), inside Snowflake
- NoAutomatic fallback: Not in its docs
- YesBlock a user at a spend limit: Per-user quotas block within minutes
Amazon AgentCore Gateway
AWS · 2 of 3
- YesModels from other providers: Bedrock, OpenAI, Anthropic and OpenAI-compatible services
- YesAutomatic fallback: Chain of up to 2 fallbacks, with a circuit breaker
- NoBlock a user at a spend limit: Not in its docs
Foundry AI Gateway (API Management)
Azure · 2 of 3
- YesModels from other providers: Foundry models, plus Bedrock and others through API Management
- YesAutomatic fallback: Priority load balancing with a circuit breaker
- NoBlock a user at a spend limit: Token quotas, not money: a quota returns 403
Databricks Unity Gateway covers all 3 as generally available features.
From each maker's documentation, read 7 November 2026. A gateway also sits where your models already run: the fit on paper matters less than the platform your data and bills are on.
Databricks Unity Gateway
Databricks describes Unity Gateway as governance for the runtime traffic between models, agents, MCP servers and tools, built on Unity Catalog. Everything it governs is a Unity Catalog object you grant like a table: the foundation models Databricks serves, external providers, MCP servers, the functions agents call as tools, and the HTTP connections they use.
External providers are model provider services: OpenAI, Azure OpenAI, Anthropic, Amazon Bedrock, Microsoft Foundry, Google Gemini Enterprise or a custom endpoint, each registered once with credentials Databricks encrypts and never returns. Users call the provider through the gateway and never see the key.
On traffic, a model service can split requests by percentage across up to 5 destinations, with session affinity so one conversation stays on one model, and fall back in a set order when a destination returns a 429 or 5xx. Rate limits are requests or tokens per minute for the whole service, a default per user, or a custom limit for a user, service principal or group. There are none until you set them.
Where it goes furthest is money. A Unity Gateway budget can set a monthly threshold per user and block that user when it is reached, enforced in near real time. Databricks is careful about what that means: enforcement runs on an estimate, so a user may be stopped a little early or spend a little past the line, and it is not a hard cap on the final bill. Spend on external providers can count too, in beta, estimated from token counts and the providers' published prices.
Guardrails are service policies on a model or MCP service: built-in ones, your own functions, or a partner vendor's decisions through an external policy.
Snowflake Cortex AI Gateway
Cortex AI Gateway went generally available on 3 November 2026 for inference, observability and cost management. There is nothing to create: each account has one gateway, named SNOWFLAKE, and every role can use it by default because USAGE is granted to PUBLIC. Revoke it if you want to choose who sends traffic.
REVOKE USAGE ON AI GATEWAY SNOWFLAKE FROM ROLE PUBLIC;
GRANT USAGE ON AI GATEWAY SNOWFLAKE TO ROLE app_team;
GRANT MONITOR ON AI GATEWAY SNOWFLAKE TO ROLE cost_admin;
The endpoint speaks the formats clients already use: OpenAI Chat Completions for most models, the Anthropic Messages API for Claude, and the OpenAI Responses API in preview. A coding agent or SDK that lets you change its base URL can point at it. The models are the ones Snowflake serves (OpenAI, Claude, Grok, Llama, Mistral, DeepSeek and Snowflake's own), and inference runs inside Snowflake. Billing is the same as calling Cortex directly: tokens at each model's rate.
What the gateway adds is attribution. Every request is charged to the Snowflake user whose token sent it, and its spans (the models called, timing, tokens, errors) land in a trace table. Snowflake is explicit that the gateway governs the path traffic takes, not which models a user may use; model access still comes from Cortex's own privileges.
For spending, Snowflake offers 2 controls. A shared resource budget covers a team by tag and can notify or run a stored procedure you write, but it is evaluated periodically, so spend can pass the line by up to 6.5 hours before anything runs (1 hour with the low-latency option). A per-user quota blocks the user within minutes. If usage must actually stop, Snowflake's own advice is the quota.
Not to be confused: Snowflake ML has a separate GATEWAY object that splits traffic between model serving endpoints for A/B tests. Cortex AI Gateway governs Cortex inference traffic instead.
Amazon Bedrock AgentCore Gateway
AWS built AgentCore Gateway for agents first. It turns APIs, Lambda functions and existing services into MCP tools, combines many MCP servers into one, handles OAuth into and out of each tool, and lets an agent search thousands of tools by meaning rather than loading them all into its prompt. Our AgentCore guide covers that side.
It now routes model traffic as well. An inference target sends each request to Amazon Bedrock, OpenAI, Anthropic or an OpenAI-compatible service based on the model name in it, translating between the Bedrock, OpenAI and Anthropic formats where needed. Amazon Bedrock Guardrails and AgentCore Policy apply to every call whichever provider serves it.
Its fallback is the most detailed of the 4. A gateway rule pins a model to a target and adds a chain of 1 or 2 alternatives, which can even be a different model, triggered by status codes, connection errors, a slow first byte or an unroutable target. A circuit breaker skips a destination that keeps failing. 2 details matter in production: once a streamed response has started, the gateway can't fall back, and every attempt is billed as a gateway invocation, on top of the model cost.
What its documentation does not describe is a per-user token or spend limit at the gateway. On AWS, limits by person or team are yours to build, and our agent budget limits guide covers building them.
The AI Gateway in Microsoft Foundry
Foundry's AI Gateway is Azure API Management connected to a Foundry resource. You create a new Basic v2 instance or pick an existing v2-tier one, and new projects in the resource go through it by default (existing ones you add by hand). Each project then gets its own tokens-per-minute limit and a token quota: over the minute limit a request returns 429, over the quota it returns 403. There is a free tier for API Management.
Because API Management is underneath, its AI policies are available too. The llm-token-limit policy sets a limit or a quota from an hour to a year on any key, such as the calling app's subscription. Backends can be load balanced round-robin, by weight, by priority (send to your provisioned capacity first) or by session, with a circuit breaker that respects the provider's Retry-After. Prompts can be checked with Azure AI Content Safety, and a semantic cache can return a stored answer for a prompt close enough to an earlier one. API Management also fronts models outside Azure, such as Amazon Bedrock. If you are still deciding between Azure's models and OpenAI's own API, Azure OpenAI vs OpenAI is the place to start.
Limits here count tokens, not money. Turning tokens into a monthly budget per person is your sum.
Side by side
| Unity Gateway | Cortex AI Gateway | AgentCore Gateway | Foundry AI Gateway | |
|---|---|---|---|---|
| Built on | Unity Catalog | Snowflake account | AgentCore | Azure API Management |
| Models from | Databricks, plus OpenAI, Anthropic, Bedrock, Foundry, Gemini and custom | Snowflake-served only | Bedrock, OpenAI, Anthropic, compatible | Foundry, plus others via API Management |
| Fallback | Ordered list | Not in docs | Chain of 1-2, circuit breaker | Priority pools, circuit breaker |
| Limits | Requests and tokens a minute, per user or group | Per-user quota | Not in docs | Tokens a minute and quota, per project or key |
| Stops spend | Per-user monthly budget | Per-user quota | Not in docs | Token quota only |
| MCP tools | Yes | Preview | Core purpose | Preview |
| You pay for | Model tokens | Tokens at each model's rate | Each gateway call, plus models | API Management tier, plus models |
How to choose
Start from where your models and data already are. A gateway is worth most when every call already passes through the platform you govern everything else with: Unity Catalog grants on Databricks, roles on Snowflake, IAM on AWS, Entra and Foundry projects on Azure. Running a second platform's gateway in front of the first means 2 sets of permissions to keep in step.
Then check the 2 things that differ most:
- Do you need models from more than one provider? Cortex AI Gateway stays inside Snowflake. The other 3 route to outside providers, and model routing between them is only worth it with a fallback for when one fails.
- Must a person be stopped at a spend limit? Only Unity Gateway budgets and Snowflake quotas do that as a gateway feature. Foundry stops at a token count, and on AgentCore you build it yourself.
For a platform-free design of the same idea, our LLM model routing guide covers routing in your own code.
Questions
- What is an AI gateway?
- An AI gateway is one endpoint that sits between your applications or agents and the models and tools they call. It decides who may call which model, limits how many tokens or requests each caller uses, routes or retries requests across providers, and records every call for cost and audit.
- What is the Databricks AI gateway called now?
- Databricks calls it Unity Gateway. It is built on Unity Catalog and governs access, rate limits, traffic splitting, fallbacks, budgets and guardrails for model services and MCP services, including external providers such as OpenAI and Anthropic.
- Does Snowflake Cortex AI Gateway support OpenAI and Anthropic SDKs?
- Yes. Its endpoint accepts the OpenAI Chat Completions format, the Anthropic Messages format and, in preview, the OpenAI Responses format, so the official SDKs and coding agents that let you change the base URL can use it. The models are ones Snowflake serves, and inference runs inside Snowflake.
- Is Amazon Bedrock AgentCore Gateway only for MCP tools?
- No. AWS describes it as a gateway for agentic traffic to tools, other agents and models. Its inference targets route model requests to Amazon Bedrock, OpenAI, Anthropic or OpenAI-compatible services by model name, with optional fallback chains.
- How does the AI gateway in Microsoft Foundry work?
- Foundry's AI Gateway runs on Azure API Management. You connect a Foundry resource to a new or existing API Management instance, and each project then gets its own tokens-per-minute limit and token quota. A request over the minute limit returns 429 and one over the quota returns 403.
- Which AI gateway can stop a user at a spending limit?
- From their documentation, Databricks Unity Gateway budgets can block a user at a monthly per-user threshold in near real time, and Snowflake per-user quotas block a user within minutes. Foundry's gateway limits tokens rather than money, and AgentCore Gateway's documentation lists no spend cap.
Sources
Where each fact comes from
- 1AI governance with Unity Gateway
- 2Apply rate limits to model and MCP services
- 3Configure routing and fallbacks for models
- 4Manage budgets for Unity Gateway
- 5Model providers external to Databricks
- 6Cortex AI Gateway
- 7Inference with Cortex AI Gateway
- 8Cost management for Cortex AI Gateway
- 9Cortex AI Gateway inference, observability, and cost management (General availability)
- 10Amazon Bedrock AgentCore Gateway
- 11Inference targets
- 12Model routing and fallback
- 13Amazon Bedrock AgentCore pricing
- 14Configure AI Gateway in your Foundry resources
- 15AI gateway in Azure API Management