AgentCore Gateway: one endpoint for tools, agents and models

Amazon Bedrock AgentCore Gateway is one authenticated endpoint for everything an AI agent calls: its tools, the other agents it hands work to, and the models it thinks with. Behind that endpoint it turns your APIs and Lambda functions into a single MCP server, forwards traffic to other agents, and sends each model request to Amazon Bedrock, OpenAI or Anthropic depending on the model name. Since 22 October 2026, in every Region with Gateway except those in China, it can also move a failed model request to another provider on its own.

It is one service of Amazon Bedrock AgentCore, and it works without the others: the agent can run on AgentCore Runtime, on Lambda, in a container or on a laptop.

3 kinds of target, and they behave differently

A gateway has targets, the backends it reaches, and every target falls into one of 3 groups. Which group decides how clients address it and what the gateway does to the traffic.

The default limits are generous: 100 targets per gateway, 1,000 tools per target, and 15 minutes for a single invocation, all adjustable.

Tools: many APIs become one MCP server

For MCP targets the gateway is an MCPModel Context ProtocolAn open standard for connecting AI applications to tools and data. A system offered once as an MCP server works in any application that speaks MCP. server whose tool list is the sum of every target's tools. The agent connects once and sees them all, each renamed as targetName___toolName with 3 underscores: a tool get_order behind a target called orders is orders___get_order to the agent. Code that routes on tool names, such as a Lambda handler serving several tools, has to strip the prefix.

Hundreds of tools in one list is its own problem, because every tool description costs tokens in every request and makes the model's choice harder. In the Regions that support it, turn on semantic search when you create the gateway and the agent can call a built-in tool, x_amz_bedrock_agentcore_search, with a sentence describing the task, and get back only the tools that fit. Our piece on MCP tool bloat explains why that matters for accuracy as well as cost.

A remote MCP server can be a target too. The gateway synchronizes its tools, prompts and resources, and with MCP sessions and response streaming turned on it passes through progress messages, elicitation (the server asking the user a question mid-call) and sampling (the server asking the client's model for a completion).

Models: the model name picks the provider

For inference, point the SDK you already use at the gateway instead of the provider. The gateway authenticates the caller, then calls the provider with the key it holds, so no agent ever sees an OpenAI or Anthropic key.

from openai import OpenAI
import anthropic

GATEWAY = "https://<gateway-id>.gateway.bedrock-agentcore.us-west-2.amazonaws.com"

openai_client = OpenAI(base_url=f"{GATEWAY}/inference/v1", api_key="<gateway token>")
reply = openai_client.chat.completions.create(
    model="openai/gpt-5.5",  # prefix = target name: always the openai target
    messages=[{"role": "user", "content": "Summarise ticket 4182"}],
)

claude = anthropic.Anthropic(base_url=f"{GATEWAY}/inference", api_key="<gateway token>")
answer = claude.messages.create(
    model="anthropic/claude-sonnet-5-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Draft the reply"}],
)

The gateway translates between the Bedrock, OpenAI and Anthropic request formats, so a client written for one can reach a provider that speaks another. Since 22 October the Bedrock InvokeModel and InvokeModelWithResponseStream APIs work through it too, so existing Bedrock code needs only a new endpoint. GET /inference/v1/models lists every model on every target.

The routing trap: an unprefixed name can land anywhere

A model name without a prefix, such as gpt-5.5, is matched against every inference target. If one matches, it goes there. If several match equally, the gateway prefers the Bedrock target when there is one; otherwise it picks one at random on each request. With an OpenAI target and an OpenAI-compatible proxy both serving the same name, identical requests can go to different providers, with different prices, data terms and rate limits.

Two fixes: prefix the model with the target name, as in the code above, or write a gateway rule that pins the model to a target. Rules are also how you run a canary or an A/B test: each has a priority from 1 to 1,000,000, lowest first, and can match on the model, the caller's IAM principal or the request path. The general idea is on our model routingModel RoutingSending each request to the cheapest model that handles it well, by rules, a classifier or a cascade, and checking with an eval that quality holds. page, and choosing which model gets which request covers the cost side.

Fallback: when the provider says no

Model fallback, added on 22 October, keeps an agent answering when a provider throttles it, runs out of capacity or stops responding. You add a fallback block to a rule's route: up to 2 more destinations, each a target and optionally a different model, and the conditions that make the gateway move on.

aws bedrock-agentcore-control create-gateway-rule \
  --gateway-identifier my-gateway-abc1234567 \
  --priority 100 \
  --conditions '[{"matchModels": {"anyOf": [{"value": "gpt-5.5"}]}}]' \
  --actions '[{"routeToTarget": {"staticRoute": {
      "targetName": "bedrock-mantle", "model": "gpt-5.5",
      "fallback": {
        "triggerOn": {"statusCodes": ["5XX", "429"],
                      "errors": ["CONNECTION_ERROR", "FIRST_BYTE_TIMEOUT", "UNROUTABLE"]},
        "firstByteTimeoutMilliseconds": 5000,
        "targetFallback": [
          {"targetName": "openai"},
          {"targetName": "bedrock-mantle", "model": "gpt-5.4"}
        ]}}}}]'

This is AWS's own example: gpt-5.5 on Bedrock first, the same model from OpenAI next, and only then the older gpt-5.4. The order says what you value. Putting the older model second instead would keep the traffic on one provider and accept a weaker answer before paying another company.

Four behaviours decide whether fallback does what you expect:

• Only the listed failures move the request. A 400 for a malformed request is not a trigger in this rule, so it goes straight back to the caller. Status codes must be in the 4xx or 5xx ranges; a success is never retried elsewhere.
• A stalled provider needs a first-byte timeout. A provider that accepts the connection and then says nothing returns no error to react to; FIRST_BYTE_TIMEOUT turns that silence into a trigger. Here the primary gets 5 seconds.
• Streams cannot be rescued. Once the first bytes of a streaming answer have gone to the caller, the gateway cannot switch. The caller gets the partial answer and an error, and has to retry.
• The circuit breaker skips a destination that keeps failing. It is on by default. While it is open, requests go straight to the next destination instead of paying for a failed attempt first; now and then it lets a request through to test whether the primary has recovered. Turn it off only if you would rather keep trying a cheaper primary than fail over fast to a dearer one.

Fallback is invisible to the caller: one request, one response, and no field saying who served it. The gateway's routing logs record that. Each attempt is billed as a gateway invocation, and each attempt's tokens are billed by the provider that handled it.

Try itWhere does one request end up?

Primary: gpt-5.5 on bedrock-mantle

·Answers·Throttled (429)·Server error (503)·Stalls, no first byte·Rejects the request (400)·Cuts off mid-stream

Fallback 1: gpt-5.5 on openai

·Answers·Throttled (429)·Server error (503)·Stalls, no first byte·Rejects the request (400)·Cuts off mid-stream

Fallback 2: gpt-5.4 on bedrock-mantle

·Answers·Throttled (429)·Server error (503)·Stalls, no first byte·Rejects the request (400)·Cuts off mid-stream

Circuit breaker

On (default)Off

The primary has been failing for

A momentSeveral minutes

Requests like this a month

1,000,000

10K1M100M

1. Primary

   gpt-5.5 on bedrock-mantle

   Failed, moved on

   Returned 429, a trigger. The gateway tries the next destination. Billed as an invocation.

2. Fallback 1

   gpt-5.5 on openai

   Served

   Tried once and answered. The caller gets this response, with no sign of which destination served it.

3. Fallback 2

   gpt-5.4 on bedrock-mantle

   Not reached

   Not tried: the request was already answered or ended.

If your models are deprecated on a schedule, a fallback to the successor model is also a safety net for the day the old one stops answering; our model retirement checklist covers the rest of that job.

Who may call, and with whose keys

Every gateway must say how callers prove who they are. Inbound, the choices are OAuthOAuthThe standard way to let one app act for you at another service without giving it your password; the app gets a limited, revocable access token instead. MCP servers use it to let AI agents sign in. with a JWT from your identity provider, IAM with AWS Signature Version 4, authenticate only (the gateway checks the token and the target decides what is allowed), or no authorization, which is for testing.

Outbound, each target gets its own credentials. Lambda and Smithy targets use the gateway's IAM role. OpenAPI and MCP server targets take an API key or OAuth client stored in AgentCore Identity, SigV4 signing, or nothing for a public endpoint. MCP targets also support three-legged OAuth, where the user signs in to the third-party service and the agent acts with that user's rights instead of one shared account. AgentCore Identity costs nothing when used through the gateway.

For checks no setting covers, a request interceptor runs your Lambda function before the gateway calls a target: reject the call, add a header, or rewrite the model name. A response interceptor does the same on the way back. For rules about which tool calls are allowed at all, AgentCore Policy checks each call against Cedar rules outside the model, where a prompt cannot argue with them; the guardrailsGuardrailChecks placed around a model that stop or change what goes in and what comes out, such as a filter on user messages or an approval step before an agent acts. page explains the difference.

Rate limits: per user, per tool, per model

A rate limit groups traffic by up to 10 keys and gives each group a rate. The keys can be the target, the tool, the model, a claim in the caller's JWT (such as sub or a team claim), the IAM principal, or a header or body field. A header or body key must sit beside an identity key, so a caller cannot dodge the limit by changing a header. Inference targets can be limited in tokens per minute, not only requests, and a rate of 0 blocks a caller outright. Rate limits are free.

Read the small print before you rely on them:

• They are checked before any routing rule, and every limit that applies must pass.
• They fail open: if the rate limit service is down or a key cannot be read from the request, the request goes through. AWS says plainly that they are not a security boundary.
• A new or idle limit can let a short burst past before it settles, and token limits settle more slowly, because a request's real token count is only known when the model finishes.

There is also no built-in cap on how long a streamed answer runs. Without a token limit on a target, one request can stream without end, and every user of that target spends from the same provider key. Our page on LLM API rate limits covers the provider side, and setting an AI agent's budget limits covers stopping spend before the invoice. The rate limitRate LimitThe most an API lets you send in a period, counted in requests and tokens per minute. Past it, requests fail with a 429 error until capacity comes back, so code waits and retries. concept explains the mechanism.

What it costs

The gateway bills by the call, and it is rarely the large line on the invoice.

A worked month: an agent handles 1 million conversations. Each lists its tools once, runs 1 tool search, calls 4 tools and makes 3 model requests through the gateway. That is 8 million invocations ($40.00), 1 million searches ($25.00) and 200 indexed tools ($0.04): $65.04. The 3 million model requests themselves are billed separately by the provider that serves them, and at a few thousand tokens each they usually cost far more than the gateway; the agent cost simulator prices that part. AWS's own example, an HR assistant with 200 tools, 50 million searches and 200 million tool calls a month, comes to $2,250.04.

Fallback changes the count, not the price: a request that fails on the primary and succeeds on the second destination is 2 invocations. With the breaker open, the skipped primary costs nothing. Runtime, Memory and the other AgentCore services are priced separately, in the AgentCore guide and Amazon Bedrock pricing.

When a gateway in front is worth it

Put AgentCore Gateway in front when several agents share tools, when tool calls must carry each user's own rights, when you call models from more than one provider, or when you need one place to see, limit and log all of it. It replaces the custom glue each team would otherwise write for authentication, key storage, retries and per-user quotas.

Skip it for one agent with three tools and one model: calling the tools and the model directly is simpler to debug, and the gateway's per-call price is not what you save. If your data and agents live on Databricks rather than AWS, see Databricks Unity Gateway. For connecting agents to data platforms through MCP specifically, see MCP servers for data platforms.