Production AI on AWS: Amazon Bedrock and AgentCore in a private setup

On AWS, the pieces for a production agent have changed more in the last twelve months than in the three years before: Bedrock Agents became Agents Classic, AgentCore took over as the agent platform, and Bedrock now has four inference tiers. This is the setup I would build today for an agent that has to pass a security review: least-privilege IAM, PrivateLink instead of the internet, a managed knowledge base, guardrails on input and output, evaluations in the pipeline and every invocation logged. All facts link to the AWS documentation.

Dominik Heinrich

November 7, 202611 min read

1. When AWS is the right home, and the state of Bedrock in late 2026

AWS is the obvious choice when your data and workloads already live there: S3 buckets with the documents, Aurora or DynamoDB with the business data, IAM and Organizations as the security backbone. An agent that runs next to that data needs fewer integrations and inherits controls you already audit.

The product landscape has moved. Amazon Bedrock is the model and inference layer, and Amazon Bedrock AgentCore is the agentic platform for building, deploying and operating agents securely at scale, with any framework and foundation model. The Bedrock user guide now states that you can no longer sign up for Amazon Bedrock Agents, which AWS calls Agents Classic; existing users can continue, and new code-defined agents are pointed to AgentCore. Several AgentCore parts reached general availability in 2026, for example Evaluations on 31 March 2026 and the managed agent harness on 17 June 2026.

For architecture decisions I use the AWS Well-Architected Generative AI Lens as the checklist. It covers the same ground as this article, from data protection to cost, and is a good document to hand to a security or architecture board.

2. Accounts and IAM: least privilege for people, apps and agents

Put non-production and production in separate AWS accounts under AWS Organizations. Model access, quotas, guardrails and logs then differ per account, and an experiment can't touch production data by accident.

Give every component its own IAM role and scope it tightly. The application role may invoke one or two specific models, the knowledge base role may read one S3 prefix, and nothing gets bedrock:* on all resources. For the people and systems your agent acts for, AgentCore Identity manages agent identity, access and authentication and works with existing identity providers such as Amazon Cognito, Okta, Microsoft Entra ID or Auth0, so you don't migrate users or rebuild sign-in.

The simplest high-value control is an explicit allowlist of model IDs in IAM. It stops a developer from switching production to an expensive model with a one-line change, and it gives the security team one place to review which models are in use.

json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "InvokeOnlyApprovedModels",
      "Effect": "Allow",
      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
      "Resource": [
        "arn:aws:bedrock:eu-central-1::foundation-model/<approved-model-id>"
      ]
    }
  ]
}

Tip from practice

Many current models are called through inference profiles for cross-Region inference. Check the exact resource ARN the model's documentation lists before you write the policy, or the call fails with AccessDenied.

3. Private network: PrivateLink endpoints and AgentCore in VPC mode

With AWS PrivateLink you access Amazon Bedrock as if it were in your VPC, without an internet gateway, NAT device, VPN or Direct Connect. The Bedrock documentation lists separate interface endpoints per API family: bedrock for the control plane, bedrock-runtime for inference, bedrock-agent and bedrock-agent-runtime for the agent and knowledge base APIs, bedrock-mantle, and FIPS variants. Create the ones your application actually calls, enable private DNS, and attach an endpoint policy that only allows your roles.

The agent itself should live in your network too. AgentCore Runtime and its built-in tools such as Code Interpreter can run in VPC mode: you give them your subnets and a security group, and they reach your databases and internal APIs privately. Outbound traffic then follows your VPC routing, so a central egress filter decides which external hosts an agent may reach.

This is also where data residency is decided. Choose the Region deliberately, and check whether a model is only offered through cross-Region inference before you promise a customer that data never leaves a geography.

bash

# Interface endpoint for inference (repeat for bedrock-agent-runtime etc.)
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-0123456789abcdef0 \
  --vpc-endpoint-type Interface \
  --service-name com.amazonaws.eu-central-1.bedrock-runtime \
  --subnet-ids subnet-0123456789abcdef0 subnet-0123456789abcdef1 \
  --security-group-ids sg-0123456789abcdef0 \
  --private-dns-enabled

# AgentCore Runtime in VPC mode (network part; add your agent artifact and role)
aws bedrock-agentcore-control create-agent-runtime \
  --agent-runtime-name "MyAgentRuntime" \
  --network-configuration '{
    "networkMode": "VPC",
    "networkModeConfig": {
      "subnets": ["subnet-0123456789abcdef0", "subnet-0123456789abcdef1"],
      "securityGroups": ["sg-0123456789abcdef0"]
    }
  }'

AWS architecture: users reach the application through an Application Load Balancer; inside a VPC the app and the AgentCore Runtime in VPC mode call Amazon Bedrock models through interface endpoints; the agent uses AgentCore Gateway, Memory and a managed Bedrock knowledge base backed by S3; Bedrock Guardrails check input and output; IAM and AgentCore Identity control access; CloudWatch and AgentCore Evaluations observe the agent. — Reference setup on AWS: AgentCore Runtime in VPC mode, Bedrock through PrivateLink, a managed knowledge base, guardrails on every call and evaluations in CloudWatch.

4. Your data: a managed knowledge base before a custom RAG pipeline

Amazon Bedrock Knowledge Bases offers two types. With a managed knowledge base, Bedrock handles ingestion, indexing, storage and retrieval, including multimodal data, storage auto-scaling and agentic retrieval for multi-hop questions; AWS recommends this type for retrieval accuracy and a managed experience. With a customer-managed knowledge base, you run your own vector store such as Amazon OpenSearch Serverless, Amazon Aurora or Amazon Neptune and control parsing, indexing and storage yourself.

My default is the managed type for a first release. Most RAG problems I see are not about the vector database; they come from stale documents, missing permissions on sources and chunks that cut a table in half. Spend the saved time on the data: a clear owner per source, a sync schedule, and a test set of questions where you know which document holds the answer.

Move to a customer-managed store when you have a concrete reason: an existing OpenSearch cluster with your search tuning, a graph use case for Neptune, or regulatory requirements on where and how the index is stored.

5. Models and capacity: Standard, Flex, Priority, Reserved or provisioned

Bedrock and AgentCore give you a wide model choice, including Anthropic's Claude, Amazon Nova, Meta Llama and Mistral models, and AgentCore also works with OpenAI and Google Gemini models. Choose per step: a small model for classification and extraction, a stronger one where reasoning matters, and keep the choice behind the IAM allowlist from step 2.

For capacity, Bedrock now offers four service tiers. Standard is the default for everyday work. Flex gives a price discount for workloads that can handle longer processing times, such as evaluations, summarisation or background agent jobs. Priority delivers the fastest response times at a premium and is set per request with the service_tier parameter. Reserved lets you reserve tokens-per-minute capacity for one or three months, targets 99.5% uptime, overflows to Standard when you need more, and starts at 100,000 input and 10,000 output tokens per minute.

Provisioned Throughput with model units is a separate mechanism, mainly relevant for custom models, with no commitment or one- or six-month terms. My rule: start on Standard, move batch and evaluation jobs to Flex right away, and reserve capacity only after a few weeks of real traffic data.

Tip from practice

When you size Reserved capacity, AWS notes that tokens-per-minute consumption includes cache-write input tokens if you use prompt caching. Sum both metrics in CloudWatch, or the reservation will be too small.

6. The agent on AgentCore: runtime, gateway, memory and tools

AgentCore is modular, and you only use what you need. The Runtime is a serverless environment for deploying and scaling agents and tools, with session isolation, built-in identity and support for long-running, asynchronous work. The managed harness lets you define an agent in configuration, with model, system prompt and tools inline, and runs each session in an isolated microVM. Gateway turns your APIs, Lambda functions and existing services into MCP-compatible tools and connects to existing MCP servers. Memory gives short-term memory for a conversation and long-term memory across sessions. Code Interpreter and Browser are sandboxed tools, and Observability shows each step of an agent's execution path.

The Gateway is the part I would design most carefully. Every tool you expose is an action the agent can take with real permissions. Expose narrow, purpose-built tools, for example look up an order by ID, rather than a generic database query, and give the tool's backend its own least-privilege role.

Framework choice is yours: AgentCore works with open-source frameworks such as Strands Agents, LangGraph, LangChain or LlamaIndex. Pick the one your team can debug at 2 a.m., not the one with the most stars.

Amazon Bedrock AgentCore: Deploy & Operate AI Agents in Minutes, Amazon Web Services (YouTube). Loads from YouTube only after you click.

7. Guardrails: prompt attacks, PII and grounding checks

Amazon Bedrock Guardrails combines several safeguards in one guardrail that you attach to model calls or apply on its own. Content filters cover hate, insults, sexual content, violence, misconduct and prompt attacks, each with its own strength. Denied topics block subjects your application must never discuss. Sensitive information filters block or mask PII such as names, addresses or IDs and support custom regular expressions. Contextual grounding checks flag answers that are not supported by the retrieved source or don't answer the question, which is the cheapest hallucination check for a RAG app. Automated Reasoning checks validate answers against logical rules you define.

There are two tiers. The Standard tier extends detection, for example to harmful content hidden in code comments or variable names, and requires cross-Region inference; the Classic tier does not. Decide that consciously if data residency is strict.

Build the guardrail as code, version it, and give it its own test set of attack prompts. The example below blocks prompt attacks on input, masks e-mail addresses and phone numbers, and checks grounding and relevance of answers.

bash

aws bedrock create-guardrail \
  --name support-agent-guardrail \
  --blocked-input-messaging "Sorry, I can't help with that request." \
  --blocked-outputs-messaging "Sorry, I can't answer that." \
  --content-policy-config '{
    "filtersConfig": [
      {"type": "PROMPT_ATTACK", "inputStrength": "HIGH", "outputStrength": "NONE"},
      {"type": "MISCONDUCT", "inputStrength": "HIGH", "outputStrength": "HIGH"}
    ]
  }' \
  --sensitive-information-policy-config '{
    "piiEntitiesConfig": [
      {"type": "EMAIL", "action": "ANONYMIZE"},
      {"type": "PHONE", "action": "ANONYMIZE"}
    ]
  }' \
  --contextual-grounding-policy-config '{
    "filtersConfig": [
      {"type": "GROUNDING", "threshold": 0.8},
      {"type": "RELEVANCE", "threshold": 0.7}
    ]
  }'

Tip from practice

Prompt-attack filtering applies to the input; set its output strength to NONE as in the example. Tune the grounding threshold on your own test set: too high blocks good answers, too low lets invented ones through.

8. Evaluate, log every invocation, watch the cost, then compare the clouds

AgentCore Evaluations became generally available on 31 March 2026. It evaluates agents continuously on production traffic and in testing workflows, with 13 built-in evaluators for response quality, safety, task completion and tool usage, plus ground truth such as reference answers, behavioural assertions and expected tool sequences. Run it as a regression gate in your CI/CD pipeline: a change to instructions, tools or model must not lower the scores of the last accepted release.

For audit and debugging, turn on model invocation logging. It collects invocation logs with model input and output for all Bedrock invocations in an account and Region, into CloudWatch Logs, S3 or both. Treat these logs as sensitive data: encryption, retention and access as strict as for the source documents. AgentCore Observability adds the agent-level view, with traces of each step, on top of CloudWatch.

Pitfalls I see most: wildcard IAM on bedrock:*, agents with internet egress and no filter, a guardrail that was never tested with attack prompts, invocation logs readable by everyone, and Reserved capacity bought before anyone measured traffic. The same setup on Microsoft Azure is in /blog/azure-ai-foundry, and on Google Cloud in /blog/google-vertex-ai.

Takeaway
On AWS, the production recipe is least-privilege IAM, PrivateLink and VPC mode for privacy, a managed knowledge base for your data, guardrails as code, and AgentCore Evaluations as the gate every change has to pass.

Real projects behind this

Try it yourself

Production readiness

Want this in your company? Let's talk mail@dominikheinrich.ch

Sources and official guides

Production AI on AWS: Amazon Bedrock and AgentCore in a private setup

1. When AWS is the right home, and the state of Bedrock in late 2026

2. Accounts and IAM: least privilege for people, apps and agents

3. Private network: PrivateLink endpoints and AgentCore in VPC mode

4. Your data: a managed knowledge base before a custom RAG pipeline

5. Models and capacity: Standard, Flex, Priority, Reserved or provisioned

6. The agent on AgentCore: runtime, gateway, memory and tools

7. Guardrails: prompt attacks, PII and grounding checks

8. Evaluate, log every invocation, watch the cost, then compare the clouds

Get in touch