Skip to main content

Bedrock

Recent Announcements​

See: AWS re:Invent 2025 - Bedrock Updates - Major partnership expansion bringing GPT-5.5, Codex, Nova 2 family (Sonic, Lite, Forge, Omni, Act), AgentCore enhancements, 18 new open weight models, and reinforcement fine-tuning capabilities.

Web Crawl​

Memory using Agents​

Cost Optimization​

From the AWS Cloud Intelligence Dashboards session Maximize Cost Effectiveness of generative AI workloads (Aug 2026); prices quoted there are approximate. FinOps for AI was the top forward-looking priority in the FinOps Foundation's State of FinOps 2026 (up six places).

  • Billing basics: Bedrock bills input and output tokens, and tokenization is model-specific, so the same text costs differently across models. Choose by modality, context window, cost, latency (time to first token), regional availability and per-model quotas (requests and tokens per minute); over-provisioning a model is like over-sizing an EC2 instance. Example: 20M input + 1M output tokens is about 25onHaiku4.5vsabout25 on Haiku 4.5 vs about 125 on Opus 4.6 with reasoning
  • Inference routing: in-region keeps data in the region but limits quota; a geographic cross-region profile (US/EU/APAC prefix) raises throughput while keeping residency within the geography; a global profile is about 10% cheaper with no residency guarantee (fine for dev and synthetic data)
  • Capacity options: on-demand with a service_tier of standard, flex (discounted, higher latency, back of the queue) or priority (front of the queue, higher price); a reserved tier (1 or 3 month commitment to fixed RPM/TPM, paid whether used or not, guaranteed throughput, but no discount); batch inference (CreateModelInvocationJob, prompts in S3, results within 24h) at 50% off on-demand
  • Cut input tokens: strip whitespace, boilerplate and pleasantries; condense long inputs with a small model first; for chat, keep a sliding window of the last N turns or summarize history, since models are stateless and history is resent on every call
  • Cut output tokens: ask for concise answers, prefer CSV over JSON, and set max_tokens near the real average, because quota is reserved against max_tokens, not actual output (5,000 max vs 1,000 actual is billed at 1,000 but counts 5,000 against quota)
  • Prompt caching: add cache points; cache writes cost about 10% more than input tokens and reads are about 90% cheaper; default TTL is 5 minutes (up to 1 hour on some models)
  • Intelligent prompt routing: a router over a cheaper and a stronger model with a quality threshold; use the router ARN as the model id, and invokedModelId in the response shows which model answered. Build evaluation in from the start, because changing models changes answer quality
  • Monitoring: CloudWatch has tokens, requests, latency, throttles and cache hits but no cost. In Cost Explorer, third-party models (e.g. Anthropic) bill as AWS Marketplace spend, and Cost Anomaly Detection does not cover Marketplace spend, so use Budgets or CloudWatch alarms. CUR 2.0 adds a line_item_iam_principal column. The open-source Cloud Intelligence Dashboards (CloudFormation plus QuickSight, about 25-30 min to deploy, typically under $100 a month) include a Bedrock section with cost per 1M tokens and spend by model, usage type, tag, inference profile and region
  • Cost allocation: individual API calls cannot be tagged; tag inference profiles, tag IAM roles/principals, or use IAM session tags via AssumeRole, all of which propagate to CUR and Cost Explorer with no application changes
  • Governance: enable only approved models (least privilege), set budgets and anomaly alerts (agent loops burn tokens), and track unit economics (cost per customer, call or task) and ROI rather than cost per 1M tokens alone

Learning​