Technical Leader’s GenAI & AI Agent Playbook: 15 Checkpoints from POC to Production, Boston

Well Architechted Gen AI by AWS - project to be open sourced by AWS next week

Factors impacting model latency and token costs: Input tokens- User prompt or App tasks System prompt Knowledge Base Query history or Response history for context Tools from MCP Servers or AC gateway Tool execution result for context Pricing varies by region and model

Snowball effect in Agents User -> Agent -> LLM -> Agent -> Tools -> Agent Cost adds up

Output tokens- Response Tool Invocations Pricing varies by region and model

Agents Business Value- Productivity increase from time savings - Deliver More features, Increase Revenue Cost savings Better Customer experience

Is Agents response valid? Bedrock Guardrails -> Relevance Score and Grounding Score -> if score > threshold allow else block

complex rule based Bedrock Guardrails -> Pre built AR policies -> Valid/Invalid

PII and sensitive data Bedrock Guardrails -> Allow/Deny

undesired topic Bedrock Guardrails -> Deny Topic

protect from promt leakage or system attacks Bedrock Guardrails -> Prompt Attack Filter

Agent evaluation- Online evaluation -> Prod monitoring On-Demand evaluation -> Dev and CI/CD

Agentcore eval metrics- Session level metrics Trace level metrics Span/Tool level metrics

Agentcore Policy - Limit access to agents using IAM Lambda Interceptor - Inject user identity on Agentcore Gateway