Max Tkacz

Aug 20, 2025 • World Congress 2025

The AI Agent Path to Prod: Building for Reliability

Your AI agent works in demos, but will it break in production? Learn to build the evaluation frameworks and guardrails necessary for true reliability.

#1about 4 minutes

Why AI agents fail in production environments

AI agents often fail in production because the probabilistic nature of LLMs conflicts with the need for reliability at scale.

#2about 5 minutes

Scoping an AI agent for a specific business problem

Start by identifying a low-risk, high-impact task, like automating free trial extensions, to establish a viable solution scope.

#3about 3 minutes

Walking through the naive V1 customer support agent

The initial agent uses an LLM with tools to fetch user data and extend trials, but its reliability is unknown without testing.

#4about 4 minutes

Using evaluations to test the happy path case

Evaluations are introduced as a testing framework to run the agent against specific test cases, revealing inconsistencies even in the happy path.

#5about 4 minutes

Improving agent consistency with prompt engineering

By adding explicit rules and few-shot examples to the system prompt, the agent's tool usage and response quality become more consistent.

#6about 5 minutes

Testing for prompt injection and other edge cases

A new evaluation case for prompt injection reveals a security flaw, which is fixed by adding specific security rules to the system prompt.

#7about 6 minutes

Applying production guardrails beyond evaluations

Beyond evals, production readiness requires adding human-in-the-loop processes, custom error handling, rate limiting, and model redundancy.

Why AI agents fail in production environments

Scoping an AI agent for a specific business problem

Walking through the naive V1 customer support agent

Using evaluations to test the happy path case

Improving agent consistency with prompt engineering

Testing for prompt injection and other edge cases

Applying production guardrails beyond evaluations

Matching moments

The challenge of moving AI from demo to production

What’s New with Google Gemini?

Moving agentic AI from proof of concept to production

Building Blocks for Agentic Solutions in your Enterprise

Overcoming the challenges of productionizing AI models

Navigating the AI Revolution in Software Development

Key takeaways for building reliable LLM agents

The Limits of Prompting: ArchitectingTrustworthy Coding Agents

The challenge of building production-ready AI applications

One AI API to Power Them All

The promise and pitfalls of implementing agentic AI

Rethinking Customer Experience in the Age of AI

Live demonstration of deploying Azure resources from a prompt

Infrastructure as Prompts: Creating Azure Infrastructure with AI Agents

Previewing the "AI or knockout" conference talk

From Learning to Leading: Why HR Needs a ChatGPT License

Featured Partners

Related Videos

Agents for the Sake of Happiness

Beyond Chatbots: How to build Agentic AI systems

On a Secret Mission: Developing AI Agents

Beyond Prompting: Building Scalable AI with Multi-Agent Systems and MCP

The Limits of Prompting: ArchitectingTrustworthy Coding Agents

You are not my model anymore - understanding LLM model behavior

Three years of putting LLMs into Software - Lessons learned

The State of GenAI & Machine Learning in 2025

Related Articles

From learning to earning

Product Owner/Projektleiter (m/w/d)

Part-Time - AI Operations Support (Voice AI & Automation)

AI & Embedded ML Engineer (Real-Time Edge Optimization)

AI Engineer

Product Owner generative KI

Application Engineer - AI Enablement

Machine Learning Engineer - AI-Features, Agenten & Multi Agenten Workflows

Conversational AI & Machine Learning Engineer

Conversational AI & Machine Learning Engineer