From the course: Operating AI Agents: Failure and Recovery
Unlock this course with a free trial
Join today to access over 25,600 courses taught by industry experts.
Detect agent failures - Github Codespaces Tutorial
From the course: Operating AI Agents: Failure and Recovery
Detect agent failures
In the real world, your security team probably has scanning tools in place, looking at the environment to ensure there are no data leaks, no PII laying around anywhere, no API keys checked into source code. But we intentionally let the shopping agent do something allowed from a runtime perspective, but still wrong from a governance perspective. It wrote sensitive information into the output. Now, we're going to take the exact artifacts that most teams already have, the agent's output file, and a structured log file, and we're going to turn them into an incident report. Let's open up scanner.py. This code has one job. First, scan for sensitive patterns inside the agent output. Then, if it finds anything, walk the logs and identify which agent run and which tool call wrote that output. I do want to call out though in agent definitions. This is where I've added an operation ID. And if you remember, this file is where we've defined all of our agents. Here, scrolling down on line 47, I've…