In this episode of Security & GRC Decoded, Raj Krishnamurthy sits down with Omar Santos, Distinguished Engineer for AI Security at Cisco, for an unusually concrete look at what it takes to run autonomous offensive security safely — and what breaks when you try.

Omar co-chairs the Coalition for Secure AI, chairs the CSAF technical committee, led the DEF CON Red Team Village, and has authored more than 25 books. He walks through the architecture his team actually runs: the three-modality evaluation loop that turns static analysis findings into working exploits, the sandbox and kill-switch design for agents interacting with live systems, and the budget agents that stop a three-day run from burning tens of thousands of dollars in tokens. The conversation covers model alignment and agent drift, harness engineering, context compression and long-term memory, Cisco's open-source Foundry Security Spec and Project CodeGuard, where traditional machine learning still beats an LLM, and what happened at DEF CON when seven teams solved every flag.

Key Takeaways:

• Agents running unattended over a weekend are returning zero days complete with exploits, unit tests, regression tests, and CWE classification.
• You will never have a perfect harness — the goal is a self-improving one that selects better models and burns fewer tokens each run.
• Model alignment and agent drift are different problems that compound: a model told to act like an attacker will do whatever completes the task.
• The highest-leverage move is converting what an agent discovered into a deterministic rule you can run at scale without tokens.
• Evaluation is not one size fits all — offensive security, SOC operations, and compliance agents each need a different definition of done.

What You'll Learn:

• How a three-modality pipeline (static analysis → live system interaction → working exploit) removes false positives by definition
• Why the sandbox question is an infrastructure question, not a Docker question
• What Cisco's Foundry Security Spec contains: eight agent roles, five extensions, ~130 functional requirements, and a constitution
• How to think about long-term memory when a vector database, a markdown file, and Postgres all now work
• Where traditional ML classifiers still belong in an agentic stack, exposed as tools

This podcast is brought to you by ComplianceCow — the smarter way to manage compliance. Automate evidence collection, eliminate screenshots, and scale your program with confidence. Learn more: https://www.compliancecow.com

Watch more episodes: https://www.compliancecow.com/podcast

Connect With Our Guest: Omar Santos | Distinguished Engineer, AI Security | Cisco

Connect on LinkedIn: https://www.linkedin.com/in/santosomar/

Links Mentioned In Episode: 

• Foundry Security Spec
• https://project-codeguard.org/
• https://becomingahacker.org/loop-the-claude-code-command-that-is-changing-the-conversation-about-software-engineering-46a16eb4be30
• https://www.langchain.com/blog/multi-needle-in-a-haystack

Rate, review, and share if you enjoyed the show!

Subscribe to Security & GRC Decoded wherever you get your podcasts:

Spotify: https://open.spotify.com/show/5pigcMwOrYIA6d9OOOsxqr?si=416b82ab5c474683

Apple Podcasts: https://podcasts.apple.com/us/podcast/security-grc-decoded/id1795144450