Episode Details

David Brauchler says AI red teaming has proven that eliminating prompt injection is a lost cause. And many developers inadvertently introduce serious threat vectors into their applications – risks they must later eliminate before they become ingrained across application stacks.

NCC Group’s AI security team has surveyed dozens of AI applications, exploited their most common risks, and discovered a set of practical architectural patterns and input validation strategies that completely mitigate natural language injection attacks. David's talk aimed at helping security pros and developers understand how to design/test complex agentic systems and how to model trust flows in agentic environments. He also provided information about what architectural decisions can mitigate prompt injection and other model manipulation risks, even when AI systems are exposed to untrusted sources of data.

More about David's Black Hat talk:

• Video of the talk and accompanying slides: https://www.nccgroup.com/research-blog/when-guardrails-arent-enough-reinventing-agentic-ai-security-with-architectural-controls/
• Talk abstract: https://www.blackhat.com/us-25/briefings/schedule/#when-guardrails-arent-enough-reinventing-agentic-ai-security-with-architectural-controls-46112
• Slide presentation only: https://i.blackhat.com/BH-USA-25/Presentations/USA-25-Brauchler-When-Guardrails-Arent-Enough.pdf

Additional blogs by David about AI security:

• Analyzing Secure AI Architectures: https://www.nccgroup.com/research-blog/analyzing-secure-ai-architectures/
• Analyzing Secure AI Design Principles: https://www.nccgroup.com/research-blog/analyzing-secure-ai-design-principles/
• Analyzing AI Application Threat Models: https://www.nccgroup.com/research-blog/analyzing-ai-application-threat-models/
• Building Security‑First AI Applications: A Best Practices Guide for CISOs: https://www.nccgroup.com/building-security-first-ai-applications-a-best-practices-guide-for-cisos/
• Building Trust by Design for Secure AI Applications: Tips for CISOs: https://www.nccgroup.com/building-trust-by-design-for-secure-ai-applications-tips-for-cisos/
• AI and Cyber Security: New Vulnerabilities CISOs Must Address: https://www.nccgroup.com/ai-and-cyber-security-new-vulnerabilities-cisos-must-address/

An op-ed on CSO Online ma