Episode Details
Back to Episodes
What If Your Legal AI Works in a Demo but Fails in Practice? | Josh Kreamer
Description
An impressive AI demo doesn't guarantee success in real legal work. But how do you properly test these tools when the client data needed is too sensitive to use?
A Careers and the Business of Law (CBL) Pre-TLTF Podcast Series conversation.
David Cowen welcomes back Josh Kreamer, co-founder and CEO of Seedless, to explore how realistic synthetic legal data can help organizations develop and evaluate AI tools without exposing confidential client information.
Drawing on his experience in legal practice, eDiscovery, and leading legal services at AstraZeneca, Josh explains how data restrictions can slow innovation and why purpose-built simulations offer a practical alternative.
David and Josh also discuss emerging legal engineering roles, AI purchasing decisions, and the importance of realistic testing before adopting new technology.
In This Episode
- Why sensitive client data makes legal AI testing difficult
- How synthetic data enables realistic, measurable evaluations
- Why customized simulations can reveal weaknesses in AI tools
- How data restrictions at AstraZeneca inspired Seedless
- Who drives AI purchasing decisions across the legal industry
- Why focused datasets are often more valuable than one-size-fits-all solutions
Chapters
00:10 | Meet Josh Kreamer, Co-Founder and CEO of Seedless
01:15 | What Is Realistic Synthetic Legal Data?
02:14 | Why Public and Client Data Limit AI Testing
03:02 | Creating Test Data with Known Answers
03:23 | Who Uses Seedless?
04:16 | How AstraZeneca Inspired Seedless
06:20 | Custom Simulations for Investigations and Contracts
07:54 | Legal Engineers and AI Buying Decisions
09:54 | Why One Dataset Cannot Test Everything
11:09 | TLTF Startup Showcase and Industry Connections
13:03 | Closing Reflections and Josh's Music Pick
The Big Idea
A compelling AI demo shows what's possible. Rigorous testing helps determine what actually works.
By using realistic, purpose-built synthetic data, legal teams can evaluate technology against complex scenarios, make more informed decisions, and adopt AI with greater confidence.
Follow SOLID for more conversations with the leaders shaping what comes next in legal: https://solid.legal/
🎧 Subscribe on Apple Podcasts, Spotify, or wherever you listen. Share this episode and take your career from now to next!
Never eat alone!