Episode Details

Back to Episodes
What If Your Legal AI Works in a Demo but Fails in Practice? | Josh Kreamer

What If Your Legal AI Works in a Demo but Fails in Practice? | Josh Kreamer

Published 4 days, 6 hours ago
Description

An impressive AI demo doesn't guarantee success in real legal work. But how do you properly test these tools when the client data needed is too sensitive to use?

A Careers and the Business of Law (CBL) Pre-TLTF Podcast Series conversation.

David Cowen welcomes back Josh Kreamer, co-founder and CEO of Seedless, to explore how realistic synthetic legal data can help organizations develop and evaluate AI tools without exposing confidential client information.

Drawing on his experience in legal practice, eDiscovery, and leading legal services at AstraZeneca, Josh explains how data restrictions can slow innovation and why purpose-built simulations offer a practical alternative.

David and Josh also discuss emerging legal engineering roles, AI purchasing decisions, and the importance of realistic testing before adopting new technology.

In This Episode

- Why sensitive client data makes legal AI testing difficult
- How synthetic data enables realistic, measurable evaluations
- Why customized simulations can reveal weaknesses in AI tools
- How data restrictions at AstraZeneca inspired Seedless
- Who drives AI purchasing decisions across the legal industry
- Why focused datasets are often more valuable than one-size-fits-all solutions

Chapters

00:10 | Meet Josh Kreamer, Co-Founder and CEO of Seedless

01:15 | What Is Realistic Synthetic Legal Data?

02:14 | Why Public and Client Data Limit AI Testing

03:02 | Creating Test Data with Known Answers

03:23 | Who Uses Seedless?

04:16 | How AstraZeneca Inspired Seedless

06:20 | Custom Simulations for Investigations and Contracts

07:54 | Legal Engineers and AI Buying Decisions

09:54 | Why One Dataset Cannot Test Everything

11:09 | TLTF Startup Showcase and Industry Connections

13:03 | Closing Reflections and Josh's Music Pick

The Big Idea

A compelling AI demo shows what's possible. Rigorous testing helps determine what actually works.

By using realistic, purpose-built synthetic data, legal teams can evaluate technology against complex scenarios, make more informed decisions, and adopt AI with greater confidence.

Follow SOLID for more conversations with the leaders shaping what comes next in legal: https://solid.legal/

🎧 Subscribe
on Apple Podcasts, Spotify, or wherever you listen. Share this episode and take your career from now to next!

Never eat alone!

Listen Now

Love PodBriefly?

If you like Podbriefly.com, please consider donating to support the ongoing development.

Support Us