Episode Details

Francois Lopitaux is SVP of Product at ThoughtSpot, the analytics company built on the premise that you should be able to search your data like you search Google. He began his product career 18 years ago, when Salesforce acquired his Paris company, and went on to lead customer service, analytics and AI products there before joining two startups and then ThoughtSpot. He joins Randy Silver to explain how building agentic products differs from shipping deterministic software, from designing without control of the user path to keeping an LLM trustworthy when the answer has to be the same every time.

Key takeaways

1. AI has cut the time to prototype and validate, and ThoughtSpot estimates its development is two to three times faster. Productionisation still needs sound architecture, guardrails and experienced engineers.
2. When everyone can vibe code, discipline matters more. Lepiteau compares fast AI-assisted development to driving a Formula One car on a road built for a mini Austin: the speed is only useful if you can control it.
3. Agentic features are probabilistic, so product teams are now building guardrails around an LLM rather than a fixed step-by-step flow. That means more hypotheses and more testing.
4. Keep the end goal fixed and stay flexible on the technology. ThoughtSpot is now revisiting existing features to see where newly released models can replace what it built with LLMs.
5. The eval system is becoming the new PRD. It defines what good looks like and what must be avoided, in a product where the UI is a prompt and the path can't be predicted.
6. For analytics, where "what is my revenue this year?" must always return the same answer, trust comes from using an LLM only where it is needed and supplying a semantic layer. That layer defines how your business works, so the model behaves less like a new intern guessing at your terminology.
7. Adoption and repeat usage are the metric that matters most, backed by anonymised sampling of answer quality and follow-up questions. Enterprise customer interviews supply the qualitative temperature check.
8. Embedded and white-label products raise the bar on trust, because customers ship the product to their own customers and carry the accountability. Features must be rebrandable and opt-in, with flags so each customer can roll out at its own pace.
9. An AI feature is never finished. Evals are only a guess at how people will use the product, so teams must keep observing real usage after general availability. ThoughtSpot organises teams by product track, which makes that follow-up a natural continuation of the work rather than extra load.
10. Pruning matters more now that code is cheap to generate. Lepiteau's own team has built three versions of its conversational agent, and without cutting back old versions he warns of a monster where every addition breaks something else.
11. Lepiteau still hires technical, curious product managers. For a recent associate PM role, he asked candidates for a Git repo and a video explaining why what they built solves a problem.
12. Execution is speeding up, so he expects smaller teams and a falling PM-to-developer ratio. Deciding what is meaningful to customers is the remaining bottleneck.
13. 2025 was about proving AI works. Lepiteau expects 2026 and 2027 to be about proving ROI, so teams should be able to switch models easily and pick the cheapest one that does the job.

Chapters

00:00 Introduction
 00:48 Meet François and ThoughtSpot
 02:55 How AI is changing the product development lifecycle
 05:18 Why discipline matters more in the AI era
 08:00 Building probabilistic features
 09:43 Why evals are the new PRD
 13:57 Building an eval system for agentic analytics
 17:15 Creating trust with a semantic layer
 19:30 Measuring the success of AI products
 22:12 Trust in embedded analytics
 24:19 Go-to-market for