Episode Details

Soufi Esmaeilzadeh is product director for AI experiences in Google Search, where she leads AI Overviews and AI Mode. In 15 years at Google, she has watched search move from ten blue links to multi-turn conversation. 

She joins Randy Silver to explain how her team prototypes and ships at Google’s scale, why she runs lean “SWAT team” pods, and how she balances speed against the quality bar users expect from search.

Key takeaways

1. Users changed their behaviour first. Once natural language models let people drop keywords, queries grew longer and more specific, and follow-up questions became an expectation rather than a feature request.
2. Soufi frames AI in search as an evolution, not a disruption. Positioning it as a better way to serve the user’s existing need, rather than a threat to the core product, made it easier to win internal support.
3. Her team goes from hypothesis to scrappy prototype to real-data demo to a small beta as fast as possible. Debating in decks is the thing she most wants to avoid, because these are not one-way doors.
4. Fast prototyping makes the hard question “is it worth it?” Ideas are cheap, so product managers must weigh people, attention and tokens against other opportunities, and build well, not just quickly.
5. Quality bars can make or break a launch. Gen UI simulations and audio-over-image explanations landed because the team defined “good” early. A subpar first version would have fallen flat.
6. Soufi runs lean pods: one senior strategic lead and a small bench of ICs, with flat lines of communication. Extra layers strip out context, and without context it is hard to move fast and ship well.
7. Product basics matter more at speed. Clear hypotheses, shared goals, agreed metrics, and a joint definition of what good and not-good look like still come first.
8. A leader’s job in a fast-moving team is to clear the noise. Urgency energises people. Chaos, spinning and cross-team debate drain them.
9. For hiring, Soufi looks for three things: consumer product instinct, close attention to where the underlying technology is heading, and a zero-to-one builder mentality.
10. AI tools accelerate the work, but the team keeps challenging and cross-checking their output. Sharing where each person has hit the limits is a standing conversation, because those limits keep shifting.
11. Org design needs stable problem areas even when the technology shifts. Accountability over an area for two to three years lets people build deep opinions about what success looks like, while their methods change every few months.

Chapters

• 00:00 Introduction
•  01:28 Soufi’s role and background
•  03:09 How AI has changed search
•  05:30 How AI experiences in search evolved
•  07:08 Delivering AI answers at Google’s speed and scale
•  09:04 Evolution, not disruption
•  12:16 What AI means for publishers
•  14:25 Overpersonalisation and quality feedback
•  16:40 Testing and betas at Google’s scale
•  20:40 Gen UI, simulations and visualisations
•  23:39 Balancing cost and ambition
•  25:30 Structuring AI product teams
•  28:21 Keeping quality high as pace increases
•  31:07 Changing processes and measuring speed
•  33:47 Avoiding burnout and context switching
•  36:10 Hiring and promoting
•  37:53 Working with AI tools and where they fall short
•  40:27 Designing teams for the long term
•  42:52 Wrap-up

We're refreshing The Product Experience and want your input. Take our two-minute survey and help shape where the show goes next! 

Our Hosts
Lily Smith enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product manage