Episode Details

Sponsorships and tickets for the AI Engineer Summit are selling fast! See the new website with speakers and schedules live!

If you are building AI agents or leading teams of AI Engineers, this will be the single highest-signal conference of the year for you, this Feb 20-22nd in NYC.

We’re pleased to share that Karina will be presenting OpenAI’s closing keynote at the AI Engineer Summit. We were fortunate to get some time with her today to introduce some of her work, and hope this serves as nice background for her talk!

There are very few early AI careers that have been as impactful as Karina Nguyen’s. After stints at Notion, Square, Dropbox, Primer, the New York Times, and UC Berkeley, She joined Anthropic as employee ~60 and worked on a wide range of research/product roles for Claude 1, 2, and 3. We’ll just let her LinkedIn speak for itself:

Now, as Research manager and Post-training lead in Model Behavior at OpenAI, she creates new interaction paradigms for reasoning interfaces and capabilities, like ChatGPT Canvas, Tasks, SimpleQA, streaming chain-of-thought for o1 models, and more via novel synthetic model training.

Ideal AI Research+Product Process

In the podcast we got a sense of what Karina has found works for her and her team to be as productive as they have been:

* Write PRD (Define what you want)

* Funding (Get resources)

* Prototype Prompted Baseline (See what’s possible)

* Write and Run Evals (Get failures to hillclimb)

* Model training (Exceed baseline without overfitting)

* Bugbash (Find bugs and solve them)

* Ship (Get users!)

We could turn this into a snazzy viral graphic but really this is all it is. Simple to say, difficult to do well. Hopefully it helps you define your process if you do similar product-research work.

Show Notes

* Our Reasoning Price War post

* Karina LinkedIn, Website, Twitter

* OSINT visualization work

* Ukraine 3D storytelling

* Karina on Claude Artifacts

* Karina on Claude 3 Benchmarks

* Inspiration for Artifacts /

Listen Now