Episode Details
Back to Episodes“We need more empirical research on biological AI model safety” by Noga Aharony
Description
Epistemic status: We are quite confident that Biological AI models (BAIMs) safety requires further work, but uncertain about its scale. The apparent gap may be refuted by one experiment, filled by a few researchers working for a year, or prove to be substantial enough to call for an entire subfield.
Disclaimer: this post has been written with a colleague, that due to her current job can't post out of her own forum account
What we’re looking for: please poke holes in this. In particular, we’d value:
- Arguments against prioritizing this work;
- Important research or organizations we’ve missed;
- Reasons the research questions below would not actually change decisions;
- Information-hazard or capability-externality concerns we’re underweighting;
- Better questions than the ones we propose
The case in brief
Biological AI models (protein, genomic, and single-cell models trained directly on biological data) enable increasingly capable biological design. That carves the path to new vaccines and therapeutics, but also to biorisk scenarios.
These models exist for beneficial scientific or defensive purposes. However, the underlying capabilities could also be applied to harmful objectives. They are currently an important step for designing novel pathogens, and as they improve we expect them to remain a part of [...]
---
Outline:
(01:04) The case in brief
(02:42) Biological AI models are becoming more capable
(04:15) We are missing some basic measurements
(05:59) Research questions that could change what we do
(06:11) Will general-purpose AI subsume biological AI models?
(06:36) What drives BAIM performance?
(06:57) How well do capabilities generalize?
(07:32) How should BAIM safeguards work?
(08:26) Why now?
(09:24) Reasons we might be wrong
(09:27) Artificial General Intelligence will eat this problem
(09:50) BAIMs may not be the bottleneck
(10:08) Information hazards
(10:35) If you know of relevant work, disagree with our framing, or think these are the wrong questions, please tell us - is there something we are missing?
---
First published:
August 25th, 2026
---
Narrated by TYPE III AUDIO.