Episode Details

Please share this with anyone doing AI research with 3rd party providers so that they can ensure their research won’t be corrupted.

• When you ask OpenRouter[1] to give you tokens from a given model, OpenRouter sends your request to a random available provider.
• OpenRouter providers have variable quality. Ensuring that your provider is high quality is really difficult. 
• There is precedent for an AI safety paper accepted to NeurIPS having its core results entirely overturned by these issues.
• A review of influential AI Safety research codebases that use OpenRouter for their reported results found that 31/32 (97%) of them use OpenRouter unsafely.[2]
• Researchers who wish to do research using OpenRouter or similar providers should take precautions to minimize the risks to their research,[3] though the current selection of providers is insufficient for ideal scientific reliability.
• Replicators should test whether results hold up when these bugs are fixed.
• Core AI Safety codebases should make fixes so that downstream users are able to implement best practices.

This is a tangent I took for a few days during the Pivotal AI Safety Research Fellowship. I’m doing an AI Control project mentored by Adam Kaufman and James Lucassen of Redwood Research and partnered with Aniruddh [...]

---

Outline:

(02:04) OpenRouter Kills an AI Safety NeurIPS Paper

[... 16 more sections]

---

First published:
July 23rd, 2026

Source:
https://www.lesswrong.com/posts/KsyoSAyBRXtwzSugg/not-pinning-your-openrouter-provider-might-invalidate-your

---



Narrated by TYPE III AUDIO.

---