Episode Details

Back to Episodes

“Consistency Training while Mitigating Obfuscation via Rate Matching” by Sohaib Imran, Prakhar Gupta, Jannes Elstner, David Africa

Published 4 weeks, 2 days ago
Description

Sohaib Imran, Prakhar Gupta, Jannes Elstner, David Demitri Africa

Links: Paper | Code

TL;DR.

  • Models condition their behavior on extraneous input features in undesirable ways — for example, on evaluation-likeness (resulting in evaluation gaming), or on the user's preferred answer (resulting in sycophancy).
  • Consistency training teaches a model to behave the same whether or not an extraneous feature/cue is present in the input. Existing methods do this by fine-tuning LLMs to generate responses (BCT) or internal activations (ACT) that would be generated under a clean input, when given an input with a cue.
  • We show this generalises to obfuscation: after BCT, models verbalise cues far less, even when they are influenced by the cues. We introduce a new method, Rate Matching Consistency Training (RMCT), which instead uses RL to match the rate of outcomes across inputs, leaving the rest of the response unconstrained. On the sycophancy benchmark of s Chua et al. (2025), RMCT matches or exceeds BCT at reducing bias-following while leaving verbalisation of the biasing cue largely intact.
  • We think outcome-consistency training of this kind is one of the more natural tools available against evaluation gaming, and below we discuss that [...]

---

Outline:

(02:10) Extraneous features

(05:15) Rate Matching Consistency Training

(09:28) Summary of results

(14:22) Limitations

(15:27) The case for applying Consistency Training to mitigate evaluation gaming

(19:09) Why RMCT for mitigating eval gaming?

(21:04) A few other pathologies that RMCT is particularly suited to mitigate

(21:10) Propensity gaps across modalities/languages/domains/formats.

(21:36) Preference for self-generated text.

(22:23) Incorporating Consistency Training into the alignment training stack

(23:53) Call for feedback

The original text contained 8 footnotes which were omitted from this narration.

---

First published:
July 1st, 2026

Source:
https://www.lesswrong.com/posts/z2zYWnbGjcs6ypbsj/consistency-training-while-mitigating-obfuscation-via-rate

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Figure 1. Overview of Rate Matching Consistency Training (RMCT). We sample multiple trajectories from the model under biased and unbiased prompts, score each trajectory with a binary classifier, and compute per-prompt behaviour rates. In this simplified two-input case, so the unbiased rate serves as the target, and RMCT uses GRPO to push toward. Because the reward depends only on the selected trait, the chain of thought is unconstrained, including verbalisation of the bias. Responses are paraphrased for brevity.
Listen Now

Love PodBriefly?

If you like Podbriefly.com, please consider donating to support the ongoing development.

 Support Us