Episode Details

We hand coded weights for one layer MLPs that memorises labels for input token sequences of length two. The number of facts our hand-coded models can memorise with 90% accuracy[1]scales roughly linearly with the models' parameter count[2], just like trained models for the same architecture. However, our hand-coded models' scaling prefactor still falls short of trained models' by a factor of . A hybrid solution in which we handcode the MLP input weights and learn the MLP output weights, which amounts to a linear classification problem[3], falls short by a factor of .

Figure 0: Max number of facts each model can learn when requiring 90% accuracy on evaluation. The fitted lines are best fit for solving for .

We pose a challenge to the community: Find better constructions than ours that can memorise more facts in the same number of weights and close the gap to the trained solution further.[4]

Background: Why sequence memorization, and why an MLP?

The goal of this research is to get a clearer understanding of how look-ups are encoded in LLMs.

We expect that a lot of the information stored in the weights of an LLM is memorized facts, rather than general circuits. [...]

---

Outline:

(01:11) Background: Why sequence memorization, and why an MLP?

(03:54) Outline

(06:21) Training data

(07:26) Testing Various Model Architectures

(08:46) Model variations

(08:50) Mixing

(09:51) MLP

(10:21) Norms

(10:46) Results

(14:43) Scaling

(15:55) Challenge: Benchmark for understanding

(16:53) Rules

(17:48) Model architecture

(19:11) Evaluation Criteria

(20:13) How this challenge relates to known constructions

(22:08) Our attempt

(23:10) Algorithm: Summary

(25:23) Algorithm: More details

(26:15) Assigning neurons to labels

(27:45) Embedding weights

(28:44) Unembedding weights

(29:11) Hybrid model: Half hand-coded, half learned

(29:23) Hyperparameter search over S and top_fraction

(29:53) Results

(31:36) Takeaways

(32:39) Do the matrices look similar?

(34:25) Contributions and acknowledgements

(34:52) Appendix A: Near-optimal construction for

(37:18) Appendix B: More on what parts of the model matter

(41:10) MLP + Norms + No Residual around MLP + No Bias + ReLU

(42:13) Triple or pairwise comparisons

(42:32) Mixing

(45:03) MLP

(46:42) Norms

(48:52) Residual Connection around the MLP

(50:12) MLP Bias

(51:37) Activation Function

(52:28) Appendix C: More on Scaling

(53:17) Appendix D: Best S (number of neurons per label) for the hand-coded and hybrid models

(56:07) Appendix E: Best "top_fraction" for the hand-coded and hybrid models

The original text contained 30 footnotes which were omitted from this narration.

---

First published:
July 23rd, 2026

Source:
https://www.lesswrong.com/posts/KWtchKwwnJkd4bwCi/challenge-hand-coding-weights-for-efficient-sequence-1

---

Narrated by TYPE III AUDIO.

---