Episode Details

DeepSeek V4.1 Flash looks like a point release but behaves like a new base model: a 552B-parameter mixture of experts with 8B active parameters during prefill, a 20-layer causal encoder feeding a 20-layer decoder, and a global KV cache squeezed to 890 bytes per token. That last number is what makes a million-token context economically real. But bigger windows don't fix the problem anyone actually feels — the model losing the middle of its own input. We walk through the "lost in the middle" literature, the U-shaped attention bias that starves the center of a long prompt, attention sinks that park 46% of a head's budget on the first token, and Chroma Research's finding that effective windows are often under 10% of marketed maximums. Then we connect it to repetition: when a model gets uncertain, likelihood-maximizing decoding falls back on the token it just used. The architecture is impressive. The middle is still where things go wrong. Episode #198988 — open it directly at myweirdprompts.com/198988