Why do LLMs still misclassify data even with many examples?
Because attention heads compete.
What does this episode explain?
In the final seasonally-themed Floofies episode on attention, we break down attention head mechanics and why they can derail classification tasks.
What happens during classification?
Different attention heads encode different signals at once - word position, semantic relationships, and pattern matching against prior context.
Do attention heads work together?
No. They compete, and position-focused heads often dominate.
Why doesn’t adding more examples fix the issue?
More examples don’t help if they don’t resolve head competition.
What actually works?
Contrasting examples, explicit rules, and clearly defined exclusions.