brain of mat kelcey...

based on my three strikes rule i wrote a talk on the map interpretation of neural attention.

it starts with the first NLP problem i saw where attention was used, goes through the way i think about attention in terms of soft lookup in a map, shows how this soft lookup solves that NLP problem, and finishes with the small mods required to turn it in the building block for the transformer architecture.

here's a recording; check it out!