The Geometry of Attention: One Space, Two Operators
Author(s): GSO1 Originally published on Towards AI. How two operators in one space reveal what four projections hide The goal of single-headed attention is to enhance the contextual awareness of a fixed token, or query, based on content from other tokens. This …