Attention Routes Relevance
Explain attention as a mechanism for routing information between tokens in context.
The model sees the supplier name and the exception, but the answer connects only the supplier and misses the exception. Attention is selective routing across token relationships, not perfect document comprehension. The common trap is believing that because a fact is inside the context window, the model will use it correctly. Token positions The document is represented as a long sequence of token positions. Distance, formatting, and chunk boundaries can affect how easy a relationship is to use. Attention weights When predicting a token, the model assigns different weights to other token positions. Relevant tokens can influence the current representation…
Sign up free — one personalized lesson every day, matched to your role and goals.
Already have an account? Sign in