Definition
Long context refers to the capacity of transformer-based models to handle extensive input lengths, often exceeding standard limits like 2k or 4k tokens. This capability allows models to analyze entire documents, codebases, or lengthy conversations in a single pass. Achieving this requires architectural innovations such as efficient attention mechanisms (e.g., FlashAttention) or positional encoding adjustments to maintain coherence and memory over vast distances within the sequence.
Summary
The ability of a language model to process and retain information from input sequences containing thousands or millions of tokens.
Key Concepts
- Context window
- Token limit
- Attention mechanism
- Positional encoding
Use Cases
- Summarizing full legal contracts
- Analyzing complete source code repositories
- Processing long-form audio transcripts