Definition
The term ‘global’ in AI typically contrasts with ’local,’ referring to aspects that encompass the whole system. In optimization, global minima represent the best possible solution across the entire loss landscape, whereas local minima are suboptimal points within specific regions. In attention mechanisms, global attention considers all tokens in a sequence simultaneously. Similarly, global batch normalization statistics are computed over the entire dataset. Recognizing global vs. local distinctions is vital for understanding model convergence, interpretability, and computational complexity.
Summary
Describing properties, optimizations, or scopes that apply to the entire model or dataset rather than local subsets.
Key Concepts
- Global Minimum
- Full Context
- System-Wide
- Aggregation
Use Cases
- Loss Landscape Analysis
- Global Attention Mechanisms
- Batch Normalization Statistics