Definition
Self-supervised learning is a technique where the algorithm creates supervisory signals from the unlabeled data itself, typically by predicting missing parts of the input. It bridges the gap between unsupervised and supervised learning, allowing models to learn rich feature representations without manual annotation. This approach is foundational for modern large language models and vision transformers, enabling them to understand structure and semantics in vast amounts of raw data.
Summary
A training method where the model generates its own labels from input data to learn representations.
Key Concepts
- Pre-training
- Masked Language Modeling
- Contrastive Learning
- Representation Learning
Use Cases
- Training large language models
- Image representation learning
- Speech recognition systems
Code Example
| |