Definition
Predictive learning involves training neural networks to infer unobserved data points from observed inputs without explicit human labels. By solving tasks like next-token prediction in language or masked pixel reconstruction in images, the model learns rich internal representations of structure and semantics. This method leverages vast amounts of unlabeled data, enabling scalable pre-training that captures general patterns useful for downstream tasks through fine-tuning.
Summary
A self-supervised approach where models learn representations by predicting missing parts of input data.
Key Concepts
- Self-supervision
- Masked modeling
- Representation learning
- Unlabeled data
Use Cases
- Pre-training large language models
- Image inpainting tasks
- Anomaly detection in time series