Definition
The term ‘pretrained’ describes a neural network model that has undergone initial training on a massive, often generic, dataset such as ImageNet or Wikipedia. This process allows the model to learn fundamental features, syntax, or visual patterns. The pretrained model serves as a starting point for transfer learning, where it is further fine-tuned on a smaller, task-specific dataset. This strategy drastically reduces training time and data requirements while often achieving superior performance compared to training from scratch.
Summary
Refers to a model that has already been trained on a large dataset before being adapted for a specific task.
Key Concepts
- Transfer learning
- Feature extraction
- Foundation models
- Fine-tuning
Use Cases
- Initializing BERT or GPT models
- Using ResNet for image classification
- Starting point for domain-specific NLP tasks