Definition
Overfitting occurs when a model learns the training data too well, including its random noise and outliers, resulting in excellent performance on training data but poor performance on new, unseen test data. This happens because the model becomes overly complex relative to the amount of training data available. Techniques like regularization, dropout, early stopping, and cross-validation are commonly employed to mitigate overfitting and improve the model’s ability to generalize.
Summary
A modeling error where a machine learning algorithm captures noise instead of the underlying signal, harming generalization.
Key Concepts
- High variance
- Poor generalization
- Training vs test error gap
- Model complexity
Use Cases
- Diagnosing model performance issues
- Selecting regularization strength
- Determining optimal model depth