Definition
Unlike model parameters (weights and biases) that are learned from data during training, hyperparameters are external settings chosen by the practitioner before training begins. They control the structure of the model, the optimization process, and the regularization strength. Examples include learning rate, batch size, number of layers, and dropout rate. Proper selection of hyperparameters is critical for achieving optimal model performance and preventing issues like overfitting or underfitting.
Summary
A configuration variable whose value is set prior to the training process and governs the behavior of the learning algorithm.
Key Concepts
- Learning Rate
- Batch Size
- Model Architecture
- Regularization
Use Cases
- Configuring neural network training
- Setting up gradient descent steps
- Defining tree depth in random forests