Definition
Hyperparameter Optimization (HPO) refers to the broader field of automating the selection of hyperparameters. While tuning is the general act, HPO often implies the use of sophisticated algorithms like Bayesian Optimization, Evolutionary Algorithms, or Gradient-Based Optimization. These methods build a surrogate model of the objective function to predict which hyperparameter settings are likely to yield good performance, thereby reducing the number of expensive training runs required compared to manual or brute-force methods.
Summary
An automated approach to finding the optimal hyperparameter configuration, often using probabilistic models to guide the search.
Key Concepts
- Bayesian Optimization
- Surrogate Models
- Automated ML
- Search Space
Use Cases
- AutoML pipelines
- Large-scale model training
- Resource-constrained optimization