Definition
Bayesian optimization uses a probabilistic surrogate model, typically a Gaussian Process, to model the objective function. It employs an acquisition function to balance exploration and exploitation, selecting the next evaluation point that maximizes expected improvement. This method is highly efficient for tuning hyperparameters in machine learning models where each training run is computationally costly, requiring fewer evaluations than grid or random search to find near-optimal configurations.
Summary
A sequential design strategy for global optimization of black-box functions that are expensive to evaluate.
Key Concepts
- Surrogate Model
- Acquisition Function
- Exploration vs Exploitation
- Black-Box Optimization
Use Cases
- Hyperparameter tuning for deep learning models
- Optimizing experimental designs in science
- Robotics control parameter adjustment