Definition
Gradient descent is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. In machine learning, it updates model weights in the opposite direction of the gradient of the loss function, effectively descending the error landscape toward the lowest point. Variants like Stochastic Gradient Descent (SGD) and Adam improve efficiency and convergence speed. It is fundamental to training neural networks, enabling models to learn patterns from data by systematically reducing prediction errors.
Summary
An iterative optimization algorithm used to minimize a loss function by adjusting model parameters.
Key Concepts
- Loss Function
- Learning Rate
- Optimization
- Backpropagation
Use Cases
- Training deep neural networks
- Linear regression parameter tuning
- Image recognition model optimization