Definition
DP-SGD is a variant of Stochastic Gradient Descent designed to protect the privacy of training data. It works by clipping the contribution of each sample’s gradient to limit sensitivity, then adding Gaussian noise scaled to the privacy budget before updating model weights. This process ensures that the final model does not memorize specific training examples, making it resistant to membership inference attacks while maintaining reasonable utility.
Summary
An optimization algorithm that modifies standard SGD by clipping gradients and adding noise to ensure the trained model satisfies differential privacy constraints.
Key Concepts
- Gradient Clipping
- Gaussian Noise Injection
- Sample Subsampling
- Privacy Accounting
Use Cases
- Training deep neural networks on private user data
- Healthcare predictive modeling
- Financial fraud detection with regulated data