AI Terms Dictionary

Definition

The reparameterization trick is a fundamental method used in variational autoencoders and other probabilistic models. It allows gradients to flow through stochastic nodes by expressing a random variable z as a differentiable function of distribution parameters and an independent noise variable epsilon. This enables the use of backpropagation to optimize the expected log-likelihood, making training of latent variable models efficient and stable via Monte Carlo estimation.

Summary

A technique that separates stochastic variables from learnable parameters to enable gradient-based optimization in variational inference.

Key Concepts

• Variational Inference
• Gradient Estimation
• Stochastic Nodes
• Differentiable Simulation

Use Cases

• Training Variational Autoencoders (VAEs)
• Bayesian Neural Networks
• Probabilistic Graphical Models

Code Example

1
2
3
4
5

import torch
epsilon = torch.randn(100, 10)
mu = torch.zeros(100, 10)
sigma = torch.ones(100, 10)
z = mu + sigma * epsilon  # Reparameterized sampling

Related Terms

• ELBO
• Latent Variables
• Backpropagation
• Monte Carlo Estimation