Definition
Predictive State Representations (PSRs) extend traditional partially observable Markov decision processes by defining states as vectors of predictions about future observable events. Instead of relying on hidden true states, PSRs use the history of actions and observations to predict what will happen next. This allows agents to operate effectively in environments with partial observability, providing a more flexible and often more compact representation of the environment dynamics for planning and control.
Summary
A latent state formulation in reinforcement learning that predicts future observations based on action history.
Key Concepts
- Partial observability
- Future predictions
- History-based states
- Reinforcement learning
Use Cases
- Robotics in cluttered environments
- Game AI with hidden information
- Financial market forecasting