Definition
Alignment focuses on making sure AI systems do what humans actually want, rather than just what they literally ask for. It involves techniques like Reinforcement Learning from Human Feedback (RLHF) to tune models based on human preferences. Misalignment can lead to unintended harmful outcomes even if the model is technically competent. Achieving alignment requires defining clear value structures and continuously evaluating model behavior against these standards to prevent drift or exploitation of loopholes.
Summary
The process of ensuring that an AI system’s objectives and behaviors match human values and intentions.
Key Concepts
- RLHF
- Value loading
- Intent matching
- Reward modeling
Use Cases
- Tuning chatbots for politeness
- Ensuring factual accuracy in summaries
- Preventing jailbreak exploits