Definition
Prompt injection exploits the way large language models interpret user instructions by embedding hidden or conflicting directives within the input text. This can cause the model to ignore its original system prompts, leak sensitive data, or generate harmful content. It is a significant security risk in applications where user input is processed directly by the model, requiring robust sanitization and defense mechanisms to ensure safe interaction.
Summary
An adversarial attack where malicious inputs manipulate an AI model to bypass safety filters or execute unintended commands.
Key Concepts
- Adversarial Attacks
- System Prompts
- Security Risks
- Input Sanitization
Use Cases
- LLM Security Testing
- Chatbot Safety
- Data Privacy Protection