Definition
XLM-RoBERTa (Cross-lingual Language Model RoBERTa) is a large-scale multilingual model developed by Meta AI. It extends the RoBERTa architecture by pre-training on a diverse dataset covering over 100 languages. This allows the model to learn shared representations across languages, enabling strong performance in cross-lingual transfer tasks. It is widely used for machine translation, multilingual classification, and zero-shot cross-lingual information retrieval without needing language-specific fine-tuning.
Summary
A multilingual transformer model based on RoBERTa, pre-trained on massive amounts of text from 100+ languages.
Key Concepts
- Multilingual NLP
- Transformer Architecture
- Cross-lingual Transfer
- Pre-training
Use Cases
- Multilingual sentiment analysis
- Cross-lingual question answering
- Machine translation