Definition
ELMo generates context-sensitive word embeddings by processing input text through a bidirectional LSTM trained on a large corpus. Unlike static embeddings like Word2Vec, ELMo captures polysemy by producing different vector representations for the same word depending on its surrounding context. This approach significantly improved performance on various NLP benchmarks by allowing downstream tasks to leverage rich, dynamic linguistic features extracted from pre-trained language models.
Summary
Embeddings from Language Models, a deep contextualized word representation method using bidirectional LSTMs.
Key Concepts
- Contextual Embeddings
- Bidirectional LSTM
- Pre-training
- Polysemy
Use Cases
- Sentiment analysis
- Named entity recognition
- Coreference resolution