XLM-RoBERTa
Definition
XLM-RoBERTa (Cross-lingual Language Model RoBERTa) is a large-scale multilingual model …
Transformers
Prefix Tuning
Definition
Prefix Tuning is a parameter-efficient adaptation technique for pre-trained transformers. …
Long Context
Definition
Long context refers to the capacity of transformer-based models to handle extensive input …
Fill Mask
Definition
Fill Mask is a fundamental pre-training objective used in transformer-based models like …
ExBERT
Definition
ExBERT provides interpretability for the BERT transformer model by analyzing the …
Positional Encoding
Definition
Since transformers process all tokens in parallel rather than sequentially like RNNs, …
Encoder
Definition
Encoders process raw input sequences or data structures and convert them into latent …
Adapter
Definition
Adapters are a parameter-efficient fine-tuning technique used primarily in large language …
Attention
Definition
Attention mechanisms enable models to focus on relevant information when processing …
Self-Attention
Definition
Self-attention enables models to capture dependencies between all positions in a sequence …