Sycophancy
Definition
Sycophancy is a failure mode in large language models where the system prioritizes …
Evaluation
Stability
Definition
In machine learning, stability refers to the robustness of a model’s performance …
MAUVE
Definition
MAUVE is a statistical measure designed to assess how closely the output of a generative …
Leave-one-out cross-validation
Definition
Leave-one-out cross-validation (LOOCV) is a specific case of k-fold cross-validation …
Leakage
Definition
Data leakage is a critical error in machine learning where the model gains access to …
LLM-as-a-Judge
Definition
LLM-as-a-Judge is an evaluation paradigm where a Large Language Model serves as an …
Inception Score
Definition
The Inception Score (IS) is a statistical measure introduced to assess the performance of …
FrontierMath
Definition
FrontierMath is a specialized evaluation suite created to test the limits of large …
Evaluation of binary classifiers
Definition
This field involves analyzing metrics such as accuracy, precision, recall, F1-score, and …
Cross-validation
Definition
Cross-validation is a statistical method used to estimate the skill of machine learning …
Confusion matrix
Definition
A confusion matrix is a specific table layout that allows visualization of the …
Comparison of machine learning software
Definition
This term refers to the systematic assessment and benchmarking of various machine …
Category utility
Definition
This metric quantifies how well a set of categories allows one to predict the values of …
Loss Function
Definition
Also known as the cost or error function, the loss function provides a scalar value …
out-of-distribution
Definition
Out-of-distribution (OOD) detection identifies inputs that fall outside the scope of the …
high-quality
Definition
In artificial intelligence, high-quality typically describes data or model outputs that …
held-out
Definition
A ‘held-out’ dataset consists of examples intentionally excluded from the …
Test
Definition
The test set is a portion of data held out during the training process to evaluate the …
Score
Definition
Scores quantify how well a machine learning model performs against specific metrics such …
Overall
Definition
When evaluating AI models, ‘overall’ metrics provide a holistic view of …
Evidence
Definition
In artificial intelligence, evidence refers to empirical data, statistical results, or …
Benchmark
Definition
In artificial intelligence, a benchmark is a standardized test suite or dataset designed …
Bench
Definition
A benchmark serves as a standardized reference point for comparing the capabilities of …
Analysis
Definition
In the context of AI, analysis refers to the systematic examination of data, model …
Hallucination
Definition
Hallucinations occur when generative AI models produce output that appears plausible but …