Curriculum Learning
Definition
Curriculum learning mimics human education by presenting training data in a structured …
Optimization
Voice Activity Detection
Definition
VAD algorithms analyze audio streams in real-time to distinguish between active speech …
Unsloth
Definition
Unsloth is a specialized tool designed to optimize the fine-tuning and deployment of …
Vllm
Definition
vLLM (Virtual Large Language Model) is an open-source library designed to accelerate LLM …
Token maxxing
Definition
Token maxxing involves carefully crafting inputs to utilize the full capacity of a …
Three-factor learning
Definition
Three-factor learning is a specific approach within reinforcement learning that …
TFLite
Definition
TensorFlow Lite is an open-source framework designed to deploy machine learning models on …
Text Generation Inference
Definition
Text Generation Inference (TGI) is a dedicated software framework designed to serve large …
Symbolic regression
Definition
Symbolic regression is a type of regression analysis that seeks to find a mathematical …
Surrogate model
Definition
In machine learning and optimization, a surrogate model serves as a proxy for a target …
Structural risk minimization
Definition
Structural risk minimization (SRM) is a method for minimizing expected risk by …
Structured sparsity regularization
Definition
Structured sparsity regularization extends standard L1 regularization by encouraging …
Semantic folding
Definition
Semantic folding refers to the process of compressing complex, high-dimensional vector …
Reparameterization trick
Definition
The reparameterization trick is a fundamental method used in variational autoencoders and …
Regularization
Definition
Regularization is a crucial concept in machine learning designed to reduce generalization …
Random feature
Definition
Random feature maps transform inputs into a new space where linear models can approximate …
Quantized
Definition
Quantization is a model optimization technique that reduces the numerical precision of a …
Prompt Tuning
Definition
Prompt tuning involves adding trainable soft prompts (continuous vectors) to the input …
Proximal gradient methods for learning
Definition
Proximal gradient methods are iterative optimization techniques used when the loss …
PagedAttention
Definition
PagedAttention is a technique introduced by the vLLM project to improve the efficiency of …
Openvino
Definition
Developed by Intel, OpenVINO (Open Visual Inference and Neural network Optimization) …
Mxfp4
Definition
MXFP4 (Mixed eXtended Floating Point 4-bit) is a specialized data type format introduced …
Multiplicative weight update method
Definition
The multiplicative weight update method is a fundamental online learning algorithm used …
Multi-task Learning
Definition
This technique leverages the inductive bias shared among related tasks to enhance …
MobileNet
Definition
MobileNets utilize depthwise separable convolutions to drastically reduce computational …
Model Compression
Definition
This category includes methods like pruning, quantization, and knowledge distillation …
Mixed Precision Training
Definition
Mixed Precision Training (MPT) combines half-precision (FP16) and full-precision (FP32) …
Meta-learning
Definition
Meta-learning focuses on designing algorithms that can learn from previous tasks to …
Matrix regularization
Definition
Matrix regularization extends scalar regularization concepts to matrices, often used in …
Local case-control sampling
Definition
Local case-control sampling is a strategy used primarily in training contrastive learning …
Lottery ticket hypothesis
Definition
The Lottery Ticket Hypothesis suggests that within a large, randomly initialized neural …
Learning automaton
Definition
This concept originates from reinforcement learning and involves an agent interacting …
Layer Normalization
Definition
Layer Normalization stabilizes training by reducing internal covariate shift, …
Knowledge Compilation
Definition
Knowledge compilation refers to techniques in artificial intelligence that convert a …
Knowledge Distillation
Definition
Knowledge distillation is a machine learning method used to compress a large, complex …
Incremental Heuristic Search
Definition
Incremental Heuristic Search refers to algorithms that refine a candidate solution …
Instance selection
Definition
Instance selection aims to improve computational efficiency and model performance by …
Imatrix
Definition
Imatrix, short for Importance Matrix, is a technique primarily associated with GGML-based …
Hyperparameter optimization
Definition
Hyperparameter Optimization (HPO) refers to the broader field of automating the selection …
Hyperparameter Tuning
Definition
Hyperparameter tuning involves evaluating different sets of hyperparameters to find the …
Hierarchical Risk Parity
Definition
Hierarchical Risk Parity (HRP) is a portfolio construction method that addresses the …
Gradient Accumulation
Definition
This optimization strategy allows deep learning models to be trained with effective batch …
GLM MoE DSA
Definition
There is no single standard term ‘GLM MoE DSA’. However, it likely combines …
GGUF
Definition
GGUF (GPT-Generated Unified Format) is a binary file format designed specifically for …
Fp8
Definition
Floating-point 8 (FP8) is a numerical data type that offers a balance between …
Finetuned
Definition
Finetuning refers to the technique of taking a model that has already been trained on a …
Fitness approximation
Definition
Fitness approximation is used in evolutionary computation when evaluating the true …
Feature hashing
Definition
Feature hashing, also known as the hashing trick, allows machine learning models to …
Feature scaling
Definition
Feature scaling standardizes the range of input variables to prevent features with larger …
Feature Engineering
Definition
Feature engineering is the art of leveraging domain expertise to transform raw data into …
Exploration–exploitation dilemma
Definition
In decision-making processes, agents face a trade-off: they can exploit current knowledge …
Extremal optimization
Definition
Unlike genetic algorithms that maintain a population, EO works on a single solution. It …
Evolvability
Definition
In computational contexts, evolvability refers to how easily an algorithm or neural …
Empirical risk minimization
Definition
Empirical Risk Minimization (ERM) is the standard objective function for training …
Edge inference
Definition
This practice involves deploying trained AI models directly onto hardware such as …
EfficientNet
Definition
Developed by Google, EfficientNet uses a compound scaling method to balance network …
Early Stopping
Definition
Early stopping is a form of regularization used primarily in iterative training processes …
EM algorithm and GMM model
Definition
This term refers to the synergistic relationship between the Expectation-Maximization …
Differentially private stochastic gradient descent
Definition
DP-SGD is a variant of Stochastic Gradient Descent designed to protect the privacy of …
Diffusers:Ltxpipeline
Definition
The LTX pipeline is tailored for models that prioritize speed and efficiency in …
Decision tree pruning
Definition
Pruning is a method used to prevent overfitting in decision tree models by removing …
Cross-entropy method
Definition
The Cross-Entropy Method (CEM) is a powerful general-purpose optimization algorithm used …
Cost-sensitive machine learning
Definition
Cost-sensitive machine learning extends traditional supervised learning by assigning …
Contrastive Learning
Definition
Contrastive learning is a representation learning method that does not require labeled …
Compressed Tensors
Definition
Compressed tensors are multi-dimensional arrays used in deep learning where the numerical …
Computational heuristic intelligence
Definition
Computational heuristic intelligence involves algorithms that employ rules of thumb, …
Clip
Definition
In deep learning engineering, clipping is commonly applied to gradients to mitigate the …
Caching
Definition
In AI engineering, caching optimizes performance by keeping recent or frequent query …
Batch Size
Definition
Batch size is a critical hyperparameter that determines how many samples are processed …
Bayesian optimization
Definition
Bayesian optimization uses a probabilistic surrogate model, typically a Gaussian Process, …
Batch Normalization
Definition
This method adjusts and scales activations to have zero mean and unit variance within …
AlphaChip
Definition
AlphaChip is a specialized AI system designed to automate and enhance the placement and …
Algorithm selection
Definition
Algorithm selection involves evaluating different computational approaches to determine …
Admissible heuristic
Definition
In pathfinding and search problems, an admissible heuristic provides a lower bound on the …
Accelerated Linear Algebra
Definition
This field focuses on speeding up fundamental linear algebra computations, which are core …
A/B Testing
Definition
A/B testing is a randomized controlled experiment where two variants, A and B, are …
QLoRA
Definition
QLoRA combines Low-Rank Adaptation (LoRA) with 4-bit quantization to significantly reduce …
Quantization
Definition
Quantization converts high-precision floating-point numbers (like FP32) into …
Residual Connection
Definition
Residual connections, also known as skip connections, allow gradients to flow through a …
Learning Rate
Definition
The learning rate determines how much the model’s weights are updated relative to …
Gradient Descent
Definition
Gradient descent is a first-order iterative optimization algorithm for finding a local …
Distributed Training
Definition
Distributed Training accelerates model convergence by parallelizing computation over …
Adapter
Definition
Adapters are a parameter-efficient fine-tuning technique used primarily in large language …
task-specific
Definition
Task-specific refers to AI models or components tailored to excel at a narrow set of …
one-step
Definition
In machine learning and optimization, one-step methods solve problems directly without …
first-order
Definition
In artificial intelligence and mathematics, ‘first-order’ typically describes …
fine-tuned
Definition
Fine-tuning involves taking a model that has already been trained on a large, general …
Towards
Definition
In AI development, ’towards’ often describes the trajectory of optimization …
Transfer Learning
Definition
Transfer learning leverages pre-trained models to improve performance and reduce training …
Tuning
Definition
Tuning involves refining a machine learning model to achieve better accuracy or …
Rate
Definition
In AI, ‘rate’ most frequently refers to the learning rate, a hyperparameter …
Scaling
Definition
Scaling is the active methodology of expanding AI systems by adding more layers, neurons, …
Optimal
Definition
In AI and optimization theory, an optimal solution is one that achieves the highest …
Loss
Definition
Loss functions, also known as cost functions, measure how well a machine learning …
LoRA
Definition
LoRA freezes pre-trained model weights and inserts trainable decomposition matrices into …
Global
Definition
The term ‘global’ in AI typically contrasts with ’local,’ …
Fast
Definition
The term ‘fast’ describes computational efficiency within artificial …
Efficient
Definition
Efficiency is a critical metric in artificial intelligence that measures how well a model …
Distillation
Definition
This process involves transferring knowledge from a complex, high-performance …
Divergence
Definition
In the context of optimization, divergence occurs when the parameters of a model update …
Combining
Definition
This concept encompasses methods like ensemble learning, where predictions from several …
Adam
Definition
Adam (Adaptive Moment Estimation) is a popular first-order gradient-based optimization …
Fine-tuning
Definition
Fine-tuning involves taking a model already trained on a large, general dataset and …
Prompt Engineering
Definition
Prompt engineering involves crafting specific inputs, known as prompts, to elicit …