AI Terms Dictionary

Definition

Compressed tensors are multi-dimensional arrays used in deep learning where the numerical precision (e.g., from float32 to int8) or sparsity has been reduced. This technique, known as quantization or pruning, significantly decreases memory footprint and accelerates inference speeds without substantially compromising model accuracy. It is essential for deploying large models on resource-constrained devices like mobile phones or edge computing hardware, enabling faster and cheaper AI operations.

Summary

Tensors whose data precision or size has been reduced to optimize storage and computational efficiency.

Key Concepts

• Quantization
• Sparsity
• Memory Optimization
• Inference Speed

Use Cases

• Mobile AI application deployment
• Edge device processing
• Large language model serving optimization

Code Example

1
2
3
4

import torch
# Example of converting a tensor to half precision
x = torch.randn(10, 10)
x_compressed = x.half()

Related Terms

• Quantization
• Pruning
• Model Distillation
• Float16