AI Terms Dictionary

Definition

Encoders process raw input sequences or data structures and convert them into latent space representations, often called embeddings or codes. They are central to architectures like Transformers and Autoencoders. The encoder’s goal is to capture essential features and contextual information while discarding noise, creating a compact summary that downstream components, such as decoders or classifiers, can utilize effectively for prediction or generation tasks.

Summary

An encoder is a component of a neural network that transforms input data into a compressed, meaningful representation.

Key Concepts

• Feature Extraction
• Latent Space
• Sequence Processing
• Compression

Use Cases

• Processing input text in Transformer models
• Compressing images in autoencoders for denoising
• Extracting sentiment features from reviews

Code Example

1
2
3
4
5
6
7
8
9

import torch.nn as nn

class SimpleEncoder(nn.Module):
    def __init__(self, input_dim, hidden_dim):
        super().__init__()
        self.fc = nn.Linear(input_dim, hidden_dim)
    
    def forward(self, x):
        return torch.relu(self.fc(x))

Related Terms

• Decoder
• Transformer
• Autoencoder
• Latent Variable