AI Terms Dictionary

Definition

The hyperbolic tangent (Tanh) function is a non-linear activation function commonly used in neural networks. It squashes input values into the interval (-1, 1), providing zero-centered outputs which can help mitigate the vanishing gradient problem compared to sigmoid functions. Tanh is differentiable everywhere, making it suitable for backpropagation. It is frequently used in recurrent neural networks (RNNs) and LSTM cells to regulate information flow within the network architecture.

Summary

Tanh, or hyperbolic tangent, is an activation function that maps input values to a range between -1 and 1.

Key Concepts

• Activation function
• Non-linearity
• Zero-centered output
• Backpropagation

Use Cases

• Recurrent neural networks
• LSTM cell gates
• Hidden layers in MLPs

Code Example

1
2
3

import numpy as np
def tanh(x):
    return np.tanh(x)

Related Terms

• sigmoid
• relu
• neural_networks