Tanh activation function equation: tanh(x) = (e^x - e^(-x)) / (e^x + e^(-x))
Activation function
Tanh activation function equation: tanh(x) = (e^x - e^(-x)) / (e^x + e^(-x))
The tanh activation function is a hyperbolic tangent function that maps input values to a range between -1 and 1. It is used in neural networks to introduce nonlinearity, allowing the model to learn complex patterns.
The tanh function is defined mathematically as tanh(x) = (e^x - e^(-x)) / (e^x + e^(-x)). This formula uses the exponential function e^x and its reciprocal e^(-x) to calculate the output for any given input x.
The tanh function is particularly useful in neural networks because it helps the model to converge faster during training. It also helps to mitigate the vanishing gradient problem, which can occur with other activation functions like the sigmoid function.
Example
If the input x is 1, then tanh(1) = (e^1 - e^(-1)) / (e^1 + e^(-1)) ≈ 0.7615941559557649
Remember this
Understanding the equation for the tanh activation function is important for implementing it correctly in neural networks and for analyzing its behavior during the training process.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Related concepts
Discrete Fourier transform
Discrete Fourier Transform (DFT) equation: X[k] = Σ(n=0 to N-1) x[n] * e^(-j*2π*k*n/N)
Cross-entropy
Cross-entropy loss equation: H(p, q) = -Σ(p(x) * log(q(x)))
Euler's identity
Euler's identity: e^(iπ) + 1 = 0
Expected value
Expected value formula: E[X] = Σ [x * P(x)]
Chain rule
Chain rule formula: h'(x) = z'(y(x)) * y'(x)
Taylor series
Taylor series formula: f(x) = f(a) + f'(a)(x-a) + f''(a)(x-a)^2/2! +
Swipe through 100 ML concepts daily
Open Pocket Polymath