MishActivation<T>
Models & Types · AiDotNet.ActivationFunctions
Implements the Mish activation function for neural networks.
Inheritance: ActivationFunctionBase<T> → MishActivation<T>
Implements: IFusedActivation
For Beginners
The Mish activation function is a smooth, non-monotonic function that helps neural networks learn complex patterns. It was introduced in 2019 and has shown good performance in many applications.
Mathematically, Mish is defined as: f(x) = x * tanh(softplus(x)) where softplus(x) = ln(1 + e^x)
Mish combines properties of several popular activation functions:
- It’s smooth (no sharp corners like ReLU)
- It allows both positive and negative values (unlike ReLU which zeros out negatives)
- It’s unbounded on the positive side (can output large positive values)
- It’s bounded on the negative side (won’t output extremely negative values)
These properties help neural networks learn more effectively in many situations.
Example
using AiDotNet.ActivationFunctions;
var activation = new MishActivation<double>();
double y = activation.Activate(0.5);
Console.WriteLine($"MishActivation: f(0.5) = {y:F4}");
Constructors
| Constructor | Summary |
|---|---|
MishActivation() |
Properties
| Property | Summary |
|---|---|
bool SupportsGpuTraining { get; } |
Gets whether Mish supports GPU-resident training. |
Methods
| Method | Summary |
|---|---|
T Activate(T input) |
Applies Mish to a tensor via the engine so the gradient tape records the op. |
T Derivative(T input) |
Calculates the derivative (gradient) of the Mish function for a single input value. |
Tensor<T> Activate(Tensor<T> input) |
Applies Mish to a tensor via the engine so the gradient tape records the op. |
void ForwardGpu(IDirectGpuBackend backend, IGpuBuffer input, IGpuBuffer output, int size) |
Applies the Mish activation function on GPU. |