NumericalStabilityHelper

Helpers & Utilities · AiDotNet.Helpers

Provides numerical stability utilities for safe mathematical operations in machine learning.

For Beginners

Machine learning algorithms often deal with very small or very large numbers, which can cause numerical issues like:

  • Division by zero
  • Log of zero or negative numbers
  • NaN (Not a Number) values appearing in calculations
  • Infinity values from overflow

This helper provides safe versions of common operations that avoid these problems.

Methods

Method Summary
T ClampProbability(T probability, double epsilon) Clamps a value to valid probability range [epsilon, 1-epsilon].
T GetEpsilon(double? epsilon) Gets a type-appropriate epsilon value for the numeric type T.
T SafeDiv(T numerator, T denominator, double epsilon) Performs safe division, avoiding division by zero.
T SafeLog(T value, double epsilon) Computes the natural logarithm safely, avoiding log(0) and log(negative).
T SafeLogProbability(T probability, double epsilon) Computes safe log of a probability (clamps first, then takes log).
T SafeSqrt(T value, double epsilon) Computes square root safely, ensuring non-negative input.
Vector<T> ReplaceInfinity(Vector<T> vector, T replacement) Replaces infinite values in a vector with a specified replacement value.
Vector<T> ReplaceNaN(Vector<T> vector, T replacement) Replaces NaN values in a vector with a specified replacement value.
Vector<T> ReplaceNonFinite(Vector<T> vector, T replacement) Replaces all non-finite values (NaN and infinity) in a vector.
Vector<T> StableLogSoftmax(Vector<T> logits) Computes log-softmax with numerical stability.
Vector<T> StableSoftmax(Vector<T> logits) Computes softmax with numerical stability using the log-sum-exp trick.
bool ContainsInfinity(Tensor<T> tensor) Checks if a tensor contains any infinite values.
bool ContainsInfinity(Vector<T> vector) Checks if a tensor contains any infinite values.
bool ContainsNaN(Tensor<T> tensor) Checks if a tensor contains any NaN values.
bool ContainsNaN(Vector<T> vector) Checks if a tensor contains any NaN values.
bool ContainsNonFinite(Tensor<T> tensor) Checks if a tensor contains any non-finite values (NaN or infinite).
bool ContainsNonFinite(Vector<T> vector) Checks if a tensor contains any non-finite values (NaN or infinite).
bool IsFinite(T value) Checks if a value is finite (not NaN and not infinite).
bool IsInfinity(T value) Checks if a value is infinite (positive or negative infinity).
bool IsNaN(T value) Checks if a value is NaN (Not a Number).
int CountInfinity(Vector<T> vector) Counts the number of infinite values in a vector.
int CountNaN(Vector<T> vector) Counts the number of NaN values in a vector.
void AssertFinite(T value, string paramName) Asserts that a tensor contains only finite values.
void AssertFinite(Tensor<T> tensor, string paramName) Asserts that a tensor contains only finite values.
void AssertFinite(Vector<T> vector, string paramName) Asserts that a tensor contains only finite values.

Fields

Field Summary
double DefaultEpsilon Default epsilon value for numerical stability (1e-7 for float precision).
double LargeEpsilon Larger epsilon for less sensitive operations (1e-5).
double SmallEpsilon Smaller epsilon for double precision operations (1e-15).