ZScoreClipper<T>

Models & Types ยท AiDotNet.Preprocessing.OutlierHandling

Clips outliers based on Z-score (standard deviation) bounds.

Inheritance: TransformerBase<T, Matrix<T>, Matrix<T>> โ†’ ZScoreClipper<T>

For Beginners

Z-score measures how many standard deviations a value is from the mean.

  • A Z-score of 0 means the value equals the mean
  • A Z-score of 2 means the value is 2 standard deviations above the mean
  • A Z-score of -3 means the value is 3 standard deviations below the mean

This clipper replaces extreme values (those with high absolute Z-scores) with the boundary values, reducing the impact of outliers while preserving data size.

How It Works

ZScoreClipper identifies outliers as values that deviate from the mean by more than a specified number of standard deviations. Values beyond this threshold are clipped to the boundary values.

How It Works: For each feature:

  1. Calculate the mean and standard deviation
  2. Compute bounds: [mean - thresholdstd, mean + thresholdstd]
  3. Clip values outside these bounds

Common Thresholds:

  • 2.0: Aggressive (clips ~5% of normally distributed data)
  • 3.0: Standard (clips ~0.3% of normally distributed data)
  • 3.5: Conservative (clips ~0.05% of normally distributed data)

Constructors

Constructor Summary
ZScoreClipper(double threshold, int[] columnIndices) Creates a new instance of ZScoreClipper.

Properties

Property Summary
Vector<T> LowerBounds { get; } Gets the computed lower bounds for each feature.
Vector<T> Means { get; } Gets the computed means for each feature.
Vector<T> StandardDeviations { get; } Gets the computed standard deviations for each feature.
Vector<T> UpperBounds { get; } Gets the computed upper bounds for each feature.
bool SupportsInverseTransform { get; } Gets whether this transformer supports inverse transformation.
double Threshold { get; } Gets the Z-score threshold for clipping.

Methods

Method Summary
Matrix<T> GetZScores(Matrix<T> data) Calculates the Z-scores for all values in the data.
bool[] GetOutlierMask(Matrix<T> data) Gets a boolean mask indicating which values are outliers.
string[] GetFeatureNamesOut(string[] inputFeatureNames) Gets the output feature names after transformation.