DocumentFrequencyFS<T>

Models & Types · AiDotNet.Preprocessing.FeatureSelection.NLP

Document Frequency based feature selection for text data.

Inheritance: TransformerBase<T, Matrix<T>, Matrix<T>> → DocumentFrequencyFS<T>

Implements: IFixedCountFeatureSelector<T>

For Beginners

Some words appear in almost every document (like “is” or “the”) and don’t help distinguish documents. Other words appear only once or twice in the entire corpus and might be typos or too specific. DF filtering keeps words that appear in a reasonable number of documents - common enough to be meaningful but not so common that everyone uses them.

How It Works

Document Frequency (DF) selection filters terms based on how many documents they appear in. Terms appearing in too few or too many documents are removed, as they are either too rare to be useful or too common to be discriminative.

Constructors

Constructor Summary
DocumentFrequencyFS(int nFeaturesToSelect, int minDf, int maxDf, int[] columnIndices)

Properties

Property Summary
bool SupportsInverseTransform { get; }
int MaxDf { get; }
int MinDf { get; }
int NFeaturesToSelect { get; }
int[] DocumentFrequencies { get; }
int[] SelectedIndices { get; }

Methods

Method Summary
bool[] GetSupportMask()
string[] GetFeatureNamesOut(string[] inputFeatureNames)