TfIdfFS<T>

Models & Types · AiDotNet.Preprocessing.FeatureSelection.NLP

TF-IDF based feature selection for text data.

Inheritance: TransformerBase<T, Matrix<T>, Matrix<T>> → TfIdfFS<T>

Implements: IFixedCountFeatureSelector<T>

For Beginners

In text data, some words appear often but are meaningless (like “the”), while others are rare but important. TF-IDF balances term frequency (how often a word appears in a document) with inverse document frequency (how rare it is across all documents). Words that are common in some documents but rare overall get high scores.

How It Works

TF-IDF (Term Frequency-Inverse Document Frequency) based selection identifies features (terms) that are both frequent within documents and discriminative across the corpus. Features with high TF-IDF scores are informative for distinguishing documents.

Constructors

Constructor Summary
TfIdfFS(int nFeaturesToSelect, double minDf, double maxDf, int[] columnIndices)

Properties

Property Summary
Vector<T> TfIdfScores { get; }
bool SupportsInverseTransform { get; }
double MaxDf { get; }
double MinDf { get; }
int NFeaturesToSelect { get; }
int[] SelectedIndices { get; }

Methods

Method Summary
bool[] GetSupportMask()
string[] GetFeatureNamesOut(string[] inputFeatureNames)