TFIDFSelector<T>

Models & Types ยท AiDotNet.Preprocessing.FeatureSelection.Text

TF-IDF based feature selection for text/document data.

Inheritance: TransformerBase<T, Matrix<T>, Matrix<T>> โ†’ TFIDFSelector<T>

Implements: IFixedCountFeatureSelector<T>

For Beginners

TF-IDF finds words that are frequent in specific documents but rare overall. These terms help distinguish documents and are better features than common words appearing everywhere.

How It Works

Uses Term Frequency-Inverse Document Frequency to select terms that are important in some documents but not ubiquitous across all documents.

Constructors

Constructor Summary
TFIDFSelector(int nFeaturesToSelect, double minDF, double maxDF, int[] columnIndices)

Properties

Property Summary
Vector<T> IDFScores { get; }
Vector<T> MaxTFIDF { get; }
bool SupportsInverseTransform { get; }
int NFeaturesToSelect { get; }
int[] SelectedIndices { get; }

Methods

Method Summary
Matrix<T> FitTransform(Matrix<T> data)
bool[] GetSupportMask()
string[] GetFeatureNamesOut(string[] inputFeatureNames)