TextFeatureSelector<T>

Models & Types · AiDotNet.Preprocessing.FeatureSelection.DomainSpecific

Feature selection for text/NLP data.

Inheritance: TransformerBase<T, Matrix<T>, Matrix<T>> → TextFeatureSelector<T>

Implements: IFixedCountFeatureSelector<T>, ISupervisedFeatureSelector<T>

For Beginners

Text data has unique properties - some words are common but uninformative (like “the”), while rare words might be very distinctive. This selector picks words/features that best distinguish between different text categories.

How It Works

TextFeatureSelector is designed for selecting features from text data, such as word frequencies, TF-IDF scores, or word embeddings. It considers text-specific properties like term frequency and document frequency.

Constructors

Constructor Summary
TextFeatureSelector(int nFeaturesToSelect, double minDocumentFrequency, double maxDocumentFrequency, bool useChiSquare, int[] columnIndices)

Properties

Property Summary
Vector<T> DocumentFrequencies { get; }
Vector<T> FeatureScores { get; }
bool SupportsInverseTransform { get; }
bool UseChiSquare { get; }
double MaxDocumentFrequency { get; }
double MinDocumentFrequency { get; }
int NFeaturesToSelect { get; }
int[] SelectedIndices { get; }

Methods

Method Summary
Matrix<T> FitTransform(Matrix<T> data, Vector<T> target)
bool[] GetSupportMask()
string[] GetFeatureNamesOut(string[] inputFeatureNames)
void Fit(Matrix<T> data, Vector<T> target)