TextFeatureSelector<T>
Models & Types · AiDotNet.Preprocessing.FeatureSelection.DomainSpecific
Feature selection for text/NLP data.
Inheritance: TransformerBase<T, Matrix<T>, Matrix<T>> → TextFeatureSelector<T>
Implements: IFixedCountFeatureSelector<T>, ISupervisedFeatureSelector<T>
For Beginners
Text data has unique properties - some words are common but uninformative (like “the”), while rare words might be very distinctive. This selector picks words/features that best distinguish between different text categories.
How It Works
TextFeatureSelector is designed for selecting features from text data, such as word frequencies, TF-IDF scores, or word embeddings. It considers text-specific properties like term frequency and document frequency.
Constructors
| Constructor | Summary |
|---|---|
TextFeatureSelector(int nFeaturesToSelect, double minDocumentFrequency, double maxDocumentFrequency, bool useChiSquare, int[] columnIndices) |
Properties
| Property | Summary |
|---|---|
Vector<T> DocumentFrequencies { get; } |
|
Vector<T> FeatureScores { get; } |
|
bool SupportsInverseTransform { get; } |
|
bool UseChiSquare { get; } |
|
double MaxDocumentFrequency { get; } |
|
double MinDocumentFrequency { get; } |
|
int NFeaturesToSelect { get; } |
|
int[] SelectedIndices { get; } |
Methods
| Method | Summary |
|---|---|
Matrix<T> FitTransform(Matrix<T> data, Vector<T> target) |
|
bool[] GetSupportMask() |
|
string[] GetFeatureNamesOut(string[] inputFeatureNames) |
|
void Fit(Matrix<T> data, Vector<T> target) |