LDAVectorizer<T>
Models & Types · AiDotNet.Preprocessing.TextVectorizers
Converts text documents to topic distribution vectors using Latent Dirichlet Allocation (LDA).
Inheritance: TextVectorizerBase<T> → LDAVectorizer<T>
For Beginners
LDA discovers the “themes” in your documents:
- A news corpus might have topics: “sports”, “politics”, “technology”
- Each document gets a probability for each topic (e.g., 70% sports, 20% politics, 10% tech)
- Great for topic modeling, document clustering, and content recommendation
- Unlike LSA, LDA topics are interpretable probability distributions
How It Works
LDA is a generative probabilistic model that discovers latent topics in a corpus. Each document is represented as a mixture of topics, and each topic is characterized by a distribution over words.
The algorithm assumes:
- Each document is generated by first choosing a topic distribution
- Then for each word, choosing a topic from that distribution
- Finally choosing a word from that topic’s word distribution
Constructors
| Constructor | Summary |
|---|---|
LDAVectorizer(int nTopics, int maxIterations, double alpha, double beta, double tolerance, int minDf, double maxDf, int? maxFeatures, ValueTuple<int, int>? nGramRange, bool lowercase, int? randomState, Func<string, IEnumerable<string>> tokenizer, HashSet<string> stopWords, ITokenizer advancedTokenizer) |
Creates a new instance of LDAVectorizer. |
Properties
| Property | Summary |
|---|---|
bool IsFitted { get; } |
|
double[] TopicWordDistribution { get; } |
Gets the topic-word distribution matrix (phi). |
int FeatureCount { get; } |
|
int NTopics { get; } |
Gets the number of topics. |
Methods
| Method | Summary |
|---|---|
Matrix<T> Transform(IEnumerable<string> documents) |
Transforms documents to topic distribution vectors. |
string[][] GetTopWordsPerTopic(int nWords) |
Gets the top words for each topic. |
void Fit(IEnumerable<string> documents) |
Fits the LDA model to the corpus using collapsed Gibbs sampling. |