LDAVectorizer<T>

Models & Types · AiDotNet.Preprocessing.TextVectorizers

Converts text documents to topic distribution vectors using Latent Dirichlet Allocation (LDA).

Inheritance: TextVectorizerBase<T> → LDAVectorizer<T>

For Beginners

LDA discovers the “themes” in your documents:

  • A news corpus might have topics: “sports”, “politics”, “technology”
  • Each document gets a probability for each topic (e.g., 70% sports, 20% politics, 10% tech)
  • Great for topic modeling, document clustering, and content recommendation
  • Unlike LSA, LDA topics are interpretable probability distributions

How It Works

LDA is a generative probabilistic model that discovers latent topics in a corpus. Each document is represented as a mixture of topics, and each topic is characterized by a distribution over words.

The algorithm assumes:

  • Each document is generated by first choosing a topic distribution
  • Then for each word, choosing a topic from that distribution
  • Finally choosing a word from that topic’s word distribution

Constructors

Constructor Summary
LDAVectorizer(int nTopics, int maxIterations, double alpha, double beta, double tolerance, int minDf, double maxDf, int? maxFeatures, ValueTuple<int, int>? nGramRange, bool lowercase, int? randomState, Func<string, IEnumerable<string>> tokenizer, HashSet<string> stopWords, ITokenizer advancedTokenizer) Creates a new instance of LDAVectorizer.

Properties

Property Summary
bool IsFitted { get; }
double[] TopicWordDistribution { get; } Gets the topic-word distribution matrix (phi).
int FeatureCount { get; }
int NTopics { get; } Gets the number of topics.

Methods

Method Summary
Matrix<T> Transform(IEnumerable<string> documents) Transforms documents to topic distribution vectors.
string[][] GetTopWordsPerTopic(int nWords) Gets the top words for each topic.
void Fit(IEnumerable<string> documents) Fits the LDA model to the corpus using collapsed Gibbs sampling.