EmbeddingModelTokenAdapter<T>
Models & Types · AiDotNet.RetrievalAugmentedGeneration.Embeddings
Adapts any real single-vector IEmbeddingModel into a per-token
ITokenEmbeddingModel by tokenizing the text and embedding each token through the
underlying model. This is what lets ColBERT-style late interaction run over the library’s existing
real embedders (Word2Vec / GloVe / FastText / ONNX sentence-transformer / hosted providers) instead of
any placeholder — the token vectors are exactly the real model’s output for each token.
Implements: ITokenEmbeddingModel<T>
For Beginners
ColBERT scores a query against a document by comparing them token-by-token, so it needs a vector per word. Most embedding models return a single vector for a whole string. This adapter bridges the two: it splits the text into words and asks the real model to embed each word, giving you genuine per-token embeddings with no fake/placeholder data.
How It Works
Contextual (transformer) token embeddings are strictly stronger than embedding words in isolation; supply a contextual token model directly if you have one. This adapter makes any word/sentence embedder usable, which is the correct real fallback — not a synthetic hash.
Constructors
| Constructor | Summary |
|---|---|
EmbeddingModelTokenAdapter(IEmbeddingModel<T> embeddingModel) |
Creates a token-embedding adapter over a real single-vector embedding model. |
Properties
| Property | Summary |
|---|---|
int EmbeddingDimension { get; } |
Methods
| Method | Summary |
|---|---|
Vector<T>[] EmbedTokens(string text) |