CohereModelBuilder<T>

Helpers & Utilities · AiDotNet.ModelLoading.Pretrained

Reconstructs a Cohere (Command-R) decoder from a Hugging Face HuggingFaceConfig and loads its weights. Cohere differs from the dense LLaMA stack by using true (mean-centered) LayerNorm instead of RMSNorm, a parallel residual (one norm feeds both attention and FFN), tied embeddings, and a logit_scale multiplier on the output.

How It Works

Bias-free (matching Command-R). The logit_scale is baked into the (tied) LM-head weights on load. QK-normalization (some Command-R+ variants) is not applied.

Properties

Property Summary
IReadOnlyList<string> SupportedArchitectures { get; } The architecture family names this builder recognizes.

Methods

Method Summary
NeuralNetwork<T> Build(HuggingFaceConfig config, INamedTensorSource weights, int warmupSequenceLength) Builds and weight-loads a Cohere decoder.