Gemma2ModelBuilder<T>
Helpers & Utilities · AiDotNet.ModelLoading.Pretrained
Reconstructs a Gemma-2 decoder from a Hugging Face HuggingFaceConfig and loads its weights.
Gemma-2 differs from the dense LLaMA stack by sandwiched RMSNorms (norm before AND after each
sublayer), GeGLU FFN, the Gemma (1 + weight) RMSNorm and √hidden embedding scale, and final-logit
soft-capping.
How It Works
Applied faithfully: 4-norm sandwich blocks, (1+weight) norms, √hidden embedding scale, GeGLU, explicit
head_dim, and final-logit soft-capping (cap·tanh(logits/cap)). Not yet applied (numerical
refinements): attention-logit soft-capping and alternating sliding-window attention — within the trained
window the sliding mask equals the causal mask, so short-context generation is unaffected.
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedArchitectures { get; } |
The architecture family names this builder recognizes. |
Methods
| Method | Summary |
|---|---|
NeuralNetwork<T> Build(HuggingFaceConfig config, INamedTensorSource weights, int warmupSequenceLength) |
Builds and weight-loads a Gemma-2 decoder. |