Gemma2ModelBuilder<T>

Helpers & Utilities · AiDotNet.ModelLoading.Pretrained

Reconstructs a Gemma-2 decoder from a Hugging Face HuggingFaceConfig and loads its weights. Gemma-2 differs from the dense LLaMA stack by sandwiched RMSNorms (norm before AND after each sublayer), GeGLU FFN, the Gemma (1 + weight) RMSNorm and √hidden embedding scale, and final-logit soft-capping.

How It Works

Applied faithfully: 4-norm sandwich blocks, (1+weight) norms, √hidden embedding scale, GeGLU, explicit head_dim, and final-logit soft-capping (cap·tanh(logits/cap)). Not yet applied (numerical refinements): attention-logit soft-capping and alternating sliding-window attention — within the trained window the sliding mask equals the causal mask, so short-context generation is unaffected.

Properties

Property Summary
IReadOnlyList<string> SupportedArchitectures { get; } The architecture family names this builder recognizes.

Methods

Method Summary
NeuralNetwork<T> Build(HuggingFaceConfig config, INamedTensorSource weights, int warmupSequenceLength) Builds and weight-loads a Gemma-2 decoder.