LlamaModelBuilder<T>

Helpers & Utilities · AiDotNet.ModelLoading.Pretrained

Reconstructs a LLaMA-family decoder (LLaMA / Mistral / Qwen2) from a Hugging Face HuggingFaceConfig and loads its pretrained weights from a safetensors/GGUF tensor source. The result is a plain NeuralNetwork whose layer stack is the canonical servable decoder — [EmbeddingLayer, PreLNTransformerBlock×N, RMSNorm, DenseLayer] — so it runs through the existing paged / continuous-batching / tensor-parallel serving path unchanged.

How It Works

Faithfulness: RMSNorm + RoPE grouped-query attention + gated SwiGLU FFN, all bias-free, matching the reference architecture (Touvron 2023; Shazeer 2020). Weight layouts follow the AiDotNet layer conventions: attention and dense projections store [in, out], so Hugging Face’s [out, in] matrices are transposed on load; the embedding table is [vocab, hidden] in both.

Properties

Property Summary
IReadOnlyList<string> SupportedArchitectures { get; } The architecture family names this builder recognizes (matched against config.json’s architectures[0] / model_type, case-insensitively).

Methods

Method Summary
NeuralNetwork<T> Build(HuggingFaceConfig config, INamedTensorSource weights, DecoderOptions<T> options, int warmupSequenceLength) Builds and weight-loads a decoder for config from weights.