GgufModelSource
Models & Types · AiDotNet.ModelLoading.Pretrained
Opens a GGUF (llama.cpp) checkpoint as a decoder-ready source: it reads the architecture from the GGUF
metadata into a HuggingFaceConfig and exposes the weights under the Hugging Face parameter
names LlamaModelBuilder expects, translating each read to the GGUF tensor name
(token_embd.weight, blk.N.attn_q.weight, …).
Implements: IDisposable, INamedTensorSource
For Beginners
GGUF is the format llama.cpp uses. This class understands a GGUF file well enough to rebuild the same model inside AiDotNet and serve it.
How It Works
GGUF stores linear weights [out, in] row-major — identical to Hugging Face safetensors — so no
re-layout is needed; the builder’s usual [out,in]→[in,out] transpose applies unchanged. Quantized
tensors (Q4_0/Q4_1/Q8_0/Q4_K/Q6_K) and F16/F32 are dequantized on read by GgufFile.
Dispose the source to close the file handle.
Properties
| Property | Summary |
|---|---|
HuggingFaceConfig Config { get; } |
The architecture read from the GGUF metadata. |
IReadOnlyCollection<string> TensorNames { get; } |
Methods
| Method | Summary |
|---|---|
GgufModelSource Open(string path) |
Opens a GGUF file and prepares its config + Hugging Face name mapping. |
ITokenizer BuildTokenizer() |
Builds the model’s tokenizer from the GGUF tokenizer.ggml.* metadata: the vocabulary (tokens), BPE merge ranks (merges), added/control tokens (token_type), and the BOS/EOS/UNK/PAD ids. |
bool TryReadQ8_0(string hfName, sbyte[] qs, float[] scales) |
Reads a directly-mapped tensor’s native Q8_0 blocks (int8 payload + one fp32 scale per 32-value block) WITHOUT dequantizing, resolving the Hugging-Face hfName to its GGUF name. |
double[] ReadAsDouble(string name) |
|
void Dispose() |