DbrxModelBuilder<T>

Helpers & Utilities · AiDotNet.ModelLoading.Pretrained

Reconstructs a DBRX decoder from a Hugging Face HuggingFaceConfig and loads its weights. DBRX is a LayerNorm sparse-MoE decoder that fuses q/k/v (Wqkv) and stacks all experts into single tensors; a DbrxTensorSource (wrapped in a FusedProjectionSource) exposes the weights under standard Hugging Face names so the usual loaders apply.

How It Works

Bias-free attention + LayerNorm norms. The clip_qkv post-projection clamp is not applied (it rarely binds at DBRX’s default clip and would require attention-layer support).

Properties

Property Summary
IReadOnlyList<string> SupportedArchitectures { get; } The architecture family names this builder recognizes.

Methods

Method Summary
NeuralNetwork<T> Build(HuggingFaceConfig config, INamedTensorSource rawWeights, int warmupSequenceLength) Builds and weight-loads a DBRX decoder.