DbrxTensorSource
Models & Types · AiDotNet.ModelLoading.Pretrained
Adapts a DBRX checkpoint’s tensors to the Hugging Face parameter names the pretrained builders expect.
DBRX names blocks transformer.blocks.{i}.*, fuses q/k/v into a single attn.Wqkv, and stacks
all experts’ gate/up/down into single tensors (ffn.experts.mlp.w1/v1/w2); this source renames them,
slices each expert out, and transposes the down projection so the standard loaders can consume them.
Implements: INamedTensorSource
How It Works
The stacked expert tensors are [num_experts · ffn, hidden] row-major, so expert e occupies
the element range [e·ffn·hidden, (e+1)·ffn·hidden). w1/v1 are [out=ffn, in=hidden]
(exposed as-is for the usual [out,in]→[in,out] load transpose); w2 is stored [ffn, hidden]
(i.e. already [in,out]), so it is transposed to [hidden, ffn] here to match the down-projection
convention. The fused Wqkv is exposed as self_attn.qkv_proj.weight for a wrapping
FusedProjectionSource to split.
Constructors
| Constructor | Summary |
|---|---|
DbrxTensorSource(INamedTensorSource inner, HuggingFaceConfig config) |
Wraps a raw DBRX tensor source, deriving the Hugging Face name mapping from config. |
Properties
| Property | Summary |
|---|---|
IReadOnlyCollection<string> TensorNames { get; } |
Methods
| Method | Summary |
|---|---|
double[] ReadAsDouble(string name) |