FusedProjectionSource
Models & Types · AiDotNet.ModelLoading.Pretrained
Wraps a tensor source that stores fused projections (Phi-3: a single
self_attn.qkv_proj and mlp.gate_up_proj per layer) and exposes the individual
q_proj/k_proj/v_proj and gate_proj/up_proj tensors that
LlamaModelBuilder expects — slicing the fused matrix’s output rows on read.
Any tensor already present under its split name (a non-fused checkpoint) passes straight through.
Implements: INamedTensorSource
How It Works
Fused matrices are [out, in] row-major, so an output-row range maps to a contiguous element
range: rows [r0, r1) → elements [r0*in, r1*in). QKV rows are stacked
[Q(numHeads·headDim); K(numKVHeads·headDim); V(numKVHeads·headDim)]; gate/up are
[gate(intermediate); up(intermediate)].
Constructors
| Constructor | Summary |
|---|---|
FusedProjectionSource(INamedTensorSource inner, HuggingFaceConfig config) |
Wraps inner, deriving split-projection names from config. |
Properties
| Property | Summary |
|---|---|
IReadOnlyCollection<string> TensorNames { get; } |
Methods
| Method | Summary |
|---|---|
double[] ReadAsDouble(string name) |