FusedProjectionSource

Models & Types · AiDotNet.ModelLoading.Pretrained

Wraps a tensor source that stores fused projections (Phi-3: a single self_attn.qkv_proj and mlp.gate_up_proj per layer) and exposes the individual q_proj/k_proj/v_proj and gate_proj/up_proj tensors that LlamaModelBuilder expects — slicing the fused matrix’s output rows on read. Any tensor already present under its split name (a non-fused checkpoint) passes straight through.

Implements: INamedTensorSource

How It Works

Fused matrices are [out, in] row-major, so an output-row range maps to a contiguous element range: rows [r0, r1) → elements [r0*in, r1*in). QKV rows are stacked [Q(numHeads·headDim); K(numKVHeads·headDim); V(numKVHeads·headDim)]; gate/up are [gate(intermediate); up(intermediate)].

Constructors

Constructor Summary
FusedProjectionSource(INamedTensorSource inner, HuggingFaceConfig config) Wraps inner, deriving split-projection names from config.

Properties

Property Summary
IReadOnlyCollection<string> TensorNames { get; }

Methods

Method Summary
double[] ReadAsDouble(string name)