Qwen2MoEModelBuilder<T>

Helpers & Utilities · AiDotNet.ModelLoading.Pretrained

Reconstructs a Qwen2-MoE decoder from a Hugging Face HuggingFaceConfig and loads its weights. Like Mixtral it routes each token through the top-k of many experts, but adds an always-on shared expert gated by sigmoid(shared_expert_gate(x)), uses moe_intermediate_size for the routed experts and shared_expert_intermediate_size for the shared one, and (Qwen2) biases the q/k/v attention projections.

Properties

Property Summary
IReadOnlyList<string> SupportedArchitectures { get; } The architecture family names this builder recognizes.

Methods

Method Summary
NeuralNetwork<T> Build(HuggingFaceConfig config, INamedTensorSource weights, int warmupSequenceLength) Builds and weight-loads a Qwen2-MoE decoder.