Qwen2MoEModelBuilder<T>
Helpers & Utilities · AiDotNet.ModelLoading.Pretrained
Reconstructs a Qwen2-MoE decoder from a Hugging Face HuggingFaceConfig and loads its
weights. Like Mixtral it routes each token through the top-k of many experts, but adds an always-on
shared expert gated by sigmoid(shared_expert_gate(x)), uses moe_intermediate_size for
the routed experts and shared_expert_intermediate_size for the shared one, and (Qwen2) biases the
q/k/v attention projections.
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> SupportedArchitectures { get; } |
The architecture family names this builder recognizes. |
Methods
| Method | Summary |
|---|---|
NeuralNetwork<T> Build(HuggingFaceConfig config, INamedTensorSource weights, int warmupSequenceLength) |
Builds and weight-loads a Qwen2-MoE decoder. |