InstructionTunedArchitectureType
Enums · AiDotNet.VisionLanguage.Encoders
Specifies the architecture type for instruction-tuned vision-language models.
Fields
| Field | Summary |
|---|---|
InstructionTunedArchitectureType CrossAttentionResampler |
Cross-attention resampler: vision encoder -> cross-attention resampler -> LLM (Qwen-VL series). |
InstructionTunedArchitectureType DirectPatch |
Direct patch embedding: raw image patches go directly into the language model without a separate vision encoder (Fuyu). |
InstructionTunedArchitectureType MLPProjection |
MLP projection: vision encoder -> MLP connector -> LLM (LLaVA, InternVL, DeepSeek-VL, Phi-3-Vision). |
InstructionTunedArchitectureType QFormerProjection |
Q-Former projection: vision encoder -> Q-Former -> linear projection -> LLM (MiniGPT-4, MiniGPT-v2). |
InstructionTunedArchitectureType VisualAbstractor |
Visual abstractor: vision encoder -> learnable visual abstractor module -> LLM (mPLUG-Owl series). |
InstructionTunedArchitectureType VisualExpert |
Visual expert: vision encoder -> visual expert modules interleaved in every LLM layer (CogVLM). |
int value__ |