InstructionTunedArchitectureType

Enums · AiDotNet.VisionLanguage.Encoders

Specifies the architecture type for instruction-tuned vision-language models.

Fields

Field Summary
InstructionTunedArchitectureType CrossAttentionResampler Cross-attention resampler: vision encoder -> cross-attention resampler -> LLM (Qwen-VL series).
InstructionTunedArchitectureType DirectPatch Direct patch embedding: raw image patches go directly into the language model without a separate vision encoder (Fuyu).
InstructionTunedArchitectureType MLPProjection MLP projection: vision encoder -> MLP connector -> LLM (LLaVA, InternVL, DeepSeek-VL, Phi-3-Vision).
InstructionTunedArchitectureType QFormerProjection Q-Former projection: vision encoder -> Q-Former -> linear projection -> LLM (MiniGPT-4, MiniGPT-v2).
InstructionTunedArchitectureType VisualAbstractor Visual abstractor: vision encoder -> learnable visual abstractor module -> LLM (mPLUG-Owl series).
InstructionTunedArchitectureType VisualExpert Visual expert: vision encoder -> visual expert modules interleaved in every LLM layer (CogVLM).
int value__