DualStreamArchitecture<T>

Models & Types · AiDotNet.NeuralNetworks

A neural network architecture for vision + text two-stream models (CLIP-family encoders: BASIC, DFNCLIP, EVACLIP, LiT, RegionCLIP, etc.) that hosts a separate vision encoder and text encoder.

Inheritance: NeuralNetworkArchitecture<T> → DualEncoderArchitecture<T> → DualStreamArchitecture<T>

For Beginners

Some vision-language models have two separate “brains” — one that looks at images and one that reads text — and they only meet at the very end when comparing embeddings. If you want to swap in a custom layer stack for both halves, this architecture lets you describe both halves at once, instead of cramming them into a single flat list (which silently drops the text half).

How It Works

Concrete vision-language specialisation of DualEncoderArchitecture: the abstract base holds the two encoder stacks under modality-neutral names (EncoderALayers / EncoderBLayers); this subclass exposes them under their semantic vision-language aliases VisionLayers / TextLayers so call sites read naturally.

Constructors

Constructor Summary
DualStreamArchitecture(IEnumerable<ILayer<T>> visionLayers, IEnumerable<ILayer<T>> textLayers, InputType inputType, NeuralNetworkTaskType taskType, NetworkComplexity complexity, int inputSize, int inputHeight, int inputWidth, int inputDepth, int outputSize, bool shouldReturnFullSequence, int imageEmbeddingDim, int textEmbeddingDim, int inputFrames) Initializes a new dual-stream architecture with explicit vision and text encoder layers.

Properties

Property Summary
IReadOnlyList<ILayer<T>> TextLayers { get; } Gets the layer stack for the text encoder.
IReadOnlyList<ILayer<T>> VisionLayers { get; } Gets the layer stack for the vision encoder.