ParallelStreamsLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Splits input along the feature axis into two equal halves, processes each half through its own independent sub-network (stream), and concatenates the two stream outputs.

Inheritance: LayerBase<T> → ParallelStreamsLayer<T>

Implements: IShapeContract

For Beginners

Many real-world neural networks need to process two different types of information at the same time. For example:

  • An audio-visual model processes audio features and video features separately
  • A multi-modal model processes text embeddings and image embeddings in parallel
  • A siamese network processes two inputs through shared or separate encoders

This layer solves the problem of expressing parallel processing within a sequential layer stack. Instead of needing complex custom forward logic, you can simply place this layer in your network and it will:

  1. Split the input features into two equal halves (e.g., [256 audio | 256 visual])
  2. Process each half through its own set of layers (Stream A and Stream B)
  3. Concatenate the two outputs back together

All operations are tracked on the gradient tape so backpropagation works correctly through both streams simultaneously.

Example: If you have 512 input features representing concatenated audio+visual features:

How It Works

How it works internally:

  • The input tensor is sliced along the last axis into two equal halves using

Engine.TensorSlice (tape-tracked)

  • Each half is passed through its respective stream’s layers sequentially
  • The two stream outputs are concatenated along the last axis using

Engine.TensorConcatenate (tape-tracked)

Gradient flow: Because the split and concatenation use Engine operations that record on the autodiff tape, gradients flow correctly through both streams during backpropagation. Each stream receives only the gradients relevant to its portion of the input, ensuring proper learning for both sub-networks.

Constructors

Constructor Summary
ParallelStreamsLayer(int inputSize, int streamAOutputSize, int streamBOutputSize, IEnumerable<ILayer<T>> streamALayers, IEnumerable<ILayer<T>> streamBLayers)

Properties

Property Summary
bool SupportsTraining { get; } Gets whether this layer supports training through backpropagation.

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
void ResetState() Resets the internal state of all layers in both streams.
void SetTrainingMode(bool isTraining) Sets training or inference mode for all layers in both streams.
void UpdateParameters(T learningRate) Updates parameters in both streams using the given learning rate.