WhisperConvStemLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Whisper’s audio-encoder stem (Radford et al. 2022, §2.2): two width-3 convolutions over time, each followed by GELU, the second with stride 2. It maps [frames, mels] log-mel features to [frames / 2, width] encoder inputs.

Inheritance: LayerBase<T> → WhisperConvStemLayer<T>

Implements: IShapeContract

For Beginners

Before attention looks at the audio, two small filters slide along time to pick out local patterns and halve the number of steps, which makes the attention four times cheaper.

How It Works

Matches the reference AudioEncoder: Conv1d(n_mels, width, 3, padding=1) then Conv1d(width, width, 3, stride=2, padding=1). A 30-second clip’s 3,000 frames become the encoder’s 1,500 positions.

Constructors

Constructor Summary
WhisperConvStemLayer(int numMels, int width) Creates the stem.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
int OutputFrames(int frames) The number of encoder positions a clip of frames frames gives.
void ResetState()
void UpdateParameters(T learningRate)