WhisperConvStemLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Whisper’s audio-encoder stem (Radford et al. 2022, §2.2): two width-3 convolutions over time, each followed by GELU, the second with stride 2. It maps [frames, mels] log-mel features to [frames / 2, width] encoder inputs.
Inheritance: LayerBase<T> → WhisperConvStemLayer<T>
Implements: IShapeContract
For Beginners
Before attention looks at the audio, two small filters slide along time to pick out local patterns and halve the number of steps, which makes the attention four times cheaper.
How It Works
Matches the reference AudioEncoder: Conv1d(n_mels, width, 3, padding=1) then
Conv1d(width, width, 3, stride=2, padding=1). A 30-second clip’s 3,000 frames become the encoder’s 1,500
positions.
Constructors
| Constructor | Summary |
|---|---|
WhisperConvStemLayer(int numMels, int width) |
Creates the stem. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
int OutputFrames(int frames) |
The number of encoder positions a clip of frames frames gives. |
void ResetState() |
|
void UpdateParameters(T learningRate) |