FeedForwardTransformerBlock<T>

Layers · AiDotNet.NeuralNetworks.Layers

The feed-forward Transformer (FFT) block of FastSpeech: multi-head self-attention followed by a two-layer 1D convolutional network, each with a residual connection, dropout and post-layer normalization.

Inheritance: LayerBase<T> → FeedForwardTransformerBlock<T>

Implements: IShapeContract

For Beginners

Attention lets every phoneme (or mel frame) look at every other one; the small convolution then mixes each position with its immediate neighbours, which suits speech, where neighbouring sounds blend into each other.

How It Works

FastSpeech (Ren et al. 2019, §3.1) replaces the Transformer’s position-wise dense FFN with a 2-layer 1D convolution and ReLU, “motivated by the observation that the adjacent hidden states are more closely related in the character/phoneme and mel-spectrogram sequence”; residual connections, layer normalization and dropout follow the self-attention and the convolution respectively. FastSpeech 2 (Ren et al. 2021, App. A) uses the same block with hidden 256, 2 heads and kernel sizes 9 and 1 (256→1024→256):

Input and output are [batch, time, hidden]; the convolutions run on the transposed [batch, hidden, time] view with “same” padding, so the time axis keeps its length.

With a non-zero conditionSize both normalizations are ConditionalLayerNormalizationLayer, whose scale and bias come from a conditioning vector passed as a second input — AdaSpeech’s mel decoder (Chen et al. 2021, §2.2), which conditions every decoder layer normalization on the speaker embedding.

With prependCondition the second input is instead one hidden-wide vector placed before the sequence as an extra position, attended to by every position and dropped from the output — PromptTTS (Guo et al. 2023, §2.3–2.4), which prepends its style representation to the input of every Transformer block.

Constructors

Constructor Summary
FeedForwardTransformerBlock(int hiddenSize, int numHeads, int filterSize, int firstKernelSize, int secondKernelSize, double dropoutRate, int conditionSize, bool prependCondition) Creates an FFT block.

Properties

Property Summary
bool PrependsCondition { get; } Whether the second input is prepended to the sequence as one more position.
bool RequiresMultipleInputs { get; }
bool SupportsTraining { get; }
double DropoutRate { get; } Dropout probability.
int ConditionSize { get; } Width of the conditioning vector; 0 when the block uses ordinary layer normalization.
int FilterSize { get; } Output channels of the first convolution.
int FirstKernelSize { get; } Kernel of the first convolution.
int HiddenSize { get; } Model width.
int NumHeads { get; } Attention heads.
int SecondKernelSize { get; } Kernel of the second convolution.

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
Tensor<T> Forward(Tensor<T> input, Tensor<T> condition) Runs the block with its layer normalizations conditioned on condition.
void ResetState()