CbhgLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Tacotron’s CBHG module: a 1-D Convolution Bank, Highway network and bidirectional GRU.

Inheritance: LayerBase<T> → CbhgLayer<T>

Implements: IShapeContract

For Beginners

The convolution bank looks at the sequence through windows of many sizes at once, like reading a sentence by its letters, syllables and words together; the GRUs then read the result in both directions.

How It Works

Tacotron (Wang et al. 2017, §3.1, Fig. 2, Table 1): the input is convolved with K sets of 1-D filters of widths 1..K, the outputs are stacked and max-pooled along time (stride 1, width 2), passed through two fixed-width 1-D projection convolutions, added to the input (residual), fed to a 4-layer highway network, and read by a bidirectional GRU whose forward and backward outputs are concatenated. Batch normalization is used for every convolution.

What the paper leaves unstated follows the reference implementation (keithito/tacotron, models/modules.py): convolutions use “same” padding (for an even width, ⌊(k−1)/2⌋ frames in front and the rest behind), apply their activation and then batch normalization; the max pooling pads at the end; when the residual sum’s width differs from the highway width (the post-processing CBHG reads 80 mel bins into 128-wide highways) a linear layer maps it first; highway gates start biased to −1.

Input [time, channels] or [batch, time, channels]; output [.., time, 2 · gruUnits].

Constructors

Constructor Summary
CbhgLayer(int inputChannels, int bankSize, int bankChannels, int[] projections, int highwayWidth, int gruUnits, CbhgVariant variant, double dropoutRate) Creates a CBHG module.

Properties

Property Summary
bool SupportsTraining { get; }
int OutputChannels { get; } Width of the output (both GRU directions).

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
void ResetState()