LSTMCellLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

One step of a long short-term memory cell, (h', c') = LSTM(x, h, c), for recurrent decoders that run a step at a time.

Inheritance: LayerBase<T> → LSTMCellLayer<T>

Implements: IShapeContract

For Beginners

An LSTM keeps a separate long-term memory that it adds to and erases from through gates, which lets it remember things over many steps.

How It Works

Hochreiter & Schmidhuber (1997) with forget gates (Gers et al. 2000): i, f, o = σ(W x + U h + b), g = tanh(W_g x + U_g h + b_g), c' = f ⊙ c + i ⊙ g, h' = o ⊙ tanh(c'). An optional cap clips c' to ±cap (Non-Attentive Tacotron, Table 6: “LSTM cell abs value cap 10.0”).

Inputs: x``[batch, input] and the state [batch, 2 · hidden] holding h then c; the output is the next state in the same layout (use SplitState).

Constructors

Constructor Summary
LSTMCellLayer(int inputSize, int hiddenSize, double cellClip) Creates an LSTM cell.

Properties

Property Summary
bool RequiresMultipleInputs { get; }
bool SupportsTraining { get; }
int HiddenSize { get; } Width of the hidden and cell states.
int InputSize { get; } Width of the step input.

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
Tensor<T> Forward(Tensor<T> input, Tensor<T> state) One step: the next state [h'; c'] from the step input and the state [h; c].
ValueTuple<Tensor<T>, Tensor<T>> SplitState(Tensor<T> state) Splits a step output [batch, 2 · hidden] into the hidden and cell states.
void ResetState()