LSTMLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Represents a Long Short-Term Memory (LSTM) layer for processing sequential data.

Inheritance: LayerBase<T> → LSTMLayer<T>

Implements: IShapeContract

For Beginners

An LSTM layer is like a smart memory system for your AI.

Think of it like a notepad with special features:

  • It can remember important information for a long time (unlike simpler neural networks)
  • It can forget irrelevant details (using its “forget gate”)
  • It can decide what new information to write down (using its “input gate”)
  • It can decide what information to share (using its “output gate”)

LSTMs are great for:

  • Text generation and language understanding
  • Time series prediction (like stock prices)
  • Speech recognition
  • Any task where the order and context of information matters

For example, when processing the sentence “The clouds are in the ___”, an LSTM would remember that “clouds” appeared earlier, helping it predict “sky” as the missing word.

How It Works

The LSTM layer is a specialized type of recurrent neural network (RNN) that is designed to capture long-term dependencies in sequential data. It uses a cell state and a series of gates (forget, input, and output) to control the flow of information through the network, allowing it to remember important patterns over long sequences while forgetting irrelevant information.

Constructors

Constructor Summary
LSTMLayer(int hiddenSize, IActivationFunction<T> activation, IActivationFunction<T> recurrentActivation) Lazy ctor with vector activation functions.
LSTMLayer(int hiddenSize, IVectorActivationFunction<T> vectorActivation, IVectorActivationFunction<T> recurrentActivation) Lazy ctor with vector activation functions.

Properties

Property Summary
Dictionary<string, Tensor<T>> Gradients { get; set; } Gets a dictionary containing the gradients for all trainable parameters after a backward pass.
Tensor<T> BiasC { get; } Gets the cell gate bias for weight loading.
Tensor<T> BiasF { get; } Gets the forget gate bias for weight loading.
Tensor<T> BiasI { get; } Gets the input gate bias for weight loading.
Tensor<T> BiasO { get; } Gets the output gate bias for weight loading.
Tensor<T> WeightsCh { get; } Gets the cell gate hidden weights for weight loading.
Tensor<T> WeightsCi { get; } Gets the cell gate input weights for weight loading.
Tensor<T> WeightsFh { get; } Gets the forget gate hidden weights for weight loading.
Tensor<T> WeightsFi { get; } Gets the forget gate input weights for weight loading.
Tensor<T> WeightsIh { get; } Gets the input gate hidden weights for weight loading.
Tensor<T> WeightsIi { get; } Gets the input gate input weights for weight loading.
Tensor<T> WeightsOh { get; } Gets the output gate hidden weights for weight loading.
Tensor<T> WeightsOi { get; } Gets the output gate input weights for weight loading.
bool SupportsTraining { get; } Gets a value indicating whether this layer supports training through backpropagation.

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
void ClearGradients()
void ResetState() Resets the internal state of the LSTM layer.
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate) Updates the parameters of the LSTM layer based on the calculated gradients.
void UpdateParametersGpu(IGpuOptimizerConfig config) GPU-resident parameter update with polymorphic optimizer support.
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).