EchoStateNetwork<T>

Models & Types · AiDotNet.NeuralNetworks

Represents an Echo State Network (ESN), a type of recurrent neural network with a sparsely connected hidden layer called a reservoir.

Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → SequenceModelLayoutBase<T> → EchoStateNetwork<T>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

An Echo State Network is like a pool of water that creates ripples from your input.

Think of it this way:

  • You drop a stone into a pool of water (your input)
  • The stone creates ripples that bounce around and interact in complex ways (the reservoir)
  • Someone watches the pattern of ripples and learns to predict what comes next (the output layer)
  • Only the person watching and predicting is trained - the water itself doesn’t change how it ripples

This approach is particularly good for processing sequences, like speech or time series data, because the ripples in the reservoir naturally capture patterns over time without needing complex training procedures.

How It Works

An Echo State Network is a unique type of recurrent neural network where the connections between neurons in the hidden layer (called the reservoir) are randomly generated and remain fixed during training. Only the output connections from the reservoir to the output layer are trained. The reservoir acts as a dynamic memory that transforms inputs into high-dimensional representations, enabling the network to process temporal patterns effectively. The key characteristic of ESNs is the “echo state property” which ensures that the effect of initial conditions gradually fades away.

Example

var input = Tensor<float>.CreateRandom(new[] { 1, 100, 1 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new EchoStateNetwork<float>())
    .Build(trainX, trainY);
var output = result.Predict(input);

Constructors

Constructor Summary
EchoStateNetwork() Initializes a new instance of the EchoStateNetwork class with vector activation functions.
EchoStateNetwork(NeuralNetworkArchitecture<T> architecture, int reservoirSize, ILossFunction<T> lossFunction, IActivationFunction<T> reservoirInputScalarActivation, IActivationFunction<T> reservoirOutputScalarActivation, IActivationFunction<T> reservoirScalarActivation, IActivationFunction<T> outputScalarActivation, EchoStateNetworkOptions options) Initializes a new instance of the EchoStateNetwork class with scalar activation functions.
EchoStateNetwork(NeuralNetworkArchitecture<T> architecture, int reservoirSize, ILossFunction<T> lossFunction, IVectorActivationFunction<T> reservoirInputVectorActivation, IVectorActivationFunction<T> reservoirOutputVectorActivation, IVectorActivationFunction<T> reservoirVectorActivation, IVectorActivationFunction<T> outputVectorActivation, EchoStateNetworkOptions options) Initializes a new instance of the EchoStateNetwork class with scalar activation functions.

Properties

Property Summary
bool SupportsTraining { get; } Makes a prediction using the Echo State Network.

Methods

Method Summary
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) Reports this network’s two real computation stages: the settled reservoir state and the linear readout.
List<Tensor<T>> PredictSequence(List<Tensor<T>> inputSequence, bool resetState) Processes a sequence of inputs through the Echo State Network.
ModelMetadata<T> GetModelMetadata() Gets metadata about the Echo State Network model.
Vector<T> GetParameterGradients() Trains the Echo State Network on a single batch of data.
void FinalizeTraining() Finalizes training by computing the optimal output weights.
void ResetReservoirState() Resets the reservoir state to zeros.
void SetLeakingRate(double leakingRate) Sets the leaking rate for the reservoir.
void SetRegularization(double regularization) Sets the regularization parameter for ridge regression.
void SetWarmupPeriod(int warmupPeriod) Sets the warmup period for discarding initial transient reservoir states.