WaveNet<T>

Models & Types · AiDotNet.Finance.Forecasting.Neural

WaveNet model adapted for time series forecasting.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → ForecastingModelBase<T> → WaveNet<T>

For Beginners

WaveNet is like a more sophisticated version of TCN:

Gated Activation Units: Instead of using simple ReLU activations, WaveNet multiplies two signals:

  • output = tanh(filter_output) * sigmoid(gate_output)
  • The tanh extracts features, the sigmoid controls which features pass through
  • This is inspired by LSTM gates and helps model complex patterns

Two Types of Connections:

  1. Residual Connection: Each block adds its input to output (like TCN)
  • Helps gradients flow during training
  1. Skip Connection: Each block sends output directly to the final layers
  • Allows combining features from ALL time scales
  • Skip outputs are summed at the end

Why Two Connection Types? The residual path maintains the signal through deep networks. The skip path lets early layers contribute directly to the output, ensuring fine-grained (high-frequency) and coarse (low-frequency) patterns are both captured in the final prediction.

Example: With 2 stacks of 8 layers each:

  • 16 total dilated layers
  • Dilations: [1,2,4,8,16,32,64,128, 1,2,4,8,16,32,64,128]
  • Receptive field: 2 * (2^8 - 1) + 1 = 511 time steps

How It Works

WaveNet was originally developed by DeepMind for audio generation. It uses dilated causal convolutions with gated activation units and dual residual/skip connections. This architecture has proven highly effective for time series forecasting as well.

Reference: van den Oord et al., “WaveNet: A Generative Model for Raw Audio”, 2016. https://arxiv.org/abs/1609.03499

Example

var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 256, inputWidth: 1, inputDepth: 1, outputSize: 24);
var model = new WaveNet<double>(architecture);
var onnxModel = new WaveNet<double>(architecture, "wavenet.onnx");

Constructors

Constructor Summary
WaveNet(NeuralNetworkArchitecture<T> architecture, WaveNetOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a WaveNet in native mode for training from scratch.
WaveNet(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, WaveNetOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Creates a WaveNet using pretrained ONNX model.

Properties

Property Summary
bool IsChannelIndependent { get; }
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int NumFeatures { get; }
int PatchSize { get; }
int PredictionHorizon { get; }
int SequenceLength { get; }
int Stride { get; }

Methods

Method Summary
Dictionary<string, T> Evaluate(Tensor<T> predictions, Tensor<T> actuals)
Dictionary<string, T> GetFinancialMetrics()
ModelMetadata<T> GetModelMetadata()
Tensor<T> ApplyInstanceNormalization(Tensor<T> input)
Tensor<T> AutoregressiveForecast(Tensor<T> input, int steps)