RoomImpulseResponse<T>

Models & Types · AiDotNet.Audio.Effects

FiNS — Filtered Noise Shaping network that estimates a time-domain room impulse response directly from reverberant speech.

Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → RoomImpulseResponse<T>

Implements: IAudioEnhancer<T>, IGeneratedParameterRegistrar<T>

For Beginners

Clap in a room and you hear a sharp crack followed by a smeared tail of echoes. Those two halves are different problems. The crack has real structure, so the model predicts it sample by sample. The tail is essentially noise fading out — and fading at a different rate in each frequency band, because rooms absorb treble faster than bass. So instead of predicting 48,000 tail samples, the model takes noise, splits it into 10 frequency bands, and only learns how loud each band should be over time. That is the “filtered noise shaping” idea, and it is why this network can produce a full one-second response from 128 numbers.

How It Works

A strided 1-D convolutional encoder compresses the reverberant waveform to a single latent embedding z, and a filtered noise shaping decoder expands z into the impulse response as two parts. The early component (first E samples) is predicted directly on an extra decoder output channel. The late field is synthesised by passing a noise signal through a trainable bank of M FIR band-pass filters and shaping each band with a predicted time-domain mask, h_l,m(n) = sigmoid(y_m(n)) * s_m(n). A 1x1 convolution mixes the M late bands and the early component into the final monophonic RIR.

The forward pass is written out explicitly rather than run as a flat layer walk: the encoder has a parallel residual branch per block, the decoder is FiLM-conditioned on a latent obtained by pooling, and the filterbank is driven by noise rather than by the previous layer’s output. None of that is expressible as a sequential chain.

Usage:

Constructors

Constructor Summary
RoomImpulseResponse(NeuralNetworkArchitecture<T> architecture, RoomImpulseResponseOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates an RIR estimation model in native training mode.
RoomImpulseResponse(NeuralNetworkArchitecture<T> architecture, string modelPath, RoomImpulseResponseOptions options) Creates an RIR estimation model in native training mode.

Properties

Property Summary
double EnhancementStrength { get; set; }
int LatencySamples { get; }
int NumChannels { get; }

Methods

Method Summary
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) Exposes the FiNS stage activations, which a flat layer walk cannot produce for this graph.
ModelMetadata<T> GetModelMetadata()
Tensor<T> Enhance(Tensor<T> audio)
Tensor<T> EnhanceWithReference(Tensor<T> audio, Tensor<T> reference)
Tensor<T> ForwardForTraining(Tensor<T> input) Routes TRAINING through the same explicit FiNS graph the inference path uses.
Tensor<T> ProcessChunk(Tensor<T> audioChunk)
void EstimateNoiseProfile(Tensor<T> noiseOnlyAudio)
void ResetState()