RoomImpulseResponse<T>
Models & Types · AiDotNet.Audio.Effects
FiNS — Filtered Noise Shaping network that estimates a time-domain room impulse response directly from reverberant speech.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → RoomImpulseResponse<T>
Implements: IAudioEnhancer<T>, IGeneratedParameterRegistrar<T>
For Beginners
Clap in a room and you hear a sharp crack followed by a smeared tail of echoes. Those two halves are different problems. The crack has real structure, so the model predicts it sample by sample. The tail is essentially noise fading out — and fading at a different rate in each frequency band, because rooms absorb treble faster than bass. So instead of predicting 48,000 tail samples, the model takes noise, splits it into 10 frequency bands, and only learns how loud each band should be over time. That is the “filtered noise shaping” idea, and it is why this network can produce a full one-second response from 128 numbers.
How It Works
A strided 1-D convolutional encoder compresses the reverberant waveform to a single latent
embedding z, and a filtered noise shaping decoder expands z into the impulse response as two
parts. The early component (first E samples) is predicted directly on an extra decoder output
channel. The late field is synthesised by passing a noise signal through a trainable bank of M
FIR band-pass filters and shaping each band with a predicted time-domain mask,
h_l,m(n) = sigmoid(y_m(n)) * s_m(n). A 1x1 convolution mixes the M late bands and the
early component into the final monophonic RIR.
The forward pass is written out explicitly rather than run as a flat layer walk: the encoder has a parallel residual branch per block, the decoder is FiLM-conditioned on a latent obtained by pooling, and the filterbank is driven by noise rather than by the previous layer’s output. None of that is expressible as a sequential chain.
Usage:
Constructors
| Constructor | Summary |
|---|---|
RoomImpulseResponse(NeuralNetworkArchitecture<T> architecture, RoomImpulseResponseOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates an RIR estimation model in native training mode. |
RoomImpulseResponse(NeuralNetworkArchitecture<T> architecture, string modelPath, RoomImpulseResponseOptions options) |
Creates an RIR estimation model in native training mode. |
Properties
| Property | Summary |
|---|---|
double EnhancementStrength { get; set; } |
|
int LatencySamples { get; } |
|
int NumChannels { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, Tensor<T>> GetNamedLayerActivations(Tensor<T> input) |
Exposes the FiNS stage activations, which a flat layer walk cannot produce for this graph. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Enhance(Tensor<T> audio) |
|
Tensor<T> EnhanceWithReference(Tensor<T> audio, Tensor<T> reference) |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
Routes TRAINING through the same explicit FiNS graph the inference path uses. |
Tensor<T> ProcessChunk(Tensor<T> audioChunk) |
|
void EstimateNoiseProfile(Tensor<T> noiseOnlyAudio) |
|
void ResetState() |