APNet2<T>

Models & Types · AiDotNet.TextToSpeech.Vocoders

APNet2: a high-quality, high-efficiency vocoder that predicts the amplitude and phase spectra directly with ConvNeXt v2 networks and reconstructs the waveform by an inverse STFT, trained with linear anti-wrapping phase losses and a hinge GAN loss against a multi-period and a multi-resolution discriminator.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VocoderBase<T> → GanVocoderBase<T> → AmplitudePhaseVocoderBase<T> → APNet2<T>

For Beginners

Like APNet, APNet2 predicts how loud each frequency is and where each wave is in its cycle for every frame and turns that into audio with an inverse Fourier transform; a stronger backbone lets it work at a higher sampling rate and a longer frame shift.

How It Works

References: “APNet2: High-quality and High-efficiency Neural Vocoder with Direct Prediction of Amplitude and Phase Spectra” (Du et al., NCMMSC 2023) and redmist328/APNet2 for what the paper leaves unstated.

APNet’s structure and losses (§3) with ConvNeXt v2 predictors (§3.1), the anti-wrapping function f_AW(x) = |x − 2π round(x / 2π)| (Eq. 4) in the phase losses, and L_W’s hinge GAN terms averaged over the sub-discriminators of the MPD and the MRD (Eq. 2–3).

Constructors

Constructor Summary
APNet2(NeuralNetworkArchitecture<T> architecture) Calls the fully specified overload with the omitted parameters’ defaults.
APNet2(NeuralNetworkArchitecture<T> architecture, APNet2Options options) Creates an APNet2 that runs an exported ONNX graph.
APNet2(NeuralNetworkArchitecture<T> architecture, APNet2Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a trainable APNet2.
APNet2(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates an APNet2 that runs an exported ONNX graph.
APNet2(NeuralNetworkArchitecture<T> architecture, string modelPath) Creates an APNet2 that runs an exported ONNX graph.
APNet2(NeuralNetworkArchitecture<T> architecture, string modelPath, APNet2Options options) Creates a trainable APNet2.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()