EfficientQATOptimizer<T>

Models & Types · AiDotNet.Deployment.Optimization.Quantization.Training

EfficientQAT optimizer providing memory-efficient Quantization-Aware Training for large models. Uses block-wise quantization and efficient gradient computation to reduce memory footprint.

For Beginners

Standard QAT uses a lot of memory because it keeps full-precision copies of all weights. EfficientQAT is smarter about memory, letting you train bigger models on the same hardware.

How It Works

Key Innovations:

Memory Savings: 2-4x less memory than standard QAT

Reference: “Efficient Quantization-Aware Training for Large Language Models” (ACL 2025)

Constructors

Constructor Summary
EfficientQATOptimizer(QuantizationConfiguration config, int totalEpochs) Initializes a new instance of the EfficientQATOptimizer.

Properties

Property Summary
QATTrainingHook<T> QATHook { get; } Gets the QAT training hook for applying fake quantization.
double CurrentBitWidth { get; } Gets the current effective bit width (may change with progressive quantization).

Methods

Method Summary
Vector<T> ApplyBlockWiseQuantization(Vector<T> weights, string layerName) Applies block-wise fake quantization to weights with memory efficiency.
Vector<T> ComputeQuantizationAwareGradient(Vector<T> gradOutput, Vector<T> weights, string layerName) Computes gradients with block-wise quantization awareness.
long EstimateMemoryUsage(long parameterCount) Gets memory usage estimate for current configuration.
void OnEpochStart(int epoch) Called at the start of each epoch to manage progressive quantization.
void UpdateBlockScales(Vector<T> weights, string layerName, double learningRate) Updates block scales based on observed weight distributions.