EfficientQATOptimizer<T>
Models & Types · AiDotNet.Deployment.Optimization.Quantization.Training
EfficientQAT optimizer providing memory-efficient Quantization-Aware Training for large models. Uses block-wise quantization and efficient gradient computation to reduce memory footprint.
For Beginners
Standard QAT uses a lot of memory because it keeps full-precision copies of all weights. EfficientQAT is smarter about memory, letting you train bigger models on the same hardware.
How It Works
Key Innovations:
Memory Savings: 2-4x less memory than standard QAT
Reference: “Efficient Quantization-Aware Training for Large Language Models” (ACL 2025)
Constructors
| Constructor | Summary |
|---|---|
EfficientQATOptimizer(QuantizationConfiguration config, int totalEpochs) |
Initializes a new instance of the EfficientQATOptimizer. |
Properties
| Property | Summary |
|---|---|
QATTrainingHook<T> QATHook { get; } |
Gets the QAT training hook for applying fake quantization. |
double CurrentBitWidth { get; } |
Gets the current effective bit width (may change with progressive quantization). |
Methods
| Method | Summary |
|---|---|
Vector<T> ApplyBlockWiseQuantization(Vector<T> weights, string layerName) |
Applies block-wise fake quantization to weights with memory efficiency. |
Vector<T> ComputeQuantizationAwareGradient(Vector<T> gradOutput, Vector<T> weights, string layerName) |
Computes gradients with block-wise quantization awareness. |
long EstimateMemoryUsage(long parameterCount) |
Gets memory usage estimate for current configuration. |
void OnEpochStart(int epoch) |
Called at the start of each epoch to manage progressive quantization. |
void UpdateBlockScales(Vector<T> weights, string layerName, double learningRate) |
Updates block scales based on observed weight distributions. |