MixedPrecisionScope
Models & Types · AiDotNet.MixedPrecision
Provides an ambient context for mixed-precision operations during forward and backward passes.
Implements: IDisposable
For Beginners
MixedPrecisionScope is like a “mode switch” that tells the neural network to use lower precision (FP16) for faster computation while keeping certain operations in full precision (FP32) for numerical stability.
The scope is automatically managed by the training loop - you don’t need to create it yourself. When inside a scope:
- Most operations use FP16 (faster, less memory)
- Certain layers (like LayerNorm, Softmax) stay in FP32 for stability
- The scope tracks both FP16 and FP32 versions of tensors so layers can access what they need
How It Works
Technical Details: The scope uses a thread-static pattern to provide ambient context.
Layers can check Current to determine if mixed-precision is active and whether
they should use FP16 or FP32 based on the LayerPrecisionPolicy.
Example
// Mixed precision is switched on through the builder, which opens and closes the
// scopes for you; layers check MixedPrecisionScope.Current to pick their precision.
var architecture = new NeuralNetworkArchitecture<float>(
InputType.OneDimensional, NeuralNetworkTaskType.Regression, inputSize: 4, outputSize: 2);
var trainX = Tensor<float>.CreateRandom(8, 4);
var trainY = Tensor<float>.CreateRandom(8, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new NeuralNetwork<float>(architecture))
.ConfigureMixedPrecision(AiDotNet.MixedPrecision.MixedPrecisionConfig.Conservative())
.Build(trainX, trainY);
var output = result.Predict(Tensor<float>.CreateRandom(1, 4));
Constructors
| Constructor | Summary |
|---|---|
MixedPrecisionScope(MixedPrecisionContext context, LayerPrecisionPolicy policy) |
Creates a new mixed-precision scope. |
Properties
| Property | Summary |
|---|---|
LayerPrecisionPolicy Policy { get; } |
Gets the layer precision policy for this scope. |
MixedPrecisionContext Context { get; } |
Gets the mixed-precision context associated with this scope. |
MixedPrecisionScope Current { get; } |
Gets the currently active mixed-precision scope, or null if not in a scope. |
bool IsActive { get; } |
Gets whether this scope is currently active (is the current scope). |
int RegisteredTensorCount { get; } |
Gets the number of registered tensors. |
Methods
| Method | Summary |
|---|---|
MixedPrecisionType GetLayerPrecision(string layerName) |
Gets the precision type to use for a specific layer. |
Tensor<Half> CastToFP16(Tensor<float> fp32Tensor) |
Converts an FP32 tensor to FP16. |
Tensor<Half> GetFP16Tensor(string name) |
Retrieves the FP16 version of a previously registered tensor. |
Tensor<Half> RegisterAndCastToFP16(string name, Tensor<float> fp32Tensor) |
Registers an FP32 tensor and returns its FP16 equivalent. |
Tensor<float> CastToFP32(Tensor<Half> fp16Tensor) |
Converts an FP16 tensor back to FP32. |
Tensor<float> GetFP32Tensor(string name) |
Retrieves the FP32 version of a previously registered tensor. |
bool HasTensor(string name) |
Checks if a tensor with the given name has been registered. |
bool ShouldUseFP32(string layerName) |
Determines if a layer should use full precision (FP32) based on the policy. |
bool ShouldUseHigherPrecision(string layerName) |
Determines if a layer should use higher precision than the default (FP32 or FP16 when default is FP8). |
string ToString() |
Gets a string representation of the scope’s current state. |
void ClearTensors() |
Clears all registered tensors to free memory. |
void Dispose() |
Disposes the scope and restores the previous scope (if any). |