PlanCache
Models & Types · AiDotNet.NeuralNetworks
Disk-backed store for compiled inference plans. Persists the traced plan after the
first compilation so subsequent process starts load the pre-compiled plan instead
of re-tracing + re-compiling. Directly wraps Tensors’ CompiledPlanLoader
and SaveAsync.
How It Works
PyTorch-parity equivalent: torch.jit.save(traced_module, path) +
torch.jit.load(path). The facade integration is opt-in via
AiModelBuilder.ConfigurePlanCaching(directory); once configured, save/load
is transparent to the caller.
Plans are keyed by (modelTypeName, T, structureVersion, inputShapeHash,
hardwareFingerprint). Plans compiled on one host cannot be loaded on a host with
a different PlanCompatibilityInfo — Tensors rejects the load and we
fall through to a fresh compile.
Constructors
| Constructor | Summary |
|---|---|
PlanCache(string directory) |
Properties
| Property | Summary |
|---|---|
PlanCache Current { get; } |
The currently-active plan cache, or null if caching is disabled. |
string Directory { get; } |
Methods
| Method | Summary |
|---|---|
Task SaveInferenceAsync(ICompiledPlan<T> plan, string modelTypeName, int structureVersion, int[] inputShape, CancellationToken cancellationToken) |
Persists a compiled plan to disk via atomic write (tmp-file + rename) so a crash partway through doesn’t leave a corrupt cache entry. |
Task<ICompiledPlan<T>> TryLoadInferenceAsync(string modelTypeName, int structureVersion, int[] inputShape, IEngine engine, CancellationToken cancellationToken) |
Attempts to load a pre-compiled inference plan from disk. |
string GetPlanPath(string modelTypeName, Type elementType, int structureVersion, int[] inputShape) |
Computes a stable filename for a plan identified by model type, element type, structure version, and input shape. |
void SetCurrent(PlanCache cache) |