PlanCache

Models & Types · AiDotNet.NeuralNetworks

Disk-backed store for compiled inference plans. Persists the traced plan after the first compilation so subsequent process starts load the pre-compiled plan instead of re-tracing + re-compiling. Directly wraps Tensors’ CompiledPlanLoader and SaveAsync.

How It Works

PyTorch-parity equivalent: torch.jit.save(traced_module, path) + torch.jit.load(path). The facade integration is opt-in via AiModelBuilder.ConfigurePlanCaching(directory); once configured, save/load is transparent to the caller.

Plans are keyed by (modelTypeName, T, structureVersion, inputShapeHash, hardwareFingerprint). Plans compiled on one host cannot be loaded on a host with a different PlanCompatibilityInfo — Tensors rejects the load and we fall through to a fresh compile.

Constructors

Constructor Summary
PlanCache(string directory)

Properties

Property Summary
PlanCache Current { get; } The currently-active plan cache, or null if caching is disabled.
string Directory { get; }

Methods

Method Summary
Task SaveInferenceAsync(ICompiledPlan<T> plan, string modelTypeName, int structureVersion, int[] inputShape, CancellationToken cancellationToken) Persists a compiled plan to disk via atomic write (tmp-file + rename) so a crash partway through doesn’t leave a corrupt cache entry.
Task<ICompiledPlan<T>> TryLoadInferenceAsync(string modelTypeName, int structureVersion, int[] inputShape, IEngine engine, CancellationToken cancellationToken) Attempts to load a pre-compiled inference plan from disk.
string GetPlanPath(string modelTypeName, Type elementType, int structureVersion, int[] inputShape) Computes a stable filename for a plan identified by model type, element type, structure version, and input shape.
void SetCurrent(PlanCache cache)