SpeculationPolicy

Enums · AiDotNet.Configuration

Policies for enabling/disabling speculative decoding at runtime.

Fields

Field Summary
SpeculationPolicy Auto Automatically decide based on runtime conditions (recommended).
SpeculationPolicy ForceOff Always disable speculative decoding even if enabled in config.
SpeculationPolicy ForceOn Always enable speculative decoding when configured.
SpeculationPolicy LatencyFirst Prefer speculative decoding to reduce latency, even under moderate load.
SpeculationPolicy ThroughputFirst Prefer throughput and stability: use speculative decoding only when conditions are ideal.
int value__