SpeculationPolicy
Enums · AiDotNet.Configuration
Policies for enabling/disabling speculative decoding at runtime.
Fields
| Field | Summary |
|---|---|
SpeculationPolicy Auto |
Automatically decide based on runtime conditions (recommended). |
SpeculationPolicy ForceOff |
Always disable speculative decoding even if enabled in config. |
SpeculationPolicy ForceOn |
Always enable speculative decoding when configured. |
SpeculationPolicy LatencyFirst |
Prefer speculative decoding to reduce latency, even under moderate load. |
SpeculationPolicy ThroughputFirst |
Prefer throughput and stability: use speculative decoding only when conditions are ideal. |
int value__ |