PagedKVCacheStats
Models & Types · AiDotNet.Inference.PagedAttention
Statistics about the paged KV cache.
Constructors
| Constructor | Summary |
|---|---|
PagedKVCacheStats() |
Properties
| Property | Summary |
|---|---|
BlockManagerStats BlockStats { get; set; } |
Underlying block manager statistics. |
double AverageSequenceLength { get; set; } |
Average sequence length. |
double MemoryEfficiency { get; set; } |
Memory efficiency compared to traditional allocation. |
int ActiveSequences { get; set; } |
Number of active sequences. |
long TotalTokensCached { get; set; } |
Total tokens currently cached. |