PagedKVCacheStats

Models & Types · AiDotNet.Inference.PagedAttention

Statistics about the paged KV cache.

Constructors

Constructor Summary
PagedKVCacheStats()

Properties

Property Summary
BlockManagerStats BlockStats { get; set; } Underlying block manager statistics.
double AverageSequenceLength { get; set; } Average sequence length.
double MemoryEfficiency { get; set; } Memory efficiency compared to traditional allocation.
int ActiveSequences { get; set; } Number of active sequences.
long TotalTokensCached { get; set; } Total tokens currently cached.