PagedKVCache<T>

Models & Types · AiDotNet.Tensors.Engines.Autodiff

No summary documentation available yet.

Constructors

Constructor Summary
PagedKVCache(int maxBlocks, int blockSize, int heads, int headDim)

Properties

Property Summary
int AllocatedBlocks { get; }
int BlockSize { get; }
int HeadDim { get; }
int Heads { get; }
int MaxBlocks { get; }

Methods

Method Summary
IReadOnlyList<int> GetBlockTable(int seqId)
ValueTuple<Tensor<T>, Tensor<T>> Materialize(int seqId)
int GetLength(int seqId)
void Append(int seqId, Tensor<T> newKeys, Tensor<T> newValues)
void Free(int seqId)
void ShareBlocks(int sourceSeqId, int targetSeqId, int prefixLen)