PagedKVCache<T>
Models & Types · AiDotNet.Tensors.Engines.Autodiff
No summary documentation available yet.
Constructors
| Constructor |
Summary |
PagedKVCache(int maxBlocks, int blockSize, int heads, int headDim) |
|
Properties
| Property |
Summary |
int AllocatedBlocks { get; } |
|
int BlockSize { get; } |
|
int HeadDim { get; } |
|
int Heads { get; } |
|
int MaxBlocks { get; } |
|
Methods
| Method |
Summary |
IReadOnlyList<int> GetBlockTable(int seqId) |
|
ValueTuple<Tensor<T>, Tensor<T>> Materialize(int seqId) |
|
int GetLength(int seqId) |
|
void Append(int seqId, Tensor<T> newKeys, Tensor<T> newValues) |
|
void Free(int seqId) |
|
void ShareBlocks(int sourceSeqId, int targetSeqId, int prefixLen) |
|