KVCache<T>
Models & Types · AiDotNet.Tensors.Engines.Autodiff
No summary documentation available yet.
Constructors
| Constructor |
Summary |
KVCache(int maxBatch, int maxSeq, int heads, int headDim) |
|
Properties
| Property |
Summary |
Tensor<T> Keys { get; } |
|
Tensor<T> Values { get; } |
|
int HeadDim { get; } |
|
int Heads { get; } |
|
int MaxBatch { get; } |
|
int MaxSeq { get; } |
|
Methods
| Method |
Summary |
ValueTuple<Tensor<T>, Tensor<T>> Slice(int batchIdx) |
|
int GetLength(int batchIdx) |
|
void Append(int batchIdx, Tensor<T> newKeys, Tensor<T> newValues) |
|
void Reset(int batchIdx) |
|
void ResetAll() |
|