KVCache<T>

Models & Types · AiDotNet.Tensors.Engines.Autodiff

No summary documentation available yet.

Constructors

Constructor Summary
KVCache(int maxBatch, int maxSeq, int heads, int headDim)

Properties

Property Summary
Tensor<T> Keys { get; }
Tensor<T> Values { get; }
int HeadDim { get; }
int Heads { get; }
int MaxBatch { get; }
int MaxSeq { get; }

Methods

Method Summary
ValueTuple<Tensor<T>, Tensor<T>> Slice(int batchIdx)
int GetLength(int batchIdx)
void Append(int batchIdx, Tensor<T> newKeys, Tensor<T> newValues)
void Reset(int batchIdx)
void ResetAll()