PagedAttentionConfig

Options & Configuration · AiDotNet.Inference.PagedAttention

Configuration for paged attention kernel.

Constructors

Constructor Summary
PagedAttentionConfig()

Properties

Property Summary
bool UseParallel { get; set; } Whether to use parallel processing for batched attention.
bool UseTiling { get; set; } Whether to use Flash Attention-style tiling.
int BlockSize { get; set; } Tokens per block.
int HeadDimension { get; set; } Dimension of each head.
int MaxBatchSize { get; set; } Maximum batch size for batched attention.
int NumHeads { get; set; } Number of KEY/VALUE attention heads.
int NumQueryHeads { get; set; } Number of QUERY attention heads.
int WindowSize { get; set; } Sliding-window attention size (Mistral/Mixtral-style SWA).