PagedAttentionConfig
Options & Configuration · AiDotNet.Inference.PagedAttention
Configuration for paged attention kernel.
Constructors
| Constructor | Summary |
|---|---|
PagedAttentionConfig() |
Properties
| Property | Summary |
|---|---|
bool UseParallel { get; set; } |
Whether to use parallel processing for batched attention. |
bool UseTiling { get; set; } |
Whether to use Flash Attention-style tiling. |
int BlockSize { get; set; } |
Tokens per block. |
int HeadDimension { get; set; } |
Dimension of each head. |
int MaxBatchSize { get; set; } |
Maximum batch size for batched attention. |
int NumHeads { get; set; } |
Number of KEY/VALUE attention heads. |
int NumQueryHeads { get; set; } |
Number of QUERY attention heads. |
int WindowSize { get; set; } |
Sliding-window attention size (Mistral/Mixtral-style SWA). |