BatcherStatistics
Models & Types · AiDotNet.Serving.ContinuousBatching
Statistics about the batcher’s operation.
Constructors
| Constructor | Summary |
|---|---|
BatcherStatistics() |
Properties
| Property | Summary |
|---|---|
double AverageBatchSize { get; set; } |
Average batch size per iteration. |
double MemoryUtilization { get; set; } |
Memory utilization (0-1). |
double RequestsPerSecond { get; set; } |
Requests completed per second. |
double RuntimeSeconds { get; set; } |
Total runtime in seconds. |
double TokensPerSecond { get; set; } |
Tokens generated per second. |
int RunningRequests { get; set; } |
Requests currently being processed. |
int WaitingRequests { get; set; } |
Requests currently waiting. |
long TotalIterations { get; set; } |
Total batching iterations. |
long TotalRequestsProcessed { get; set; } |
Total requests completed since start. |
long TotalTokensGenerated { get; set; } |
Total tokens generated since start. |