BatcherStatistics

Models & Types · AiDotNet.Serving.ContinuousBatching

Statistics about the batcher’s operation.

Constructors

Constructor Summary
BatcherStatistics()

Properties

Property Summary
double AverageBatchSize { get; set; } Average batch size per iteration.
double MemoryUtilization { get; set; } Memory utilization (0-1).
double RequestsPerSecond { get; set; } Requests completed per second.
double RuntimeSeconds { get; set; } Total runtime in seconds.
double TokensPerSecond { get; set; } Tokens generated per second.
int RunningRequests { get; set; } Requests currently being processed.
int WaitingRequests { get; set; } Requests currently waiting.
long TotalIterations { get; set; } Total batching iterations.
long TotalRequestsProcessed { get; set; } Total requests completed since start.
long TotalTokensGenerated { get; set; } Total tokens generated since start.