RTDETR<T>
Models & Types · AiDotNet.ComputerVision.Detection.ObjectDetection.DETR
RT-DETR (Real-Time DEtection TRansformer) - the first real-time end-to-end object detector.
Inheritance: ModelBase<T, Tensor<T>, Tensor<T>> → VisionTaskModelBase<T> → ObjectDetectorBase<T> → RTDETR<T>
Implements: IDetectionTrainingModel<T>
For Beginners
RT-DETR predicts a fixed set of boxes end to end, like DETR, but at YOLO speed. It gets there two ways. A hybrid encoder runs full attention only on the coarsest feature map and fuses the scales with convolutions. The decoder starts from the encoder’s highest-scoring positions instead of learned queries. No NMS is needed.
How It Works
The architecture follows the reference implementation (lyuwenyu/RT-DETR, rtdetr_pytorch):
- ResNet C3-C5.
- The hybrid encoder: a ConvNorm projection, AIFI on S5 with a 2-D sin-cos encoding, and a CCFM
FPN + PAN of CSPRep blocks.
- IoU-aware query selection of the top 300 encoder tokens, whose detached features become the
content queries.
- Decoder layers with
MultiScaleDeformableAttention(3 levels x 4 points) and
per-layer heads with iterative refinement.
- Contrastive denoising.
The training objective is the reference’s: varifocal + L1 + GIoU with Hungarian matching on every decoder layer and on the encoder’s selected queries, plus the fixed-assignment denoising loss of every layer.
Reference: Zhao et al., “DETRs Beat YOLOs on Real-time Object Detection”, CVPR 2024
Constructors
| Constructor | Summary |
|---|---|
RTDETR(ObjectDetectionOptions<T> options) |
Creates RT-DETR. |
Properties
| Property | Summary |
|---|---|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
DetectionResult<T> Detect(Tensor<T> image, double confidenceThreshold, double nmsThreshold) |
|
Task LoadWeightsAsync(string pathOrUrl, CancellationToken cancellationToken) |
|
double EffectiveNmsThreshold(double requested) |
|
void SaveWeights(string path) |
|
void SetTrainingMode(bool training) |
|
void Train(Tensor<T> input, Tensor<T> expectedOutput) |
|
void TrainDetections(Tensor<T> input, DetectionTrainingBatch<T> targets) |