IGroundingTrainable<T>
Interfaces · AiDotNet.VisionLanguage.Interfaces
A grounding model that trains on grounded phrases: an image, a caption, and the boxes the caption’s phrases name.
Known implementations: Ferret<T>, GroundedSAM2<T>, GroundingDINO<T>, Shikra<T>
How It Works
Such a model’s paper loss matches its predictions to the boxes and scores them against the caption’s tokens. A tensor
target cannot carry the phrases, so these models refuse Train(input, expected) and train here instead.
Methods
| Method | Summary |
|---|---|
T EvaluateGroundingLoss(Tensor<T> image, string caption, IReadOnlyList<GroundingBox> boxes) |
The paper loss on the same supervision, without training. |
void TrainGrounding(Tensor<T> image, string caption, IReadOnlyList<GroundingBox> boxes) |
One training step with the model’s paper loss. |