ConcertoCrossModalObjective<T>
Helpers & Utilities · AiDotNet.ComputerVision.Segmentation.PointCloud
Concerto’s 2D-3D cross-modal joint-embedding objective (arXiv:2510.23607).
How It Works
Aligns 3D point features with frozen 2D image-patch features. Correspondence is established exactly as the paper describes it — a 3D-to-2D projection followed by depth-based visibility verification — after which the point features falling inside each patch are mean-pooled and matched to that patch’s image embedding by cosine similarity.
Two details from the paper are load-bearing and easy to get wrong:
Methods
| Method | Summary |
|---|---|
T ComputeLoss(Tensor<T> pointFeatures, Tensor<T> pointCoordinates, IReadOnlyList<ConcertoPairedView<T>> views, double depthToleranceMeters) |
Computes the mean cosine-similarity loss between point-derived patch features and the frozen image encoder’s patch features, over every supplied view. |
Tensor<T> ComputeTapeLoss(Tensor<T> pointFeatures, Tensor<T> pointCoordinates, IReadOnlyList<ConcertoPairedView<T>> views, double depthToleranceMeters) |
The same objective as ComputeLoss, expressed in engine operations so a GradientTape can differentiate it with respect to the point features. |