ConcertoPairedView<T>

Models & Types · AiDotNet.ComputerVision.Segmentation.PointCloud

One calibrated image paired with a point cloud, supplying the inputs Concerto’s cross-modal joint-embedding objective needs (arXiv:2510.23607).

For Beginners

To teach the 3D model from a photo, you first have to work out which pixel each 3D point lands on. That needs the camera’s lens parameters (intrinsics), where the camera was standing (extrinsics), and a depth image to tell whether the point was actually visible or hidden behind something.

How It Works

Concerto aligns 3D point features with 2D image-patch features taken from a frozen DINOv2 encoder. Establishing which point corresponds to which patch requires more than the image itself: the point cloud has to be projected into the camera and then depth-checked for visibility, so the camera’s intrinsics and its pose relative to the cloud are both required. Without them, occluded points would be paired with whatever surface is drawn in front of them.

Constructors

Constructor Summary
ConcertoPairedView()

Properties

Property Summary
Tensor<T> DepthMap { get; set; } Rendered depth for this view, used to verify that a projected point is actually visible rather than occluded.
Tensor<T> Extrinsics { get; set; } Camera extrinsics as a row-major 4x4 world-to-camera transform.
Tensor<T> ImagePatchFeatures { get; set; } Patch features produced by the frozen 2D image encoder for this view.
Tensor<T> Intrinsics { get; set; } Camera intrinsics as a row-major 3x3 matrix, used to project camera-space points onto the image plane.
int PatchGridSize { get; set; } Number of patches along each axis of the image encoder’s patch grid.