Sonata<T>

Models & Types · AiDotNet.ComputerVision.Segmentation.PointCloud

Sonata: self-supervised learning of reliable point representations (Wu et al., CVPR 2025) – a Point Transformer V3 encoder pretrained by point self-distillation, read out by a linear probe.

Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → SemanticSegmentationBase<T> → Sonata<T>

For Beginners

Sonata learns what the points of a 3D scan mean without any labels: it shows a network several cropped, rotated and partly hidden copies of the same scene and trains it to describe each point the same way in every copy as a slowly-updated copy of itself (the teacher) does. Call Pretrain on unlabelled scans, then train the small classifier on top with a few labelled ones.

How It Works

What is reproduced (Pointcept Sonata-v1m1):

The losses, the target centering and an optional embedding regulariser (KoLeo) are pluggable through ISonataDistillationLoss, ISonataTargetCentering and ISonataFeatureRegularizer.

Input: [points, InputChannels] or [batch, points, InputChannels] whose first three channels are xyz in metres. Output: class logits [..., points, NumClasses].

Constructors

Constructor Summary
Sonata(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SonataOptions options) Initializes Sonata in native (trainable) mode.
Sonata(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, SonataOptions options) Initializes Sonata in ONNX (inference-only) mode.

Methods

Method Summary
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
ModelMetadata<T> GetModelMetadata()
T Pretrain(IReadOnlyList<Tensor<T>> scenes, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ISonataDistillationLoss<T> distillationLoss, ISonataTargetCentering<T> targetCentering, ISonataFeatureRegularizer<T> featureRegularizer) Runs Sonata’s point self-distillation on unlabelled scenes, one scene per step.
Tensor<T> GetClassMap(Tensor<T> image) Argmax over the class (last) axis: one class index per point.
Tensor<T> GetProbabilityMap(Tensor<T> image) Softmax over the class (last) axis.