Sonata<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.PointCloud
Sonata: self-supervised learning of reliable point representations (Wu et al., CVPR 2025) – a Point Transformer V3 encoder pretrained by point self-distillation, read out by a linear probe.
Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → SemanticSegmentationBase<T> → Sonata<T>
For Beginners
Sonata learns what the points of a 3D scan mean without any labels: it
shows a network several cropped, rotated and partly hidden copies of the same scene and trains it
to describe each point the same way in every copy as a slowly-updated copy of itself (the
teacher) does. Call Pretrain on unlabelled scans, then train the small classifier
on top with a few labelled ones.
How It Works
What is reproduced (Pointcept Sonata-v1m1):
The losses, the target centering and an optional embedding regulariser (KoLeo) are pluggable
through ISonataDistillationLoss, ISonataTargetCentering and
ISonataFeatureRegularizer.
Input: [points, InputChannels] or [batch, points, InputChannels] whose first
three channels are xyz in metres. Output: class logits [..., points, NumClasses].
Constructors
| Constructor | Summary |
|---|---|
Sonata(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SonataOptions options) |
Initializes Sonata in native (trainable) mode. |
Sonata(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, SonataOptions options) |
Initializes Sonata in ONNX (inference-only) mode. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
ModelMetadata<T> GetModelMetadata() |
|
T Pretrain(IReadOnlyList<Tensor<T>> scenes, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ISonataDistillationLoss<T> distillationLoss, ISonataTargetCentering<T> targetCentering, ISonataFeatureRegularizer<T> featureRegularizer) |
Runs Sonata’s point self-distillation on unlabelled scenes, one scene per step. |
Tensor<T> GetClassMap(Tensor<T> image) |
Argmax over the class (last) axis: one class index per point. |
Tensor<T> GetProbabilityMap(Tensor<T> image) |
Softmax over the class (last) axis. |