DepthAnythingV2<T>

Models & Types · AiDotNet.Video.Depth

Depth Anything V2 for monocular depth estimation.

Inheritance: NeuralNetworkBase<T> → DepthAnythingV2<T>

For Beginners

Depth Anything V2 is a state-of-the-art model for estimating depth maps from single images (monocular depth estimation). Given an RGB image, it predicts the relative distance of each pixel from the camera. This is useful for:

  • 3D scene understanding
  • Augmented reality applications
  • Autonomous driving
  • Video editing and VFX
  • Object detection and segmentation

Unlike stereo depth estimation which requires two cameras, Depth Anything works with a single image by learning depth cues from large-scale training data.

How It Works

Technical Details:

  • Vision Transformer (ViT) based encoder with DINOv2 initialization
  • Efficient multi-scale decoder for dense prediction
  • Scale-invariant depth loss for robust training
  • Supports various backbone sizes (Small, Base, Large)

Reference: Yang et al., “Depth Anything V2” 2024.

Example

// Create a Depth Anything V2 model for monocular depth estimation
var depthModel = new DepthAnythingV2<double>();
            
// Or configure with a specific backbone size
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.ThreeDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 256, inputWidth: 256, inputDepth: 3, outputSize: 1);
var options = new DepthAnythingV2Options(DepthAnythingV2ModelSize.Large);
var model = new DepthAnythingV2<double>(architecture, options: options);

Constructors

Constructor Summary
DepthAnythingV2() Initializes a new instance of the DepthAnythingV2 class in native (trainable) mode.
DepthAnythingV2(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, DepthAnythingV2Options options)
DepthAnythingV2(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, DepthAnythingV2Options options) Initializes a new instance of the DepthAnythingV2 class in ONNX (inference-only) mode.

Properties

Property Summary
bool SupportsTraining { get; } Gets whether training is supported.

Methods

Method Summary
List<Tensor<T>> EstimateVideoDepth(List<Tensor<T>> frames) Estimates depth for a sequence of video frames.
ModelMetadata<T> GetModelMetadata()
Tensor<T> EstimateDepth(Tensor<T> image) Estimates depth from an RGB image.
Tensor<T> ForwardForTraining(Tensor<T> input)
double GetDepthAtPoint(Tensor<T> depthMap, int x, int y) Gets the relative depth value at a specific point.