NeRFNetwork<T>

Models & Types · AiDotNet.Diffusion.ThreeD

Neural Radiance Field network for 3D representation.

Implements: IParameterSource<T>

How It Works

A NeRF MLP (Mildenhall et al., 2020) rendered by batched volume rendering: every ray of a chunk is sampled at SamplesPerRay stratified depths in [0.5, 3], the MLP is evaluated on all samples at once, and the colour is composited with C = sum_i T_i (1 - exp(-sigma_i delta)) c_i, T_i = exp(-sum_{j < i} sigma_j delta). Because the whole render is made of engine ops, the SDS update back-propagates the score-distillation gradient through the renderer into the MLP weights (DreamFusion, Poole et al., 2022), instead of perturbing them at random.

Constructors

Constructor Summary
NeRFNetwork(int hiddenDim, int numLayers, int? seed) Initializes a new NeRF network.

Properties

Property Summary
long ParameterCount { get; } Gets the total parameter count.

Methods

Method Summary
Tensor<T> Render(CameraPose cameraPose, int resolution) Renders an image from the NeRF at a given camera pose.
Vector<T> GetParameters() Gets all network parameters.
double QueryDensity(double x, double y, double z) Queries the density at a 3D point.
void Initialize() Initializes network weights.
void SetParameters(Vector<T> parameters) Sets network parameters.
void UpdateParameters(Tensor<T> gradient, CameraPose cameraPose, double learningRate) Applies one SDS step: back-propagates the image-space gradient gradient [R, R, 3] (the score-distillation residual w(t) (eps_hat - eps)) through the volume renderer at cameraPose and descends the MLP weights by learningRate times it…