NOTEARSSobolev<T>

Models & Types · AiDotNet.CausalDiscovery.ContinuousOptimization

NOTEARS with Sobolev regularization — DAG learning with smoothness constraints.

Inheritance: CausalDiscoveryBase<T> → ContinuousOptimizationBase<T> → NOTEARSSobolev<T>

For Beginners

Regular NOTEARS with neural networks might learn very wiggly functions that fit noise rather than real causal relationships. The Sobolev penalty encourages smoother functions, similar to how L2 regularization prevents large weights — but it penalizes the derivatives (wigglyness) of the learned functions, not just their magnitude.

How It Works

Extends NOTEARS nonlinear by adding a Sobolev-norm penalty on the functional relationships, which encourages smooth causal mechanisms. The Sobolev penalty penalizes the squared L2 norm of the Jacobian (first derivatives) of each MLP, preventing overfitting to noise.

Algorithm:

  • Initialize per-variable MLPs: Input(d) → Hidden(h, sigmoid) → Output(1)
  • Compute L2 reconstruction loss plus Sobolev penalty on Jacobian norms
  • Extract adjacency A[i,j] = ||W1[j][:,i]||_2 from input weights
  • Apply NOTEARS acyclicity constraint h(A) = tr(e^(A*A)) - d
  • Optimize via Adam with augmented Lagrangian for acyclicity
  • Threshold the final adjacency matrix

Sobolev Penalty: For each MLP f_j, the Sobolev penalty is: sum_i (df_j/dx_i)^2 averaged over data samples. This penalizes the sensitivity of each function to its inputs, encouraging smooth relationships. The Jacobian df_j/dx_i = sum_k W2[j][k] * sigmoid’(z_k) * W1[j][i,k] is computed analytically.

Reference: Zheng et al. (2020), “Learning Sparse Nonparametric DAGs”, AISTATS.

Constructors

Constructor Summary
NOTEARSSobolev(CausalDiscoveryOptions options) Initializes NOTEARS Sobolev with optional configuration.

Properties

Property Summary
bool SupportsNonlinear { get; }
string Name { get; }