Neural Networks

738 public types in this namespace, organized by kind.

Models & Types (206)

Type Summary
ACGAN<T> Represents an Auxiliary Classifier Generative Adversarial Network (AC-GAN), which extends conditional GANs by having the discriminator also predict the class label of the input.
AIMGenerator<T> AIM (Adaptive and Iterative Mechanism) generator for differentially private synthetic data (McKenna, Mullins, Sheldon, Miklau 2022).
AttentionNetwork<T> Represents a neural network that utilizes attention mechanisms for sequence processing.
AudioTextDualStreamArchitecture<T> A neural network architecture for audio + text two-stream models (CLAP-family encoders) that hosts a separate audio encoder and text encoder.
AudioVisualCorrespondenceNetwork<T> Audio-visual correspondence learning network for cross-modal understanding.
AudioVisualEventLocalizationNetwork<T> Neural network for audio-visual event localization - identifying WHEN and WHERE events occur in video by jointly analyzing audio and visual streams with precise temporal boundaries.
AutoDiffTabGenerator<T> AutoDiff-Tab generator that automatically searches over diffusion configurations (timesteps, noise schedules, network architecture) to find optimal settings for tabular data generation.
AutoIntClassifier<T> AutoInt implementation for classification tasks.
AutoIntNetwork<T> AutoInt (Automatic Feature Interaction Learning) neural network for tabular data.
AutoIntRegression<T> AutoInt implementation for regression tasks.
Autoencoder<T> Represents an autoencoder neural network that can compress data into a lower-dimensional representation and reconstruct it.
BGE<T> BGE (BAAI General Embedding) neural network implementation.
BayesianNetworkSynthGenerator<T> PrivBayes synthesizer (Zhang et al., SIGMOD 2014 / TODS 2017): learns a k-degree Bayesian network under differential privacy, releases noisy conditional distributions, and generates synthetic data by ancestral sampling.
BigGAN<T> BigGAN (Brock et al.
Blip2NeuralNetwork<T> BLIP-2 (Bootstrapped Language-Image Pre-training 2) neural network for vision-language tasks.
BlipNeuralNetwork<T> BLIP (Bootstrapped Language-Image Pre-training) neural network for vision-language tasks.
BlockwiseAnchorLocalErrorCreditRule<T> Blockwise anchor local error: decoupled block-local sequence credit assignment that partitions a deep network into contiguous blocks of at most AnchorStride trainable pre-readout layers, places a shared (full or candidate-sampled) tie…
BoundInputContract A static input manifest after instance configuration and concrete shape are resolved.
CLSToken<T> CLS (Classification) Token for transformer-based tabular models.
CTABGANPlusGenerator<T> CTAB-GAN+ generator for high-quality synthetic tabular data with auxiliary classifier discriminator, mixed-type encoder, and information loss.
CTGANDataSampler<T> Handles conditional vector generation and training-by-sampling for CTGAN.
CTGANGenerator<T> Conditional Tabular GAN (CTGAN) for generating realistic synthetic tabular data.
CapsuleNetwork<T> Represents a Capsule Network, a type of neural network that preserves spatial relationships between features.
CausalGANGenerator<T> Causal-GAN generator that learns causal graph structure (directed acyclic graph) and generates synthetic data respecting causal relationships between features.
ClipNeuralNetwork<T> CLIP (Contrastive Language-Image Pre-training) neural network that encodes both text and images into a shared embedding space, enabling cross-modal similarity and zero-shot classification.
ColBERT<T> ColBERT (Contextualized Late Interaction over BERT) neural network implementation.
ColumnEmbedding<T> Column (positional) embedding for tabular transformers like TabTransformer.
ColumnMetadata Describes the metadata for a single column in a tabular dataset, including its name, data type, categories (for categorical columns), and summary statistics.
ColumnTransformInfo Describes how a single original column maps into the transformed representation.
ConditionalGAN<T> Represents a Conditional Generative Adversarial Network (cGAN), which generates data conditioned on additional information such as class labels, attributes, or other contextual data.
Connection<T> Represents a connection between two nodes in a neural network, particularly used in evolving neural networks.
ContextEncoder<T> Context encoder for TabR that processes retrieved neighbors into a context representation.
ContrastivePretraining<T> Contrastive pretraining module for SAINT architecture.
ConvolutionalNeuralNetwork<T> Represents a Convolutional Neural Network (CNN) that processes multi-dimensional data.
CopulaGANGenerator<T> CopulaGAN generator for synthetic tabular data, combining Gaussian copula transformations with the CTGAN training pipeline for improved continuous column modeling.
CopulaSynthGenerator<T> Copula-Based Synthesis generator that models marginal distributions independently and couples them with a Gaussian copula to capture inter-feature dependencies.
CycleGAN<T> Represents a CycleGAN for unpaired image-to-image translation.
DCGAN<T> Represents a Deep Convolutional Generative Adversarial Network (DCGAN), an architecture that uses convolutional and transposed convolutional layers with specific design guidelines for stable training.
DPCTGANGenerator<T> Differentially Private CTGAN (DP-CTGAN) for generating synthetic tabular data with formal (epsilon, delta)-differential privacy guarantees.
DeepBeliefNetwork<T> Represents a Deep Belief Network, a generative graphical model composed of multiple layers of Restricted Boltzmann Machines.
DeepBoltzmannMachine<T> Represents a Deep Boltzmann Machine (DBM), a hierarchical generative model consisting of multiple layers of stochastic neurons.
DeepQNetwork<T> Represents a Deep Q-Network (DQN), a reinforcement learning algorithm that combines Q-learning with deep neural networks.
DenseNetNetwork<T> Implements the DenseNet (Densely Connected Convolutional Network) architecture.
DifferentiableNeuralComputer<T> Represents a Differentiable Neural Computer (DNC), a neural network architecture that combines neural networks with external memory resources.
DualStreamArchitecture<T> A neural network architecture for vision + text two-stream models (CLIP-family encoders: BASIC, DFNCLIP, EVACLIP, LiT, RegionCLIP, etc.) that hosts a separate vision encoder and text encoder.
EagleLanguageModel<T> Implements a full RWKV-5 “Eagle” language model: token embedding + N RWKVLayer blocks + RMS normalization + LM head.
EchoStateNetwork<T> Represents an Echo State Network (ESN), a type of recurrent neural network with a sparsely connected hidden layer called a reservoir.
EfficientNetNetwork<T> Implements the EfficientNet architecture with compound scaling.
EntmaxAttention<T> Entmax sparse attention function for NODE architecture.
EntmoidActivation<T> Entmoid activation function for NODE architecture.
ExtremeLearningMachine<T> Represents an Extreme Learning Machine (ELM), a type of feedforward neural network with a unique training approach.
FTTransformerClassifier<T> FT-Transformer implementation for classification tasks.
FTTransformerNetwork<T> FT-Transformer (Feature Tokenizer + Transformer) neural network for tabular data.
FTTransformerRegression<T> FT-Transformer implementation for regression tasks.
FalconMambaLanguageModel<T> Implements a Falcon Mamba language model: embedding + N MambaBlock blocks + RMS norm + LM head.
FastText<T> FastText neural network implementation, an extension of Word2Vec that considers subword information.
FeatureTokenizer<T> Implements the Feature Tokenizer that converts tabular features into embeddings for FT-Transformer.
FeedForwardNeuralNetwork<T> Represents a Feed-Forward Neural Network (FFNN) for processing data in a forward path.
FinDiffGenerator<T> FinDiff generator for synthesizing realistic financial tabular data using diffusion models with temporal correlation preservation and financial constraint enforcement.
FinchLanguageModel<T> Implements a full RWKV-6 “Finch” language model: token embedding + N RWKVLayer blocks + RMS normalization + LM head.
FlamingoNeuralNetwork<T> Flamingo neural network for in-context visual learning and few-shot tasks.
GANDALFClassifier<T> GANDALF implementation for classification tasks.
GANDALFNetwork<T> GANDALF (Gated Additive Neural Decision Forest) neural network for tabular data.
GANDALFRegression<T> GANDALF implementation for regression tasks.
GLALanguageModel<T> Implements a full GLA (Gated Linear Attention) language model: embedding + N GLA blocks + RMS norm + LM head.
GOGGLEGenerator<T> GOGGLE generator that learns feature dependency structure via a graph neural network combined with a VAE framework for high-quality synthetic tabular data generation.
GRUNeuralNetwork<T> Represents a Gated Recurrent Unit (GRU) Neural Network for processing sequential data.
GatedDeltaNetLanguageModel<T> Implements a full Gated DeltaNet language model: token embedding + N GatedDeltaNetLayer blocks + RMS norm + LM head.
GaussianDiffusion<T> Implements the Gaussian diffusion process for continuous/numerical features in TabDDPM.
GenerativeAdversarialNetwork<T> Represents a Generative Adversarial Network (GAN), a deep learning architecture that consists of two neural networks (a generator and a discriminator) competing against each other in a zero-sum game.
Genome<T> Represents a genome in a neuroevolutionary algorithm, containing a collection of connections between nodes.
GhostBatchNormalization<T> Implements Ghost Batch Normalization, a regularization technique used in TabNet that applies batch normalization to virtual mini-batches within each actual batch.
GloVe<T> GloVe (Global Vectors for Word Representation) neural network implementation.
Gpt4VisionNeuralNetwork<T> GPT-4V-style neural network that combines vision understanding with large language model capabilities.
GraphAttentionNetwork<T> Represents a Graph Attention Network (GAT) that uses attention mechanisms to process graph-structured data.
GraphClassificationModel<T> Implements a complete neural network model for graph classification tasks.
GraphGenerationModel<T> Represents a Graph Generation Model using Variational Autoencoder (VAE) architecture.
GraphIsomorphismNetwork<T> Represents a Graph Isomorphism Network (GIN) for powerful graph representation learning.
GraphNeuralNetwork<T> Represents a Graph Neural Network that can process data represented as graphs.
GraphSAGENetwork<T> Represents a GraphSAGE (Graph Sample and Aggregate) Network for inductive learning on graphs.
GriffinLanguageModel<T> Implements a Griffin language model: embedding + N RGLR blocks with local attention + layer norm + LM head.
HTMNetwork<T> Represents a Hierarchical Temporal Memory (HTM) network, a biologically-inspired sequence learning algorithm.
HawkLanguageModel<T> Implements a Hawk language model: embedding + N pure RGLR blocks + layer norm + LM head.
HopeNetwork<T> Hope architecture - a self-modifying recurrent neural network variant of Titans with unbounded levels of in-context learning.
HopfieldNetwork<T> Represents a Hopfield Network, a recurrent neural network designed for pattern storage and retrieval.
HyperbolicNeuralNetwork<T> Represents a Hyperbolic Neural Network for learning hierarchical representations in Poincare ball space.
ImageBindNeuralNetwork<T> ImageBind neural network for binding multiple modalities (6+) into a shared embedding space.
InContextLearning<T> Helper class for in-context learning in tabular foundation models like TabPFN.
InfoGAN<T> Represents an Information Maximizing Generative Adversarial Network (InfoGAN), which learns disentangled representations in an unsupervised manner by maximizing mutual information between latent codes and generated observations.
InputContractManifest Canonical input schema for a layer or model.
InputContractVariant A named set of mutually compatible input ports.
InstructorEmbedding<T> Instructor/E5 (Instruction-Tuned) embedding model implementation.
JambaLanguageModel<T> Implements a Jamba language model: embedding + HybridBlockScheduler (Mamba + Attention) + RMS norm + LM head.
LLaVANeuralNetwork<T> LLaVA (Large Language and Vision Assistant) neural network for visual instruction following.
LSTMNeuralNetwork<T> Represents a Long Short-Term Memory (LSTM) Neural Network, which is specialized for processing sequential data like text, time series, or audio.
LayerGraph<T> The wiring of a model, expressed once: which layer feeds which, and what happens on the edges between them.
LayerwiseDistillationCreditRule<T> Layerwise distillation as local credit: every hidden layer of the student is trained, with its input detached, to reproduce one hidden state of a frozen teacher model through a learned linear map `P_k : student width → teacher width…
LinkPredictionModel<T> Implements a complete neural network model for link prediction tasks on graphs.
LiquidStateMachine<T> Represents a Liquid State Machine (LSM), a type of reservoir computing neural network.
Mamba2LanguageModel<T> Implements a Mamba-2 language model: token embedding + N Mamba2Blocks + layer normalization + LM head.
MambaLanguageModel<T> Implements a full Mamba language model: token embedding + N MambaBlocks + RMS normalization + LM head.
MambaModelState<T> Per-token (KV-cached) decoding state for a MambaLanguageModel: one MambaStepState per Mamba block, in layer order.
MambularClassifier<T> Mambular implementation for classification tasks.
MambularNetwork<T> Mambular (State Space Model for Tabular Data) neural network.
MambularRegression<T> Mambular implementation for regression tasks.
MatryoshkaEmbedding<T> Matryoshka Representation Learning (MRL) neural network implementation.
MedGANGenerator<T> medGAN — generates synthetic patient records by pairing a pre-trained autoencoder with a generative adversarial network whose generator produces points in the autoencoder’s latent space.
MemoryNetwork<T> Represents a Memory Network, a neural network architecture designed with explicit memory components for improved reasoning and question answering capabilities.
MeshCNN<T> Implements the MeshCNN architecture for processing 3D triangle meshes.
MisGANGenerator<T> MisGAN generator for learning from incomplete data using dual generator/discriminator pairs for both data values and missingness patterns.
MixtureOfExpertsNeuralNetwork<T> Represents a Mixture-of-Experts (MoE) neural network that routes inputs through multiple specialist networks.
MobileNetV2Network<T> Implements the MobileNetV2 architecture for efficient mobile inference.
MobileNetV3Network<T> Implements the MobileNetV3 architecture for efficient mobile inference.
MultinomialDiffusion<T> Implements the multinomial diffusion process for categorical features in TabDDPM.
NEAT<T> NeuroEvolution of Augmenting Topologies (Stanley & Miikkulainen 2002): evolves the structure and the weights of feed-forward networks with historical markings, speciation and explicit fitness sharing.
NODEClassifier<T> NODE implementation for classification tasks.
NODENetwork<T> NODE (Neural Oblivious Decision Ensembles) neural network for tabular data.
NODERegression<T> NODE implementation for regression tasks.
NeuralNetworkArchitecture<T> Defines the structure and configuration of a neural network, including its layers, input/output dimensions, and task-specific properties.
NeuralNetwork<T> A neural network implementation that processes data through multiple layers to make predictions.
NeuralTuringMachine<T> Represents a Neural Turing Machine, which is a neural network architecture that combines a neural network with external memory.
NodeClassificationModel<T> Implements a complete neural network model for node classification tasks on graphs.
OCTGANGenerator<T> OCT-GAN (Kim et al.
OccupancyNeuralNetwork<T> Represents a Neural Network specialized for occupancy detection and prediction in spaces.
OctonionNeuralNetwork<T> Represents an Octonion-valued Neural Network for processing data in 8-dimensional hypercomplex space.
PATEGANGenerator<T> PATE-GAN (Jordon, Yoon and van der Schaar, ICLR 2019): a GAN made differentially private by replacing its discriminator with the PATE mechanism.
Pix2Pix<T> Represents a Pix2Pix GAN for paired image-to-image translation tasks.
PlanCache Disk-backed store for compiled inference plans.
ProgressiveGAN<T> Progressive GAN (ProGAN, Karras et al.
QuantumNeuralNetwork<T> Represents a Quantum Neural Network, which combines quantum computing principles with neural network architecture.
REaLTabFormerGenerator<T> REaLTabFormer generator using GPT-2 style autoregressive transformer for synthetic tabular data generation by treating columns as a sequence of tokens.
RWKV4LanguageModel<T> Implements a full RWKV-4 language model: token embedding + N RWKVLayer blocks + layer normalization + LM head.
RWKV7LanguageModel<T> Implements a full RWKV-7 “Goose” language model: token embedding + N RWKV7Blocks + RMS normalization + LM head.
RadialBasisFunctionNetwork<T> Represents a Radial Basis Function Network, which is a type of neural network that uses radial basis functions as activation functions.
RecurrentGemmaLanguageModel<T> Implements a RecurrentGemma language model: embedding + N RGLR blocks + layer norm + LM head.
RecurrentNeuralNetwork<T> Represents a Recurrent Neural Network, which is a type of neural network designed to process sequential data by maintaining an internal state.
ResNetNetwork<T> Represents a ResNet (Residual Network) neural network architecture for image classification.
ResidualNeuralNetwork<T> Represents a Residual Neural Network, which is a type of neural network that uses skip connections to address the vanishing gradient problem in deep networks.
RestrictedBoltzmannMachine<T> Represents a Restricted Boltzmann Machine, which is a type of neural network that learns probability distributions over its inputs.
RetrievalContext<T> Context returned by the retrieval module.
RetrievalModule<T> Retrieval module for TabR (Retrieval-Augmented Tabular Learning).
SAGAN<T> Self-Attention GAN (SAGAN) — a deep convolutional GAN whose generator maps a 1D latent vector to an image and whose discriminator scores images as real or fake.
SAINTClassifier<T> SAINT implementation for classification tasks.
SAINTNetwork<T> SAINT (Self-Attention and Intersample Attention Transformer) neural network for tabular data.
SAINTRegression<T> SAINT implementation for regression tasks.
SGPT<T> SGPT (Sentence GPT) neural network implementation using decoder-only transformer architectures.
SMOTENCGenerator<T> SMOTE-NC generator that creates synthetic minority samples by interpolating between existing minority samples and their k-nearest neighbors, supporting both continuous and categorical features.
SPLADE<T> SPLADE (Sparse Lexical and Expansion Model) neural network implementation.
SambaLanguageModel<T> Implements a Samba language model: embedding + HybridBlockScheduler (Mamba + sliding window attention) + RMS norm + LM head.
SampledTiedReadoutLocalErrorCreditRule<T> Sampled tied-readout local error: zero-auxiliary-parameter sequence local credit assignment that evaluates each hidden layer’s next-token cross-entropy over a compact candidate subset U ≪ V of the network’s shared readout (all unique…
SelfOrganizingMap<T> Represents a Self-Organizing Map (Kohonen map): an unsupervised neural network that maps high-dimensional input onto a 2-D grid of prototype neurons, preserving input topology.
SiameseNetwork<T> Implements a Siamese Neural Network for comparing pairs of inputs and determining their similarity.
SiameseNeuralNetwork<T> Sentence-BERT (SBERT) style shared sentence-encoder tower: a transformer encoder that maps a tokenized input to a fixed-size embedding (default 768-d, BERT vocab 30522, max length 512) for semantic similarity and retrieval.
SimCSE<T> SimCSE (Simple Contrastive Learning of Sentence Embeddings) neural network implementation.
SparseNeuralNetwork<T> Represents a Sparse Neural Network with efficient sparse weight matrices.
Sparsemax<T> Implements the Sparsemax activation function, which projects input onto the probability simplex with sparse outputs (many exact zeros).
SpikingNeuralNetwork<T> Represents a Spiking Neural Network, which is a type of neural network that more closely models biological neurons with temporal dynamics.
SpiralNet<T> Implements the SpiralNet++ architecture for mesh-based deep learning.
StyleGAN<T> Represents a StyleGAN (Style-Based Generator Architecture for GANs) that generates high-quality images with fine-grained control over image style at different levels.
SuperNet<T> SuperNet implementation for gradient-based neural architecture search (DARTS).
TVAEGenerator<T> Tabular Variational Autoencoder (TVAE) for generating synthetic tabular data.
TabDDPMGenerator<T> TabDDPM (Tabular Denoising Diffusion Probabilistic Model) for generating synthetic tabular data.
TabDPTClassifier<T> TabDPT implementation for classification tasks.
TabDPTNetwork<T> TabDPT (Tabular Data Pre-Training) neural network for tabular data.
TabDPTRegression<T> TabDPT implementation for regression tasks.
TabFlowGenerator<T> TabFlow generator using flow matching with optimal transport conditional paths for high-quality, fast synthetic tabular data generation.
TabLLMGenGenerator<T> TabLLM-Gen generator that uses LLM-style schema-aware tokenization and autoregressive transformers to generate realistic tabular data.
TabMClassifier<T> TabM implementation for classification tasks.
TabMNetwork<T> TabM (Parameter-Efficient Ensemble) neural network for tabular data.
TabMRegression<T> TabM implementation for regression tasks.
TabNetClassifier<T> TabNet implementation for classification tasks.
TabNetNetwork<T> TabNet neural network for interpretable tabular learning.
TabNetRegression<T> TabNet implementation for regression tasks.
TabPFNClassifier<T> TabPFN implementation for classification tasks.
TabPFNNetwork<T> TabPFN (Prior-Fitted Network) neural network for tabular data.
TabPFNRegression<T> TabPFN implementation for regression tasks.
TabRClassifier<T> TabR implementation for classification tasks.
TabRNetwork<T> TabR (Retrieval-Augmented) neural network for tabular data.
TabRRegression<T> TabR implementation for regression tasks.
TabSynGenerator<T> TabSyn generator combining VAE pretraining with latent diffusion for state-of-the-art synthetic tabular data generation.
TabTransformerClassifier<T> TabTransformer implementation for classification tasks.
TabTransformerGenGenerator<T> TabTransformer-Gen generator that uses column-wise contextual embeddings and masked prediction to generate realistic tabular data.
TabTransformerNetwork<T> TabTransformer neural network for tabular data with categorical features.
TabTransformerRegression<T> TabTransformer implementation for regression tasks.
TableGANGenerator<T> TableGAN generator using a DCGAN-style architecture with classification and information loss for high-quality synthetic tabular data generation.
TabularDataTransformer<T> Transforms tabular data using Variational Gaussian Mixture (VGM) mode-specific normalization for continuous columns and one-hot encoding for categorical columns.
TiedReadoutLocalErrorCreditRule<T> Tied-readout local error: a sequence-model form of Local Error Signals (Nøkland & Eidnes, 2019) in which every hidden layer’s local classifier is the network’s own shared readout (for a language model: the final norm and the tied ou…
TimeGANGenerator<T> TimeGAN generator for synthesizing realistic time-series tabular data while preserving temporal dynamics using an embedding-supervisor-adversarial training framework.
TransformerArchitecture<T> Defines the architecture configuration for a Transformer neural network.
TransformerEmbeddingNetwork<T> A customizable Transformer-based embedding network.
Transformer<T> Represents a Transformer neural network architecture, which is particularly effective for sequence-based tasks like natural language processing.
TripleStreamArchitecture<T> A neural network architecture for three-stream models (e.g., perceiver/abstractor/resampler-style generative VLMs and cross-modality fusion encoders) that hosts a vision encoder, an auxiliary stream (perceiver / abstractor / resampler / cro…
UNet3D<T> Represents a 3D U-Net neural network for volumetric semantic segmentation.
UnifiedMultimodalNetwork<T> Unified multimodal network that handles text, images, audio, and video in a single architecture with cross-modal attention and any-to-any generation.
VGGNetwork<T> Represents a VGG (Visual Geometry Group) neural network architecture for image classification.
VariationalAutoencoder<T> Represents a Variational Autoencoder (VAE) neural network architecture, which is used for generating new data similar to the training data and learning compressed representations.
VideoCLIPNeuralNetwork<T> VideoCLIP neural network for video-text alignment and temporal understanding.
VisionMambaModel<T> Vision Mamba (Vim) image classification: the official network with a linear classifier on its class token.
VisionTransformer<T> Implements the Vision Transformer (ViT) architecture for image classification tasks.
VoxelCNN<T> Represents a Voxel-based 3D Convolutional Neural Network for processing volumetric data.
WGANGP<T> Represents a Wasserstein GAN with Gradient Penalty (WGAN-GP), an improved version of WGAN that uses gradient penalty instead of weight clipping to enforce the Lipschitz constraint.
WGAN<T> Represents a Wasserstein Generative Adversarial Network (WGAN), which uses the Wasserstein distance (Earth Mover’s distance) to measure the difference between the generated and real data distributions.
Word2Vec<T> Word2Vec neural network implementation supporting both Skip-Gram and CBOW architectures.
XLSTMLanguageModel<T> Implements a full xLSTM language model: token embedding + N ExtendedLSTMLayer blocks + RMS normalization + LM head.
Zamba2LanguageModel<T> Implements a Zamba2 language model: embedding + HybridBlockScheduler (Mamba2 + shared attention with LoRA) + RMS norm + LM head.
ZambaLanguageModel<T> Implements a Zamba language model: embedding + HybridBlockScheduler (Mamba + shared attention) + RMS norm + LM head.

Layers (332)

Type Summary
ABCLayer<T> Implements the ABC (Attention with Bounded-memory Control) layer from Peng et al., 2022.
ALiBiPositionalBiasLayer<T> Implements Attention with Linear Biases (ALiBi) from Press et al., 2022.
AcousticConditionEncoderLayer<T> AdaSpeech’s acoustic condition encoder: two blocks of Conv1D → ReLU → LayerNorm → Dropout, then either a mean pooling over time (one vector per utterance) or a linear projection per position (one vector per phoneme).
ActNormFlowLayer<T> Activation normalization (Kingma & Dhariwal 2018): a per-channel affine map z = b + e^s ⊙ x whose scale and bias are initialized from the first batch so that its output has zero mean and unit variance per channel.
ActivationLayer<T> A layer that applies an activation function to transform the input data.
AdaptiveAveragePoolingLayer<T> Implements adaptive average pooling that outputs a fixed spatial size regardless of input dimensions.
AddLayer<T> A layer that adds multiple input tensors element-wise and optionally applies an activation function.
AffineCouplingFlowLayer<T> An affine coupling layer with a WaveNet-like transform network (WaveGlow; Glow-TTS §3.3): the first half of the channels passes through and parameterizes a shift and log-scale applied to the second half.
AnomalyDetectorLayer<T> Represents a layer that detects anomalies by comparing predictions with actual inputs.
AttentionLayer<T> Represents an Attention Layer for focusing on relevant parts of input sequences.
AttentionalFeatureFusionLayer<T> Attentional feature fusion, AFF (Dai et al., “Attentional Feature Fusion”, WACV 2021), in its 2-D form: merges two feature maps with a per-position, per-channel soft selection computed from both.
AttentiveTransformerLayer<T> Implements the Attentive Transformer block used in TabNet architecture for feature selection.
AutoregressiveEncoderDecoderLayer<T> Owns the two execution branches of an autoregressive encoder-decoder architecture.
AveragePoolingLayer<T> Implements an average pooling layer for neural networks, which reduces the spatial dimensions of the input by taking the average value in each pooling window.
BASEDLayer<T> Implements the BASED (Bidirectional Attention with Sliding-window and Expanded features) layer from “Simple Linear Attention Language Models Balance the Recall-Throughput Tradeoff” (Arora et al., 2024).
BartEncoderDecoderLayer<T> The BART encoder-decoder Florence-2 uses as its language model (Lewis et al.
BasicBlock<T> Implements the BasicBlock used in ResNet18 and ResNet34 architectures.
BatchEnsembleLayer<T> Implements a BatchEnsemble layer that provides parameter-efficient ensembling.
BatchNormalizationLayer<T> Implements batch normalization for neural networks, which normalizes the inputs across a mini-batch.
BertEmbeddingLayer<T> Implements BERT’s parallel word, learned-position, and token-type embedding front-end.
BiaffineSpanScorerLayer<T> Biaffine span scorer per Yu et al., ACL 2020, “Named Entity Recognition as Dependency Parsing” (arXiv:2005.07150).
BiasFreeLinearLayer<T> Trainable linear projection y = xW with no bias term.
BidirectionalClozeNetworkLayer<T> ABINet’s Bidirectional Cloze Network (Fang et al., “Read Like Humans”, CVPR 2021, Sec.
BidirectionalLayer<T> Represents a bidirectional layer that processes input sequences in both forward and backward directions.
BidirectionalRecurrentLayer<T> A bidirectional recurrent layer whose forward and backward outputs are concatenated: [time, 2 · hidden], the convention of PyTorch’s bidirectional=True LSTM and GRU.
BilinearGLUFeedForwardLayer<T>
BottleneckBlock<T> Implements the BottleneckBlock used in ResNet50, ResNet101, and ResNet152 architectures.
BranchformerBlock<T> A Branchformer encoder block per Peng et al., ICML 2022, “Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding” (arXiv:2207.02971).
CapsuleLayer<T> Represents a capsule neural network layer that encapsulates groups of neurons to better preserve spatial information.
CbhgLayer<T> Tacotron’s CBHG module: a 1-D Convolution Bank, Highway network and bidirectional GRU.
CifAlignmentLayer<T> Continuous Integrate-and-Fire (CIF) alignment layer per Gao et al.
CitrinetBlockLayer<T> A single NVIDIA Citrinet residual mega-block over [B, C, T] feature data (Majumdar et al., 2021, “Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for ASR”, https://arxiv.org/abs/2104.01721).
ClipVisionTransformerLayer<T> CLIP’s vision transformer (Radford et al.
ClozeAttentionLayer<T> Bidirectional Cloze Network (BCN) attention per Fang et al., CVPR 2021, “Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition” (arXiv:2103.06495).
CohereDecoderBlock<T> Cohere (Command-R) decoder block with a parallel residual: a single LayerNorm feeds both the attention and the gated-SwiGLU FFN, whose outputs are added together to the residual — x = x + Attn(norm(x)) + FFN(norm(x)).
ConcatenateLayer<T> Represents a neural network layer that concatenates multiple inputs along a specified axis.
ConditionalLayerNormalizationLayer<T> Layer normalization whose scale and bias are computed from a conditioning vector instead of being free parameters: CLN(x, e) = (e W_γ) ⊙ (x − μ) / σ + e W_β.
ConditionalRandomFieldLayer<T> Represents a Conditional Random Field (CRF) layer for sequence labeling tasks.
ConformerBlockLayer<T> A Conformer block — the convolution-augmented transformer block of Gulati et al.
ConstantScaleLayer<T> Multiplies its input by a fixed (non-trainable) scalar.
ContextNetBlockLayer<T> A paper-faithful ContextNet convolution block over channels-first [B, C, T] features.
ContinuumMemorySystemLayer<T> Continuum Memory System (CMS) layer for neural networks.
Conv1DLayer<T> 1D convolutional layer for sequence / waveform data, with optional dilation.
Conv1DTransposeLayer<T> 1D transposed convolution (“deconvolution”) for sequence / waveform data — the learnable temporal-upsampling primitive used by HiFi-GAN (Kong et al.
Conv2DLayer<T> A 2-D convolution with explicit input and output channels, kernel, stride and zero padding per axis and an optional bias — PyTorch’s nn.Conv2d, with its weight layout [out, in, kernelHeight, kernelWidth].
Conv3DLayer<T> Represents a 3D convolutional layer for processing volumetric data like voxel grids.
Conv3DTransposeLayer<T> 3D transposed convolution (“up-convolution”) for volumetric data — the learnable upsampling primitive the 3D U-Net decoder is built from (Cicek et al.
ConvBatchNormStackLayer<T> A stack of 1-D convolutions, each followed by batch normalization, an activation and dropout: Tacotron 2’s encoder convolutions and post-net.
ConvLSTMLayer<T> Implements a Convolutional Long Short-Term Memory (ConvLSTM) layer for processing sequential spatial data.
ConvNeXtV2Block<T> A ConvNeXt v2 block: depth-wise convolution, layer norm, point-wise expansion, GELU, Global Response Normalization (GRN), point-wise projection, and a residual connection.
ConvSubsamplingLayer<T> The Conformer’s convolution subsampling front end (Gulati et al.
ConvolutionalLayer<T> Represents a convolutional layer in a neural network that applies filters to input data.
CroppingLayer<T> Represents a cropping layer that removes portions of input tensors from the edges.
CrossAttentionLayer<T> Implements cross-attention for conditioning diffusion models on text or other context.
DaViTLayer<T> DaViT, the dual-attention vision transformer that Florence-2 uses as its image encoder (Ding et al.
DbrxDecoderBlock<T> DBRX pre-LN decoder block: y = x + Attn(LayerNorm(x)) then z = y + MoE(LayerNorm(y)).
DecoderLayer<T> Represents a Decoder Layer in a Transformer architecture.
DeconvolutionalLayer<T> Represents a deconvolutional layer (also known as transposed convolution) in a neural network.
DeformableConvolutionalLayer<T> Deformable Convolutional Layer that learns spatial sampling offsets.
DeltaFormerLayer<T> Implements the DeltaFormer layer from “An Associative Memory Perspective on Transformers and DeltaNet” (Li and Papailiopoulos, 2025, arXiv:2505.19488).
DeltaNetLayer<T> Implements the DeltaNet layer from “Linear Transformers with Learnable Kernel Functions” (Yang et al., 2024).
DeltaProductLayer<T> Implements the DeltaProduct layer from “DeltaProduct: Increasing the Expressivity of DeltaNet Through Products of Householders” (Siems et al., 2025).
DenseBlockLayer<T> A single layer within a DenseBlock: BN-ReLU-Conv1x1-BN-ReLU-Conv3x3.
DenseBlock<T> Implements a Dense Block from the DenseNet architecture.
DenseLayer<T> Represents a fully connected (dense) layer in a neural network.
DepthwiseConv1DLayer<T> A depthwise 1-D convolution over [B, C, T] feature/waveform data: each input channel is convolved with its own temporal filter (no cross-channel mixing), producing C * multiplier output channels.
DepthwiseSeparableConvolutionalLayer<T> Represents a depthwise separable convolutional layer that performs convolution as two separate operations.
DiffusionConvLayer<T> Implements diffusion convolution for mesh surface processing using the heat diffusion equation.
DigitCapsuleLayer<T> Represents a digit capsule layer that implements the dynamic routing algorithm between capsules.
DilatedConvolutionalLayer<T> Represents a dilated convolutional layer for neural networks that applies filters with gaps between filter elements.
DilatedResidualConvBlock<T> SpeedySpeech’s residual block: n repetitions of a dilated 1D convolution, ReLU and temporal batch normalization, with a residual connection around them: y = x + f(x).
DirectionalGraphLayer<T> Implements Directional Graph Networks for directed graph processing with separate in/out aggregations.
DocFormerEncoderLayer<T> DocFormer’s multi-modal self-attention encoder (reference DocFormerEncoder + MultiModalAttentionLayer).
DocFormerSpatialEmbeddingLayer<T> DocFormer’s spatial embedding of one stream (Appalaraju et al., ICCV 2021; reference DocFormerEmbeddings).
DropoutLayer<T> Implements a dropout layer for neural networks to prevent overfitting.
DuelingCombinationLayer<T> Dueling DQN combination head (Wang et al.
EdgeConditionalConvolutionalLayer<T> Implements Edge-Conditioned Convolution for incorporating edge features in graph convolutions.
EmbeddingLayer<T> Represents an embedding layer that converts discrete token indices into dense vector representations.
ExpertLayer<T> Represents an expert module in a Mixture-of-Experts architecture, containing a sequence of layers.
ExtendedLSTMLayer<T> Implements the Extended LSTM (xLSTM) layer from Hochreiter et al., 2024.
FastTextEmbeddingLayer<T> Reusable FastText word/subword embedding front end.
FeatureTokenizerLayer<T> Feature tokenizer for tabular transformers: embeds each scalar input feature into its OWN learnable embedding vector, producing a [features, embedding] token sequence that a transformer encoder can attend over.
FeatureTransformerLayer<T> Implements the Feature Transformer block used in TabNet architecture.
FeedForwardLayer<T> Represents a fully connected (dense) feed-forward layer in a neural network.
FeedForwardTransformerBlock<T> The feed-forward Transformer (FFT) block of FastSpeech: multi-head self-attention followed by a two-layer 1D convolutional network, each with a residual connection, dropout and post-layer normalization.
FlashAttentionLayer<T> A multi-head attention layer using the Flash Attention algorithm for memory-efficient computation.
FlattenLayer<T> Represents a flatten layer that reshapes multi-dimensional input data into a 1D vector.
Florence2ProjectorLayer<T> Florence-2’s visual projector (HF Florence2MultiModalProjector).
FrozenBatchNormLayer<T> Batch normalization fixed at its running statistics: y = (x − μ) / √(σ² + ε) · γ + β per channel, with μ, σ², γ and β all held constant — torchvision’s FrozenBatchNorm2d, for a pretrained network used as a fixed feature extractor.
FullyConnectedLayer<T> Represents a fully connected layer in a neural network where every input neuron connects to every output neuron.
GRUCellLayer<T> One step of a gated recurrent unit: h' = GRU(x, h), for recurrent decoders that run a step at a time.
GRULayer<T> Represents a Gated Recurrent Unit (GRU) layer for processing sequential data.
GandalfGFLULayer<T> GANDALF feature backbone: a stack of Gated Feature Learning Units (GFLUs), the defining component of GANDALF (Joseph & Raj 2022, “GANDALF: Gated Adaptive Network for Deep Automated Learning of Features”).
GatedConvolutionBlockLayer<T> Deep Voice 3’s convolution block: dropout, a 1-D convolution to twice the channels, a gated linear unit, and an optional residual connection scaled by √0.5.
GatedDeltaNetLayer<T> Implements the GatedDeltaNet layer from NVIDIA, ICLR 2025.
GatedDeltaProductLayer<T> Implements the Gated DeltaProduct layer from “DeltaProduct: Increasing the Expressivity of DeltaNet Through Products of Householders” (Siems et al., 2025).
GatedFeatureLearningUnitLayer<T> Gated Feature Learning Unit (GFLU) for GANDALF architecture.
GatedFusionLayer<T> Gated fusion of two equally-wide feature streams, as used by ABINet’s fusion stage.
GatedLinearAttentionLayer<T> Implements the Gated Linear Attention (GLA) layer from Yang et al., 2023.
GatedLinearUnitLayer<T> Represents a Gated Linear Unit (GLU) layer in a neural network that combines linear transformation with multiplicative gating.
GatedSlotAttentionLayer<T> Implements the Gated Slot Attention (GSA) layer from Li et al., 2024.
GaussianMixtureClassifierLayer<T> A Gaussian mixture classifier: one diagonal-covariance Gaussian mixture per class, emitting each class’s log-likelihood log p(x | class).
GaussianNoiseLayer<T> A neural network layer that adds random Gaussian noise to inputs during training.
GeGLUFeedForwardLayer<T>
Gemma2DecoderBlock<T> Gemma-2 decoder block with sandwiched RMSNorms — a norm both before and after each sublayer: x = x + postAttnNorm(Attn(inputNorm(x))) then x = x + postFfnNorm(GeGLU(preFfnNorm(x))).
GlobalPoolingLayer<T> Represents a global pooling layer that reduces spatial dimensions to a single value per channel.
GotOcr2ProjectorLayer<T> GOT-OCR2’s vision-to-language projector (HF GotOcr2MultiModalProjector).
GraFPrintCoordinateAugmentationLayer<T> Builds the three-channel GraFPrint peak-extractor input from a single-channel spectrogram.
GraFPrintFrequencyStrideLayer<T> Applies GraFPrint’s frequency-only peak-extractor stride while preserving every time frame.
GraFPrintGraphBlockLayer<T> Implements one GraFPrint Grapher + residual feed-forward block.
GraphAttentionLayer<T> Implements Graph Attention Network (GAT) layer for processing graph-structured data with attention mechanisms.
GraphConvolutionalLayer<T> Represents a Graph Convolutional Network (GCN) layer for processing graph-structured data.
GraphIsomorphismLayer<T> Implements Graph Isomorphism Network (GIN) layer for powerful graph representation learning.
GraphSAGELayer<T> Implements GraphSAGE (Graph Sample and Aggregate) layer for inductive learning on graphs.
GraphTransformerLayer<T> Implements Graph Transformer layer using self-attention mechanisms on graph-structured data.
GroupNormalizationLayer<T> Represents a Group Normalization layer that normalizes inputs across groups of channels.
GroupedConv1DLayer<T> A grouped 1-D convolution (PyTorch Conv1d(groups = g)): the channels are split into g groups and each output group sees only its input group.
GroupedInvertibleConvFlowLayer<T> Glow-TTS’s invertible 1×1 convolution with channel grouping: the channels are split into groups of nSplit and the same nSplit × nSplit matrix mixes each group.
GroupedQueryAttentionLayer<T> Implements Grouped-Query Attention (GQA) from Ainslie et al., 2023.
HGRN2Layer<T> Implements the HGRN2 layer from “HGRN2: Gated Linear RNNs with State Expansion” (Qin et al., 2024).
HGRNLayer<T> Implements the Hierarchically Gated Recurrent Neural Network (HGRN) layer from NeurIPS 2023.
HedgehogLayer<T> Implements the Hedgehog layer from “The Hedgehog and the Porcupine: Expressive Linear Attentions with Softmax Mimicry” (Zhang et al., 2024, ICLR 2024).
HeterogeneousGraphLayer<T> Implements Heterogeneous Graph Neural Network layer for graphs with multiple node and edge types.
HeterogeneousGraphMetadata Represents metadata for heterogeneous graphs with multiple node and edge types.
HiFiGANResBlockLayer<T> HiFi-GAN Multi-Receptive Field (MRF) fusion module (Kong et al.
HighwayLayer<T> Represents a Highway Neural Network layer that allows information to flow unchanged through the network.
HippoMemoryCellLayer<T> Implements the recurrent memory cell from the original HiPPO paper.
HouseholderSubspaceLayer<T> Matrix-free orthogonal projection y = x Q (or y = x Qᵀ) parameterized as a product of k Householder reflections Q = H₀ H₁ ⋯ H_{k−1}, where each Hⱼ = I − (2 / (‖uⱼ‖₂² + ε)) uⱼ uⱼᵀ uses only O(k · D) parameters and 4 k · M · D F…
HtsatFusionPatchEmbeddingLayer<T> HTS-AT’s patch embedding with LAION-CLAP’s 2-D feature fusion (Wu et al.
HtsatSpectrogramImageLayer<T> HTS-AT’s reshape_wav2img (Chen et al.
HybridBlockScheduler<T> Implements a composable hybrid block that schedules SSM and attention layers according to configurable patterns used in modern hybrid architectures (Jamba, Zamba, Samba).
HyenaLayer<T> Implements the Hyena layer from “Hyena Hierarchy: Towards Larger Convolutional Language Models” (Poli et al., 2023, arXiv:2302.10866).
HyperbolicLinearLayer<T> Represents a fully connected layer operating in hyperbolic (Poincare ball) space.
InputLayer<T> Represents an input layer that passes input data through unchanged to the next layer in the neural network.
InstanceNormalizationLayer<T> Represents an Instance Normalization layer that normalizes each channel independently across spatial dimensions.
InteractingLayer<T> Interacting Layer for AutoInt architecture.
InternImageBlockLayer<T> Implements one pre-normalized InternImage block with a deformable spatial mixer and a GELU feed-forward network, each protected by a residual path.
IntersampleAttentionLayer<T> Intersample (Row) Attention for SAINT architecture.
InvertedResidualBlock<T> Implements an Inverted Residual Block (MBConv) used in MobileNetV2 and MobileNetV3.
KairosMultiSizePatchLayer<T> Kairos Mixture-of-Size patch embedder: emits N parallel patch-size paths through the SAME transformer backbone shape (numPatches varies per path but hiddenDim is fixed), then combines them via a learned router that weights each path per-inp…
KimiLinearAttentionLayer<T> Implements the Kimi KDA (Key-Value Driven Gated Linear Attention) layer from the “Kimi-VL Technical Report” (Kimi Team, 2025, arXiv:2510.26692).
KosmosImageResamplerLayer<T> Turns image tokens into a fixed number of image embeddings for the language model.
LSTMCellLayer<T> One step of a long short-term memory cell, (h', c') = LSTM(x, h, c), for recurrent decoders that run a step at a time.
LSTMLayer<T> Represents a Long Short-Term Memory (LSTM) layer for processing sequential data.
LambdaLayer<T> Represents a customizable layer that applies user-defined functions for both forward and backward passes.
LayerBase<T> Represents the base class for all neural network layers, providing common functionality and interfaces.
LayerCloning Cloning for layers, built on the construction state layers already record for serialization.
LayerGraphContract Declares a non-sequential graph boundary inside an iterator-based layer factory.
LayerNormalizationLayer<T> Represents a Layer Normalization layer that normalizes inputs across the feature dimension.
LayerPort Declares a named input or output port on a layer.
LayoutEmbeddingLayer<T> Paper-faithful text + 2D-layout embedding front-end for layout-aware document models (Xu et al., KDD 2020, “LayoutLM: Pre-training of Text and Layout for Document Image Understanding”, §3.2 “Model Architecture”).
LearnableLogitScaleLayer<T> Applies exp(clamp(logScale, 0, log(100))) to similarity logits.
LearnedPositionalEmbeddingLayer<T> Adds a LEARNED per-position vector to a sequence of feature vectors — the position embedding used by BERT, ViT, BEiT and everything descended from them.
LearnedTokenTypeEmbeddingLayer<T> Adds a learned BERT token-type embedding to [S,C] or [B,S,C] states.
LiltLayoutEmbeddingLayer<T> LiLT’s layout-flow embedding (Wang et al., ACL 2022; reference LiltLayoutEmbeddings).
LinearRecurrentUnitLayer<T> Implements the Linear Recurrent Unit (LRU) layer from Orvieto et al., 2023.
LocallyConnectedLayer<T> Represents a Locally Connected layer which applies different filters to different regions of the input, unlike a convolutional layer which shares filters.
LocationSensitiveAttentionLayer<T> Location-sensitive attention (Chorowski et al.
LogLinearAttentionLayer<T> Implements the Log-Linear Attention layer from Zhang et al., 2025.
LogVarianceLayer<T> Represents a layer that computes the logarithm of variance along a specified axis in the input tensor.
LogitSoftcapLayer<T> Applies logit soft-capping — y = cap · tanh(x / cap) — elementwise, bounding values to (-cap, cap).
LonghornLayer<T> Implements the Longhorn layer from “Longhorn: State Space Models are Amortized Online Learners” (Liu et al., 2024).
MEGALayer<T> Implements the MEGA (Moving Average Equipped Gated Attention) layer from Ma et al., 2023.
MLPMixerBlockLayer<T> A single MLP-Mixer block: temporal-axis MLP + channel-axis MLP, each wrapped with pre-norm and residual, per Tolstikhin et al.
MagnetoDecoderLayer<T> KOSMOS’s MAGNETO causal language model (reference Kosmos2TextTransformer + tied lm_head).
Mamba2Block<T> Implements the Mamba-2 block using the State Space Duality (SSD) framework from Dao and Gu, 2024.
MambaBlock<T> Implements a general-purpose Mamba block (Selective State Space Model) from Gu and Dao, 2023.
MaskingLayer<T> Represents a layer that masks specified values in the input tensor, typically used to ignore padding in sequential data.
MaxPool3DLayer<T> Represents a 3D max pooling layer for downsampling volumetric data.
MaxPoolingLayer<T> Implements a max pooling layer for neural networks, which reduces the spatial dimensions of the input by taking the maximum value in each pooling window.
MeanLayer<T> Represents a layer that computes the mean (average) of input values along a specified axis.
MeasurementLayer<T> Represents a layer that performs quantum measurement operations on complex-valued input tensors.
MegalodonLayer<T> Implements the Megalodon layer from “Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length” (Ma et al., 2024, arXiv:2404.08801).
MemoryReadLayer<T> Represents a layer that reads from a memory tensor using an attention mechanism.
MemoryWriteLayer<T> Represents a layer that writes to a memory tensor using an attention mechanism.
MesaNetLayer<T> Implements the MesaNet layer from Grazzi et al., 2025.
MeshEdgeConvLayer<T> Implements edge convolution for mesh-based neural networks (MeshCNN style).
MeshPoolLayer<T> Implements mesh pooling via edge collapse for MeshCNN-style networks.
MessagePassingLayer<T> Implements a general Message Passing Neural Network (MPNN) layer.
MinGRULayer<T> Implements the minGRU layer from “Were RNNs All We Needed?” (Feng et al., 2024).
MinLSTMLayer<T> Implements the minLSTM layer from “Were RNNs All We Needed?” (Feng et al., 2024).
MixtureOfExpertsBuilder<T> A builder class that helps create and configure Mixture-of-Experts layers with sensible defaults.
MixtureOfExpertsLayer<T> Implements a Mixture-of-Experts (MoE) layer that routes inputs through multiple expert networks.
MixtureOfMambaLayer<T> Implements the Mixture-of-Mamba layer from Jiang et al., 2025 (arXiv:2501.16295).
MixtureOfMemoriesLayer<T> Implements the Mixture of Memories (MoM) layer from Chou et al., 2025.
MoEDecoderBlock<T> Pre-LN decoder block whose feed-forward sublayer is a sparse mixture-of-experts (Mixtral-style): y = x + Attn(RMSNorm(x)), then z = y + MoE(RMSNorm(y)).
MoEFeedForwardLayer<T> Sparse mixture-of-experts feed-forward sublayer (Mixtral / sparse-MoE): a linear router selects the top-k of E gated-SwiGLU experts per token and mixes their outputs by the renormalized router weights — `y = Σ_{e∈topk(x)} softmax(router…
MultiHeadAttentionLayer<T> Implements a multi-head attention layer for neural networks, a key component in transformer architectures.
MultiLatentAttentionLayer<T> Implements the Multi-Latent Attention (MLA) layer from DeepSeek-V2 (Aixin Liu et al., 2024).
MultimodalEditMapperLayer<T> Maps joint image/instruction hidden states to a fixed-length diffusion context using learned queries and an encoder-decoder transformer with true cross-attention.
MultiplyLayer<T> Represents a layer that performs element-wise multiplication of multiple input tensors.
NodeEnsembleLayer<T> NODE ensemble: a set of differentiable oblivious decision trees run in PARALLEL on the same input, with their outputs concatenated (Popov et al.
NoisyDenseLayer<T> Noisy linear layer for exploration in reinforcement learning (Fortunato et al.
NormedConv1DLayer<T> A 1-D convolution (or transposed convolution) with weight or spectral normalization, stride, dilation and groups — PyTorch’s weight_norm(Conv1d(...)) / spectral_norm(Conv1d(...)) as GAN vocoders use them.
NormedConv2DLayer<T> A 2-D convolution with optional weight or spectral normalization, stride, padding and depthwise grouping — PyTorch’s Conv2d(...), weight_norm(Conv2d(...)) and spectral_norm(Conv2d(...)) as GAN encoders and discriminators use them.
ObliviousDecisionTreeLayer<T> Oblivious Decision Tree (ODT) for NODE architecture.
OccupancyNetworkDecoder<T> Implements the decoder of an Occupancy Network (Mescheder et al., CVPR 2019, arXiv:1812.03828): the fully-connected, conditional-normalization ResNet that maps a queried 3D point p ∈ ℝ³ to an occupancy probability, conditioned on a latent…
OctonionLinearLayer<T> Represents a fully connected layer using octonion-valued weights and inputs.
PANNsPoolingLayer<T> Implements CNN14’s channel-preserving frequency/time aggregation for NCHW features.
PReLULayer<T> Implements a Parametric ReLU (PReLU) layer with learnable negative-slope coefficients.
PaTHAttentionLayer<T> Implements the PaTH Attention (Positional-aware Transformer via Householder) layer from Mao et al., 2025 (arXiv:2505.16381).
PaddingLayer<T> Represents a layer that adds padding to the input tensor.
ParallelStreamsLayer<T> Splits input along the feature axis into two equal halves, processes each half through its own independent sub-network (stream), and concatenates the two stream outputs.
PatchEmbeddingLayer<T> Implements a patch embedding layer for Vision Transformer (ViT) architecture.
PatchGANDiscriminator<T> PatchGAN: the convolutional “Markovian” discriminator introduced by pix2pix (Isola et al., 2017).
PickGcnLayer<T> One graph-convolution step of PICK’s GLCN (Yu et al., ICPR 2020; reference GCNLayer) over node-edge-node triplets: H_ij = ReLU(x_i W_vi + x_j W_vj + alpha_ij + b), x_i' = ReLU((sum_j A_ij H_ij) W_node), `alpha_ij’ = ReLU(H_ij W_alpha)…
PickGraphLayer<T> PICK’s graph learning-convolutional network (reference GLCN): it learns a soft adjacency over the document’s text segments from their embeddings, then runs numLayers``PickGcnLayer steps seeded by an embedding of the six box-relation fea…
PiecewiseLinearEncodingLayer<T> Piecewise linear encoding (PLE) of numerical features, from Gorishniy, Rubachev and Babenko, “On Embeddings for Numerical Features in Tabular Deep Learning” (NeurIPS 2022).
PixelShuffleLayer<T> Pixel shuffle (sub-pixel convolution) layer for efficient spatial upsampling.
PoolingLayer<T> Represents a layer that performs pooling operations on input tensors.
PortShapeConstraint A small, decidable shape language shared by generated contracts, runtime validation and test synthesis.
PositionalEncodingLayer<T> Represents a layer that adds positional encodings to input sequences.
PostNormTransformerDecoderBlock<T> The decoder layer of the original Transformer (Vaswani et al.
PreLNTransformerBlock<T> Pre-Layer-Normalization transformer block with RMSNorm and a caller-supplied self-attention sublayer.
PrependCLSTokenLayer<T> Prepends a learnable [CLS] token to a sequence-of-embeddings input, as introduced by BERT (Devlin et al.
PrimaryCapsuleLayer<T> Represents a primary capsule layer for capsule networks.
PrincipalNeighbourhoodAggregationLayer<T> Implements Principal Neighbourhood Aggregation (PNA) layer for powerful graph representation learning.
PrototypeAlignmentLayer<T> A learned prototype-alignment layer per Sun et al.
PyramidSpanLayer<T> The Pyramid nested-NER decoder (Wang et al.
QuantumLayer<T> Represents a neural network layer that uses quantum computing principles for processing inputs.
QwenDecoderLayer<T> A Qwen-family causal decoder (Bai et al.
RBFLayer<T> Represents a Radial Basis Function (RBF) layer for neural networks.
RBMLayer<T> Represents a Restricted Boltzmann Machine (RBM) layer for neural networks.
RMSNormalizationLayer<T> Root-Mean-Square Layer Normalization (Zhang & Sennrich 2019).
RRDBLayer<T> Residual in Residual Dense Block (RRDB) - the core building block of ESRGAN and Real-ESRGAN generators.
RRDBNetGenerator<T> RRDBNet Generator - the full generator architecture from ESRGAN and Real-ESRGAN.
RWKV7Block<T> Implements a single RWKV-7 “Goose” block with the WKV-7 kernel featuring dynamic state evolution.
RWKVLayer<T> Implements the RWKV (Receptance Weighted Key Value) layer, a linear attention RNN from Peng et al., 2024.
ReGLUFeedForwardLayer<T>
ReadoutLayer<T> Represents a readout layer that performs the final mapping from features to output in a neural network.
RealGatedLinearRecurrenceLayer<T> Implements the Real-Gated Linear Recurrence Unit (RG-LRU) from Google DeepMind’s Griffin architecture.
RebasedLayer<T> Implements the ReBased linear attention layer from “Linearizing Large Language Models” (Bick et al., 2024).
ReconstructionLayer<T> Represents a reconstruction layer that uses multiple fully connected layers to transform inputs into outputs.
RecurrentLayer<T> Represents a recurrent neural network layer that processes sequential data by maintaining a hidden state.
RelativePositionTransformerBlock<T> A post-normalization Transformer encoder layer with relative position representations (Shaw et al.
RepParameterizationLayer<T> Represents a reparameterization layer used in variational autoencoders (VAEs) to enable backpropagation through random sampling.
RepViTBlockLayer<T> Implements the token mixer and channel mixer from a RepViT block.
ReservoirLayer<T> Represents a reservoir layer used in Echo State Networks (ESNs) for processing sequential data with fixed random weights.
ReshapeLayer<T> Represents a reshape layer that transforms the dimensions of input data without changing its content.
ResidualConvReluNormLayer<T> A residual stack of Conv1D → LayerNorm → ReLU → Dropout layers with a zero-initialized 1×1 output projection: y = x + P(f(x)), which starts as the identity.
ResidualDenseBlock<T> Residual Dense Block (RDB) as used in ESRGAN and Real-ESRGAN generators.
ResidualLayer<T> Represents a residual layer that adds the identity mapping (input) to the output of an inner layer.
RetNetLayer<T> Implements the RetNet (Retentive Network) layer from Sun et al., 2023.
RobertaEncoderLayer<T> The RoBERTa text encoder (Liu et al.
RodimusLayer<T> Implements the Rodimus layer from “Rodimus: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions” (He et al., 2025).
RotaryPositionalEncodingLayer<T> Implements Rotary Position Embedding (RoPE) as described by Su et al., 2021.
Rwkv7Stack<T> A stack of RWKV7Block layers that threads the RWKV-7 value residual between them.
S4DLayer<T> Implements a Diagonal State Space (S4D) layer from Gu et al., 2022.
S5Layer<T> Implements the Simplified State Space (S5) layer from Smith et al., 2023.
S6Scan<T> Provides reusable S6 (Selective Structured State Space Sequence) scan operations for Mamba-family architectures.
SSMQuantizationHelper<T> Provides SSM-specific quantization utilities for reducing memory and accelerating inference.
SSMStateCache<T> Manages hidden state caching across autoregressive inference steps for SSM models.
STCConnectorLayer<T> Implements the Spatial-Temporal Convolution (STC) connector from VideoLLaMA 2 (Cheng et al.
SVTRMixingBlockLayer<T> SVTR local/global component-mixing block with the released model’s pre-normalization and MLP.
SVTRThinPlateSplineLayer<T> SVTR’s trainable STN_ON rectifier: a six-block localization CNN predicts control points and a thin-plate-spline grid resamples the source image to 32x100.
SamViTDetEncoderLayer<T> The SAM ViTDet image encoder that GOT-OCR2 uses (Kirillov et al.
ScaledPositionalEncodingLayer<T> Adds sinusoidal positional encodings scaled by a trainable weight: x_i + α · PE(i).
SelfAttentionLayer<T> Represents a self-attention layer that allows a sequence to attend to itself, capturing relationships between elements.
SeparableConvolutionalLayer<T> Represents a separable convolutional layer that decomposes standard convolution into depthwise and pointwise operations.
SequenceLastLayer<T> A layer that extracts the last timestep from a sequence.
SequenceTokenSliceLayer<T> Collapses a transformer encoder’s [batch, seq, dim] hidden states down to [batch, dim] by selecting a single position (last, first, or a fixed middle index).
SnakeLayer<T> The Snake activation (Liu, Hartwig and Ueda 2020): f(x) = x + (1/α) sin²(αx) with a trainable frequency α per channel — a periodic nonlinearity BigVGAN uses to give its generator a periodic inductive bias.
SoftTreeLayer<T> A differentiable soft decision tree layer for GANDALF and similar architectures.
SpanRepresentationScorerLayer<T> Scores every span (start, end) of a sentence against every entity category from a span representation: the endpoint representation of PURE or the max-pooled representation of SpERT, each with a learned width embedding.
SparseLinearLayer<T> Represents a fully connected layer with sparse weight matrix for efficient computation.
SpatialPoolerLayer<T> Represents a spatial pooler layer inspired by hierarchical temporal memory (HTM) principles.
SpatialTransformerLayer<T> Represents a spatial transformer layer that enables spatial manipulations of data via a learnable transformation.
SpectralNormalizationLayer<T> Represents a spectral normalization layer that normalizes the weights of a layer by their spectral norm.
SpectrogramStripeDropLayer<T> SpecAugment-style stripe dropping on a spectrogram during training (Park et al.
SpikingLayer<T> Represents a layer of spiking neurons that model the biological dynamics of neural activity.
SpikingNetworkCore<T> A surrogate-gradient spiking-neural-network core: a stack of Leaky Integrate-and-Fire (LIF) hidden layers feeding a non-spiking leaky-integrator readout, unrolled over timeSteps and trained end-to-end by backpropagation-through-time with…
SpiralConvLayer<T> Implements spiral convolution for mesh vertex processing.
SplitLayer<T> Represents a layer that splits the input tensor along a specific dimension into multiple equal parts.
SpyNetLayer<T> SPyNet (Spatial Pyramid Network) layer for optical flow estimation.
SqueezeAndExcitationLayer<T> Represents a Squeeze-and-Excitation layer that recalibrates channel-wise feature responses adaptively.
StarCoder2DecoderBlock<T> StarCoder2 pre-LN decoder block: y = x + Attn(LayerNorm(x)) then z = y + MLP(LayerNorm(y)), where the norms are true (mean-centered) LayerNorms with bias, the attention projections carry biases, and the FFN is a non-gated two-matrix G…
StopGradientLayer<T> Passes its input through unchanged in the forward direction while blocking gradient flow backwards — the layer form of a stop-gradient / detach barrier.
SubpixelConvolutionalLayer<T> Represents a subpixel convolutional layer that performs convolution followed by pixel shuffling for upsampling.
SwiGLUFeedForwardLayer<T>
SwinPatchEmbeddingLayer<T> Patch embedding layer for Swin Transformer that converts images to patch sequences.
SwinPatchMergingLayer<T> Patch merging layer for Swin Transformer that performs downsampling between stages.
SwinTransformerBlockLayer<T> Swin Transformer block layer with windowed multi-head self-attention.
SynapticPlasticityLayer<T> Represents a synaptic plasticity layer that models biological learning mechanisms through spike-timing-dependent plasticity.
T5EncoderStack<T> A T5 encoder stack whose blocks share one relative position bias table (Raffel et al.
T5FeedForwardLayer<T> T5 feed-forward block (DenseReluDense): bias-free wo(relu(wi(x))).
T5RelativeBiasAttentionLayer<T> T5-style multi-head self-attention with learned relative position bias (Raffel et al., “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer”, JMLR 2020).
TTTLayer<T> Implements the TTT (Test-Time Training) layer from Sun et al., 2024.
TabMEnsembleLayer<T> TabM ensemble MLP (Gorishniy et al.
TabNetEncoderLayer<T> TabNet encoder (Arik & Pfister 2019): a sequential-attention block that produces an aggregated decision representation through several decision steps, each of which selects a sparse subset of features via a learnable mask.
TemporalConv3DLayer<T> A trainable anisotropic 3D convolution for video tensors in NCFHW layout.
TemporalFrameSplicingLayer<T> Concatenates adjacent temporal frames along the feature axis.
TemporalMemoryLayer<T> Represents a temporal memory layer that models sequence learning through hierarchical temporal memory concepts.
TemporalProcessorModule<T> Temporal Processor Module (TPM) from Stream-DiffVSR (Shiu et al., 2025): aligns and fuses the current frame’s features with the previous frame’s warped features to enforce temporal coherence.
TemporalShiftLayer<T> Temporal Shift Module: moves a fraction of the channels one step forward in time and another fraction one step backward, leaving the rest in place.
TiedEmbeddingHeadLayer<T> A language-model output head that shares its weights with the network’s token embedding: the logits are h · Eᵀ, where E is the embedding table [vocabulary, hidden].
TiedEmbeddingLayer<T> A token embedding whose table also projects hidden states back to token logits (weight tying, Press and Wolf 2017), as T5’s shared input/output embedding does.
TimeDistributedLayer<T> Represents a wrapper layer that applies an inner layer to each time step of a sequence independently.
TimeEmbeddingLayer<T> Represents a time embedding layer that encodes timesteps using sinusoidal embeddings for diffusion models.
TimeMoEBlockLayer<T> A single Time-MoE transformer block: multi-head self-attention + Mixture-of-Experts FFN, each wrapped in pre-norm + residual.
TimeSformerBlockLayer<T> TimeSformer encoder block with divided space-time attention.
TokenConditionedDecoderLayer<T> An autoregressive encoder-decoder whose encoder consumes token IDs.
TokenDurationPredictorLayer<T> Predicts one log-duration per token with same-padded temporal convolutions.
TorchLstmCellLayer<T> A PyTorch-compatible LSTM step retaining independent input and recurrent biases.
TransNormerLLMLayer<T> Implements the TransNormerLLM layer from “TransNormerLLM: A Faster and Better LLM” (Qin et al., 2023).
TransformerDecoderBlock<T> Pre-Layer-Normalization transformer decoder block — self-attention, a second (“cross”) attention, and a position-wise feed-forward network, each wrapped in a residual (skip) connection with layer normalization applied BEFORE the sublayer.
TransformerDecoderLayer<T> Represents a transformer decoder layer that processes sequences using self-attention, cross-attention, and feed-forward networks.
TransformerEncoderBlock<T> Transformer encoder block — multi-head self-attention and a position-wise feed-forward network, each wrapped in a residual (skip) connection with layer normalization applied before the sublayer (Pre-LN, the default) or after the residual su…
TransformerEncoderLayer<T> Represents a transformer encoder layer that processes sequences using self-attention and feed-forward networks.
TransitionLayer<T> Implements a Transition Layer from the DenseNet architecture.
TransposeLayer<T> Reorders the axes of the input tensor according to a fixed permutation.
TriaffineSpanLayer<T> The Triaffine nested-NER classifier (Yuan et al.
UNetConvBlock<T> Convolutional block for U-Net encoder with optional downsampling.
UNetDiscriminator<T> U-Net Discriminator as used in Real-ESRGAN for improved perceptual quality.
UNetUpBlock<T> Upsampling block for U-Net decoder with skip connection concatenation.
UdopTransformerLayer<T> UDOP’s unified vision-text-layout transformer (Tang et al.
Upsample3DLayer<T> Represents a 3D upsampling layer that increases the spatial dimensions of volumetric data using nearest-neighbor interpolation.
UpsamplingLayer<T> Represents an upsampling layer that increases the spatial dimensions of input tensors using nearest-neighbor interpolation.
VGGishAudioEmbedding<T> VGGish audio embedding: the AudioSet convolutional network that turns a log-mel patch into a compact per-segment embedding.
VGGishMelSpectrogramDefaults The log-mel front-end settings the VGGish embedding was trained against.
VarianceAdaptation<T> What the variance adaptor produced for one utterance.
VarianceAdaptorLayer<T> The variance adaptor of FastSpeech 2: duration prediction and length regulation, then pitch and energy prediction, each added to the expanded sequence as a quantized embedding.
VariancePredictorLayer<T> The variance predictor of FastSpeech 2: a 2-layer 1D convolutional network, each layer followed by ReLU, layer normalization and dropout, then a linear projection.
VarianceTargets<T> Ground-truth variance for one utterance (FastSpeech 2 teacher forcing).
ViTCoMerSegmentationLayer<T> Paper-structured ViT-CoMer encoder and dense-prediction head with parallel ViT/CNN branches, multi-receptive-field refinement, and bidirectional cross-branch interaction.
VideoGigaGANGeneratorLayer<T> Native VideoGigaGAN generator with a style-modulated spatial backbone, temporal inflation, anti-aliased flow-guided propagation, and a high-frequency shuttle.
VisionEncoderDecoderLayer<T> An autoregressive encoder-decoder whose encoder consumes continuous features.
VocosGeneratorLayer<T> The native Vocos generator: an isotropic ConvNeXt backbone followed by a differentiable complex-STFT head and inverse STFT synthesis.
WaveNetResidualBlockLayer<T> A single WaveNet / Parallel WaveGAN residual block (van den Oord et al.
WeightNormConv1DLayer<T> A 1-D convolution whose kernel is weight-normalized: w = g · v / ‖v‖, with the norm taken per output channel.
WhisperConvStemLayer<T> Whisper’s audio-encoder stem (Radford et al.
WordCharEmbeddingLayer<T> Paper-faithful word + character embedding front-end for sequence-labeling NER models (Lample et al., NAACL 2016, “Neural Architectures for Named Entity Recognition”, §3).
WordPairGridLayer<T> W2NER’s word-pair relation classifier (Li et al.

Base Classes (30)

Type Summary
AutoIntBase<T> Base class for AutoInt (Automatic Feature Interaction Learning).
CreditRuleBase<T> Public, extensible base class for credit-assignment (learning) rules in the Direct-Feedback-Alignment family.
DeclaredModelLayoutBase<T> Shared constructor surface for model families whose layouts are declared on a measured base.
DualEncoderArchitecture<T> Modality-agnostic base for two-encoder neural network architectures.
FTTransformerBase<T> Base implementation of FT-Transformer (Feature Tokenizer + Transformer) for tabular data.
GANDALFBase<T> Base implementation of GANDALF (Gated Additive Neural Decision Forest).
GraphModelLayoutBase<T>
ImageClassifierModelLayoutBase<T>
ImageGeneratorModelLayoutBase<T>
ImageTranslationModelLayoutBase<T>
MambularBase<T> Base class for Mambular (State Space Models for Tabular Data).
MultimodalModelLayoutBase<T>
NODEBase<T> Base class for NODE (Neural Oblivious Decision Ensembles).
NeuralNetworkBase<T> Base class for all neural network implementations in AiDotNet.
NeuralSyntheticTabularGeneratorBase<T> Shared caller-facing shape contract for neural synthetic-tabular generators.
SAINTBase<T> Base class for SAINT (Self-Attention and Intersample Attention Transformer).
SequenceModelLayoutBase<T>
SequentialVectorModelLayoutBase<T> Base class for vector models whose declared Layers collection is the complete, literal inference graph in execution order.
SyntheticTabularGeneratorBase<T> Abstract base class for synthetic tabular data generators, providing common infrastructure for fitting models on real data and generating synthetic rows.
TabDPTBase<T> Base class for TabDPT (Tabular Data Pre-Training) foundation model.
TabMBase<T> Base implementation of TabM, a parameter-efficient ensemble model for tabular data.
TabNetBase<T> Base implementation of the TabNet architecture for attentive interpretable tabular learning.
TabPFNBase<T> Base class for TabPFN (Prior-Fitted Networks) for tabular data.
TabRBase<T> Base implementation of TabR, a retrieval-augmented model for tabular data.
TabTransformerBase<T> Base implementation of TabTransformer for tabular data.
TabularNeuralNetworkBase<T> Shared public shape law for architecture-driven tabular neural networks.
TextEmbeddingModelLayoutBase<T>
TokenLanguageModelLayoutBase<T>
VectorModelLayoutBase<T>
VolumetricModelLayoutBase<T>

Interfaces (2)

Type Summary
IInputDomainProvider Supplies the complete executable behavior for a user-defined tensor value domain.
IInvertibleFlowStep<T> One invertible step of a normalizing flow over a [batch, channels, time] sequence.

Enums (19)

Type Summary
BatchNormDataLayout How BatchNormalizationLayer should interpret a rank-3 input.
CbhgVariant Which reference implementation’s CBHG conventions a CbhgLayer follows.
ColumnDataType Specifies the data type of a column in a tabular dataset.
ConvolutionNormalization How a NormedConv1DLayer reparameterizes its kernel.
FlashAttentionPrecision Precision modes for Flash Attention computation.
GLUGateType
HybridSchedulePattern Defines the scheduling pattern for hybrid SSM/attention architectures.
InputContractReadiness Whether a bound input contract can safely execute.
KosmosPositionEncoding How a MagnetoDecoderLayer encodes token positions.
LayerInputDomainCompatibility The result of proving whether one value domain can feed another.
LayerInputDomainKind The kind of values a layer’s input tensor is allowed to hold.
Position<T> Which sequence position the slice selects.
RecurrentCellType The recurrent cell a BidirectionalRecurrentLayer runs in each direction.
ShapeRelationKind How a layer’s output shape is determined by its input shape, declared per LAYER TYPE rather than per model instance.
SpanRepresentationKind How a SpanRepresentationScorerLayer represents a span before classifying it.
SpatialTransformerDataFormat
SymbolicShapeMode Strategy for how CompiledModelHost keys the compile cache.
TensorPortRole Semantic purpose of a tensor port, used in generated diagnostics and tooling.
TensorPortSource Where a tensor port obtains its value.

Structs (11)

Type Summary
AxisFinding What discovery concluded about one output axis.
ContractShadowResult How a layer’s declared shape CONTRACT compared to the shape the imperative walk resolved.
Inputs<T> Generated immutable typed input facade.
Inputs<T> Generated immutable typed input facade.
Inputs<T> Generated immutable typed input facade.
LayerInputDomain Describes what a layer will ACCEPT in its input tensor, so callers can generate or validate conforming data without inspecting the layer’s type.
LayerShape A layer’s shape, in which individual axes may be dynamic and may be named.
LayoutMismatch One layer-to-layer disagreement.
ModelInputShapeConstraint Minimum geometry a generic caller must honor when constructing a model input probe.
NamedInputs<T> Generated immutable typed input facade.
ValueDomainMismatch One incompatible value-domain edge.

Delegates (3)

Type Summary
AggregationFunction<T> Defines the aggregation function type for combining messages.
MessageFunction<T> Defines the message function type for message passing neural networks.
UpdateFunction<T> Defines the update function type for updating node features.

Options & Configuration (119)

Type Summary
ACGANOptions Configuration options for the ACGAN neural network.
AttentionNetworkOptions Configuration options for the AttentionNetwork.
AudioVisualCorrespondenceOptions Configuration options for the AudioVisualCorrespondenceNetwork.
AudioVisualEventLocalizationOptions Configuration options for the AudioVisualEventLocalizationNetwork.
AutoencoderOptions Configuration options for the Autoencoder neural network.
BGEOptions Configuration options for the BGE (BAAI General Embedding) model.
BigGANOptions Configuration options for the BigGAN neural network.
Blip2Options Configuration options for the Blip2NeuralNetwork.
BlipOptions Configuration options for the BlipNeuralNetwork.
CapsuleNetworkOptions Configuration options for the CapsuleNetwork.
ClipModelConfig Configuration for a CLIP model variant.
ClipOptions Input and embedding configuration for the ONNX-only ClipNeuralNetwork.
ColBERTOptions Configuration options for the ColBERT model.
ConditionalGANOptions Configuration options for the Conditional GAN model.
ConvolutionalNeuralNetworkOptions Configuration options for the ConvolutionalNeuralNetwork.
CycleGANOptions Configuration options for the CycleGAN neural network.
DCGANOptions Configuration options for the DCGAN (Deep Convolutional GAN) model.
DeepBeliefNetworkOptions Configuration options for the DeepBeliefNetwork.
DeepBoltzmannMachineOptions Configuration options for the DeepBoltzmannMachine.
DeepQNetworkOptions Configuration options for the DeepQNetwork.
DenseNetOptions Configuration options for the DenseNetNetwork.
DifferentiableNeuralComputerOptions Configuration options for the DifferentiableNeuralComputer.
EagleOptions Configuration options for the EagleLanguageModel.
EchoStateNetworkOptions Configuration options for the EchoStateNetwork.
EfficientNetOptions Configuration options for the EfficientNetNetwork.
EmbeddingModelOptions Shared configuration for text embedding and retrieval models (BGE, ColBERT, SGPT, SPLADE, SimCSE, Instructor, Matryoshka, Word2Vec, GloVe, FastText).
ExtremeLearningMachineOptions Configuration options for the ExtremeLearningMachine.
FalconMambaOptions Configuration options for the FalconMambaLanguageModel.
FastTextOptions Configuration options for the FastText model.
FeedForwardNeuralNetworkOptions Configuration options for the FeedForwardNeuralNetwork.
FinchOptions Configuration options for the FinchLanguageModel.
FlamingoOptions Configuration options for the FlamingoNeuralNetwork.
FlashAttentionConfig Configuration options for Flash Attention algorithm.
GLAOptions Configuration options for the GLALanguageModel.
GRUOptions Configuration options for the GRUNeuralNetwork.
GanOptions Shared configuration for generative adversarial networks (DCGAN, WGAN, WGAN-GP, BigGAN, SAGAN, StyleGAN, ProgressiveGAN, InfoGAN, Pix2Pix, CycleGAN).
GatedDeltaNetOptions Configuration options for the GatedDeltaNetLanguageModel.
GenerativeAdversarialNetworkOptions Configuration options for the GenerativeAdversarialNetwork.
GloVeOptions Configuration options for the GloVe model.
Gpt4VisionOptions Configuration options for the Gpt4VisionNeuralNetwork.
GraphAttentionNetworkOptions Configuration options for the GraphAttentionNetwork.
GraphClassificationModelOptions Configuration options for the GraphClassificationModel.
GraphClassificationOptions Configuration options for GraphClassificationModel.
GraphGenerationModelOptions Configuration options for the GraphGenerationModel.
GraphIsomorphismNetworkOptions Configuration options for the GraphIsomorphismNetwork.
GraphNeuralNetworkOptions Configuration options for the GraphNeuralNetwork.
GraphSAGEOptions Configuration options for GraphSAGENetwork.
GriffinOptions Configuration options for the GriffinLanguageModel.
HTMNetworkOptions Configuration options for the HTMNetwork.
HawkOptions Configuration options for the HawkLanguageModel.
HopeNetworkOptions Configuration options for the HopeNetwork.
HopfieldNetworkOptions Configuration options for the HopfieldNetwork.
HyperbolicNeuralNetworkOptions Configuration options for the HyperbolicNeuralNetwork.
ImageBindOptions Configuration options for the ImageBindNeuralNetwork.
InferenceArenaSettings Global opt-in switch for the inference forward-caching allocator (#1661 / Tensors #661).
InferenceFusionSettings Inference-time graph fusions applied by the layers themselves.
InfoGANOptions Configuration options for the InfoGAN neural network.
InstructorEmbeddingOptions Configuration options for the Instructor Embedding model.
JambaOptions Configures the dimensions and attention spacing of the Jamba language model.
LLaVAOptions Configuration options for the LLaVANeuralNetwork.
LSTMOptions Configuration options for the LSTMNeuralNetwork.
LinkPredictionModelOptions Configuration options for the LinkPredictionModel.
LinkPredictionOptions Configuration options for LinkPredictionModel.
LiquidStateMachineOptions Configuration options for the LiquidStateMachine.
Mamba2Options Configures the dimensions, state width, and head count of the Mamba-2 language model.
MambaOptions Configuration options for MambaLanguageModel.
MatryoshkaEmbeddingOptions Configuration options for the Matryoshka Embedding model.
MemoryNetworkOptions Configuration options for the MemoryNetwork.
MobileNetV2Options Configuration options for the MobileNetV2Network.
MobileNetV3Options Configuration options for the MobileNetV3Network.
NEATOptions Configuration options for the NEAT neural network.
NeuralNetworkDefaultOptions Configuration options for the NeuralNetwork.
NeuralTuringMachineOptions Configuration options for the NeuralTuringMachine.
NodeClassificationModelOptions Configuration options for the NodeClassificationModel.
NodeClassificationOptions Configuration options for NodeClassificationModel.
OccupancyNeuralNetworkOptions Configuration options for the OccupancyNeuralNetwork.
OctonionNeuralNetworkOptions Configuration options for the OctonionNeuralNetwork.
Pix2PixOptions Configuration options for the Pix2Pix neural network.
ProgressiveGANOptions Configuration options for the ProgressiveGAN.
QuantumNeuralNetworkOptions Configuration options for the QuantumNeuralNetwork.
RWKV4Options Configuration options for the RWKV4LanguageModel.
RWKV7Options Configuration options for the RWKV7LanguageModel.
RadialBasisFunctionNetworkOptions Configuration options for the RadialBasisFunctionNetwork.
RecurrentGemmaOptions Configuration options for the RecurrentGemmaLanguageModel.
RecurrentNeuralNetworkOptions Configuration options for the RecurrentNeuralNetwork.
ResNetOptions Configuration options for the ResNetNetwork.
ResidualNeuralNetworkOptions Configuration options for the ResidualNeuralNetwork.
RestrictedBoltzmannMachineOptions Configuration options for the RestrictedBoltzmannMachine.
SAGANOptions Configuration options for the SAGAN neural network.
SGPTOptions Configuration options for the SGPT model.
SPLADEOptions Configuration options for the SPLADE model.
SambaOptions Configures the dimensions and attention spacing of the Samba language model.
SamplingOptions Decoding/sampling configuration for autoregressive text generation (#1632 / #95).
SelfOrganizingMapNNOptions Configuration options for the SelfOrganizingMap neural network.
SequenceModelOptions Shared configuration for sequence and language models (Mamba, RWKV, Jamba, Zamba, Griffin, Hawk, xLSTM, RecurrentGemma and relatives).
SiameseNetworkOptions Configuration options for the SiameseNetwork.
SiameseNeuralNetworkOptions Configuration options for the SiameseNeuralNetwork.
SimCSEOptions Configuration options for the SimCSE model.
SparseNeuralNetworkOptions Configuration options for the SparseNeuralNetwork.
SpikingNeuralNetworkOptions Configuration options for the SpikingNeuralNetwork.
StyleGANOptions Configuration options for the StyleGAN neural network.
TransformerEmbeddingOptions Configuration options for the TransformerEmbeddingNetwork.
TransformerOptions Configuration options for the Transformer neural network.
UNet3DOptions Configuration options for the UNet3D.
UnifiedMultimodalNetworkOptions Configuration options for the UnifiedMultimodalNetwork.
VGGOptions Configuration options for the VGGNetwork.
VariationalAutoencoderOptions Configuration options for the VariationalAutoencoder.
VideoCLIPOptions Configuration options for the VideoCLIPNeuralNetwork.
VisionLanguageInputOptions Shared image, text-context, and output dimensions for vision-language models.
VisionLanguageModelOptions Shared native-tower configuration for vision-language and multimodal models (BLIP, BLIP-2, Flamingo, LLaVA, ImageBind, GPT-4 Vision, VideoCLIP and relatives).
VisionMambaOptions Configuration options for the image-classifying VisionMambaModel.
VisionTransformerOptions Configuration options for the VisionTransformer.
VoxelCNNOptions Configuration options for the VoxelCNN.
WGANGPOptions Configuration options for the WGANGP neural network.
WGANOptions Configuration options for the WGAN neural network.
Word2VecOptions Configuration options for the Word2Vec model.
XLSTMOptions Configures the dimensions and default optimizer learning rate of the xLSTM language model.
Zamba2Options Configuration options for the Zamba2LanguageModel.
ZambaOptions Configures the dimensions and attention spacing of the Zamba language model.

Helpers & Utilities (14)

Type Summary
AutoregressiveDecoder<T> Generic autoregressive decode loop (#1632 / #95): the reusable “generate” driver the codebase lacked — GPT4Vision / Blip / Flamingo each hand-rolled this loop.
ClipModelLoader Loads CLIP models from HuggingFace Hub or local directories.
CreditRuleFactory<T> Maps the public CreditRule selector to a concrete ICreditRule instance.
CreditRules Factory methods for the built-in credit-assignment (learning) rules, returning configurable ICreditRule instances for use with AiModelBuilder.ConfigureCreditRule(ICreditRule<T>).
InputContractShapeResolver Adapts a caller’s preferred probe geometry to a declared input constraint.
InputContractTensorFactory Contract-driven tensor creation for generated tests, examples and custom tooling.
InputContractValidator Central value/shape validation used by model and layer entry points.
InputDomainProviderRegistry Process-wide registry for explicit custom input-domain providers.
LayerContractValidator Checks that a chain of layers agrees on tensor LAYOUT — each layer’s declared output feeding the next layer’s declared input — and reports the first disagreement by layer index and role.
LayerGraphBuilder<T> Fluent construction of a LayerGraph: add each layer and say what feeds it.
LayerNode<T> One layer in a LayerGraph, together with the nodes feeding it and any tensor transformation applied on the way in.
SegmentationMetrics<T> Provides metrics for evaluating segmentation and classification tasks.
ShapeInference Computes output shapes from declared axis roles (TensorLayoutAttribute) plus declared axis relations (IShapeContract), without running a forward pass.
ShapeRelationDiscovery Recovers a layer’s symbolic shape relation by RUNNING it on probe shapes and fitting the simplest relation that reproduces every observation.

Exceptions (2)

Type Summary
InputContractBindingException Thrown when configuration cannot produce an executable input contract.
InputContractViolationException Thrown when a supplied tensor violates a ready input contract.