Neural Networks
738 public types in this namespace, organized by kind.
Models & Types (206)
| Type | Summary |
|---|---|
ACGAN<T> |
Represents an Auxiliary Classifier Generative Adversarial Network (AC-GAN), which extends conditional GANs by having the discriminator also predict the class label of the input. |
AIMGenerator<T> |
AIM (Adaptive and Iterative Mechanism) generator for differentially private synthetic data (McKenna, Mullins, Sheldon, Miklau 2022). |
AttentionNetwork<T> |
Represents a neural network that utilizes attention mechanisms for sequence processing. |
AudioTextDualStreamArchitecture<T> |
A neural network architecture for audio + text two-stream models (CLAP-family encoders) that hosts a separate audio encoder and text encoder. |
AudioVisualCorrespondenceNetwork<T> |
Audio-visual correspondence learning network for cross-modal understanding. |
AudioVisualEventLocalizationNetwork<T> |
Neural network for audio-visual event localization - identifying WHEN and WHERE events occur in video by jointly analyzing audio and visual streams with precise temporal boundaries. |
AutoDiffTabGenerator<T> |
AutoDiff-Tab generator that automatically searches over diffusion configurations (timesteps, noise schedules, network architecture) to find optimal settings for tabular data generation. |
AutoIntClassifier<T> |
AutoInt implementation for classification tasks. |
AutoIntNetwork<T> |
AutoInt (Automatic Feature Interaction Learning) neural network for tabular data. |
AutoIntRegression<T> |
AutoInt implementation for regression tasks. |
Autoencoder<T> |
Represents an autoencoder neural network that can compress data into a lower-dimensional representation and reconstruct it. |
BGE<T> |
BGE (BAAI General Embedding) neural network implementation. |
BayesianNetworkSynthGenerator<T> |
PrivBayes synthesizer (Zhang et al., SIGMOD 2014 / TODS 2017): learns a k-degree Bayesian network under differential privacy, releases noisy conditional distributions, and generates synthetic data by ancestral sampling. |
BigGAN<T> |
BigGAN (Brock et al. |
Blip2NeuralNetwork<T> |
BLIP-2 (Bootstrapped Language-Image Pre-training 2) neural network for vision-language tasks. |
BlipNeuralNetwork<T> |
BLIP (Bootstrapped Language-Image Pre-training) neural network for vision-language tasks. |
BlockwiseAnchorLocalErrorCreditRule<T> |
Blockwise anchor local error: decoupled block-local sequence credit assignment that partitions a deep network into contiguous blocks of at most AnchorStride trainable pre-readout layers, places a shared (full or candidate-sampled) tie… |
BoundInputContract |
A static input manifest after instance configuration and concrete shape are resolved. |
CLSToken<T> |
CLS (Classification) Token for transformer-based tabular models. |
CTABGANPlusGenerator<T> |
CTAB-GAN+ generator for high-quality synthetic tabular data with auxiliary classifier discriminator, mixed-type encoder, and information loss. |
CTGANDataSampler<T> |
Handles conditional vector generation and training-by-sampling for CTGAN. |
CTGANGenerator<T> |
Conditional Tabular GAN (CTGAN) for generating realistic synthetic tabular data. |
CapsuleNetwork<T> |
Represents a Capsule Network, a type of neural network that preserves spatial relationships between features. |
CausalGANGenerator<T> |
Causal-GAN generator that learns causal graph structure (directed acyclic graph) and generates synthetic data respecting causal relationships between features. |
ClipNeuralNetwork<T> |
CLIP (Contrastive Language-Image Pre-training) neural network that encodes both text and images into a shared embedding space, enabling cross-modal similarity and zero-shot classification. |
ColBERT<T> |
ColBERT (Contextualized Late Interaction over BERT) neural network implementation. |
ColumnEmbedding<T> |
Column (positional) embedding for tabular transformers like TabTransformer. |
ColumnMetadata |
Describes the metadata for a single column in a tabular dataset, including its name, data type, categories (for categorical columns), and summary statistics. |
ColumnTransformInfo |
Describes how a single original column maps into the transformed representation. |
ConditionalGAN<T> |
Represents a Conditional Generative Adversarial Network (cGAN), which generates data conditioned on additional information such as class labels, attributes, or other contextual data. |
Connection<T> |
Represents a connection between two nodes in a neural network, particularly used in evolving neural networks. |
ContextEncoder<T> |
Context encoder for TabR that processes retrieved neighbors into a context representation. |
ContrastivePretraining<T> |
Contrastive pretraining module for SAINT architecture. |
ConvolutionalNeuralNetwork<T> |
Represents a Convolutional Neural Network (CNN) that processes multi-dimensional data. |
CopulaGANGenerator<T> |
CopulaGAN generator for synthetic tabular data, combining Gaussian copula transformations with the CTGAN training pipeline for improved continuous column modeling. |
CopulaSynthGenerator<T> |
Copula-Based Synthesis generator that models marginal distributions independently and couples them with a Gaussian copula to capture inter-feature dependencies. |
CycleGAN<T> |
Represents a CycleGAN for unpaired image-to-image translation. |
DCGAN<T> |
Represents a Deep Convolutional Generative Adversarial Network (DCGAN), an architecture that uses convolutional and transposed convolutional layers with specific design guidelines for stable training. |
DPCTGANGenerator<T> |
Differentially Private CTGAN (DP-CTGAN) for generating synthetic tabular data with formal (epsilon, delta)-differential privacy guarantees. |
DeepBeliefNetwork<T> |
Represents a Deep Belief Network, a generative graphical model composed of multiple layers of Restricted Boltzmann Machines. |
DeepBoltzmannMachine<T> |
Represents a Deep Boltzmann Machine (DBM), a hierarchical generative model consisting of multiple layers of stochastic neurons. |
DeepQNetwork<T> |
Represents a Deep Q-Network (DQN), a reinforcement learning algorithm that combines Q-learning with deep neural networks. |
DenseNetNetwork<T> |
Implements the DenseNet (Densely Connected Convolutional Network) architecture. |
DifferentiableNeuralComputer<T> |
Represents a Differentiable Neural Computer (DNC), a neural network architecture that combines neural networks with external memory resources. |
DualStreamArchitecture<T> |
A neural network architecture for vision + text two-stream models (CLIP-family encoders: BASIC, DFNCLIP, EVACLIP, LiT, RegionCLIP, etc.) that hosts a separate vision encoder and text encoder. |
EagleLanguageModel<T> |
Implements a full RWKV-5 “Eagle” language model: token embedding + N RWKVLayer blocks + RMS normalization + LM head. |
EchoStateNetwork<T> |
Represents an Echo State Network (ESN), a type of recurrent neural network with a sparsely connected hidden layer called a reservoir. |
EfficientNetNetwork<T> |
Implements the EfficientNet architecture with compound scaling. |
EntmaxAttention<T> |
Entmax sparse attention function for NODE architecture. |
EntmoidActivation<T> |
Entmoid activation function for NODE architecture. |
ExtremeLearningMachine<T> |
Represents an Extreme Learning Machine (ELM), a type of feedforward neural network with a unique training approach. |
FTTransformerClassifier<T> |
FT-Transformer implementation for classification tasks. |
FTTransformerNetwork<T> |
FT-Transformer (Feature Tokenizer + Transformer) neural network for tabular data. |
FTTransformerRegression<T> |
FT-Transformer implementation for regression tasks. |
FalconMambaLanguageModel<T> |
Implements a Falcon Mamba language model: embedding + N MambaBlock blocks + RMS norm + LM head. |
FastText<T> |
FastText neural network implementation, an extension of Word2Vec that considers subword information. |
FeatureTokenizer<T> |
Implements the Feature Tokenizer that converts tabular features into embeddings for FT-Transformer. |
FeedForwardNeuralNetwork<T> |
Represents a Feed-Forward Neural Network (FFNN) for processing data in a forward path. |
FinDiffGenerator<T> |
FinDiff generator for synthesizing realistic financial tabular data using diffusion models with temporal correlation preservation and financial constraint enforcement. |
FinchLanguageModel<T> |
Implements a full RWKV-6 “Finch” language model: token embedding + N RWKVLayer blocks + RMS normalization + LM head. |
FlamingoNeuralNetwork<T> |
Flamingo neural network for in-context visual learning and few-shot tasks. |
GANDALFClassifier<T> |
GANDALF implementation for classification tasks. |
GANDALFNetwork<T> |
GANDALF (Gated Additive Neural Decision Forest) neural network for tabular data. |
GANDALFRegression<T> |
GANDALF implementation for regression tasks. |
GLALanguageModel<T> |
Implements a full GLA (Gated Linear Attention) language model: embedding + N GLA blocks + RMS norm + LM head. |
GOGGLEGenerator<T> |
GOGGLE generator that learns feature dependency structure via a graph neural network combined with a VAE framework for high-quality synthetic tabular data generation. |
GRUNeuralNetwork<T> |
Represents a Gated Recurrent Unit (GRU) Neural Network for processing sequential data. |
GatedDeltaNetLanguageModel<T> |
Implements a full Gated DeltaNet language model: token embedding + N GatedDeltaNetLayer blocks + RMS norm + LM head. |
GaussianDiffusion<T> |
Implements the Gaussian diffusion process for continuous/numerical features in TabDDPM. |
GenerativeAdversarialNetwork<T> |
Represents a Generative Adversarial Network (GAN), a deep learning architecture that consists of two neural networks (a generator and a discriminator) competing against each other in a zero-sum game. |
Genome<T> |
Represents a genome in a neuroevolutionary algorithm, containing a collection of connections between nodes. |
GhostBatchNormalization<T> |
Implements Ghost Batch Normalization, a regularization technique used in TabNet that applies batch normalization to virtual mini-batches within each actual batch. |
GloVe<T> |
GloVe (Global Vectors for Word Representation) neural network implementation. |
Gpt4VisionNeuralNetwork<T> |
GPT-4V-style neural network that combines vision understanding with large language model capabilities. |
GraphAttentionNetwork<T> |
Represents a Graph Attention Network (GAT) that uses attention mechanisms to process graph-structured data. |
GraphClassificationModel<T> |
Implements a complete neural network model for graph classification tasks. |
GraphGenerationModel<T> |
Represents a Graph Generation Model using Variational Autoencoder (VAE) architecture. |
GraphIsomorphismNetwork<T> |
Represents a Graph Isomorphism Network (GIN) for powerful graph representation learning. |
GraphNeuralNetwork<T> |
Represents a Graph Neural Network that can process data represented as graphs. |
GraphSAGENetwork<T> |
Represents a GraphSAGE (Graph Sample and Aggregate) Network for inductive learning on graphs. |
GriffinLanguageModel<T> |
Implements a Griffin language model: embedding + N RGLR blocks with local attention + layer norm + LM head. |
HTMNetwork<T> |
Represents a Hierarchical Temporal Memory (HTM) network, a biologically-inspired sequence learning algorithm. |
HawkLanguageModel<T> |
Implements a Hawk language model: embedding + N pure RGLR blocks + layer norm + LM head. |
HopeNetwork<T> |
Hope architecture - a self-modifying recurrent neural network variant of Titans with unbounded levels of in-context learning. |
HopfieldNetwork<T> |
Represents a Hopfield Network, a recurrent neural network designed for pattern storage and retrieval. |
HyperbolicNeuralNetwork<T> |
Represents a Hyperbolic Neural Network for learning hierarchical representations in Poincare ball space. |
ImageBindNeuralNetwork<T> |
ImageBind neural network for binding multiple modalities (6+) into a shared embedding space. |
InContextLearning<T> |
Helper class for in-context learning in tabular foundation models like TabPFN. |
InfoGAN<T> |
Represents an Information Maximizing Generative Adversarial Network (InfoGAN), which learns disentangled representations in an unsupervised manner by maximizing mutual information between latent codes and generated observations. |
InputContractManifest |
Canonical input schema for a layer or model. |
InputContractVariant |
A named set of mutually compatible input ports. |
InstructorEmbedding<T> |
Instructor/E5 (Instruction-Tuned) embedding model implementation. |
JambaLanguageModel<T> |
Implements a Jamba language model: embedding + HybridBlockScheduler (Mamba + Attention) + RMS norm + LM head. |
LLaVANeuralNetwork<T> |
LLaVA (Large Language and Vision Assistant) neural network for visual instruction following. |
LSTMNeuralNetwork<T> |
Represents a Long Short-Term Memory (LSTM) Neural Network, which is specialized for processing sequential data like text, time series, or audio. |
LayerGraph<T> |
The wiring of a model, expressed once: which layer feeds which, and what happens on the edges between them. |
LayerwiseDistillationCreditRule<T> |
Layerwise distillation as local credit: every hidden layer of the student is trained, with its input detached, to reproduce one hidden state of a frozen teacher model through a learned linear map `P_k : student width → teacher width… |
LinkPredictionModel<T> |
Implements a complete neural network model for link prediction tasks on graphs. |
LiquidStateMachine<T> |
Represents a Liquid State Machine (LSM), a type of reservoir computing neural network. |
Mamba2LanguageModel<T> |
Implements a Mamba-2 language model: token embedding + N Mamba2Blocks + layer normalization + LM head. |
MambaLanguageModel<T> |
Implements a full Mamba language model: token embedding + N MambaBlocks + RMS normalization + LM head. |
MambaModelState<T> |
Per-token (KV-cached) decoding state for a MambaLanguageModel: one MambaStepState per Mamba block, in layer order. |
MambularClassifier<T> |
Mambular implementation for classification tasks. |
MambularNetwork<T> |
Mambular (State Space Model for Tabular Data) neural network. |
MambularRegression<T> |
Mambular implementation for regression tasks. |
MatryoshkaEmbedding<T> |
Matryoshka Representation Learning (MRL) neural network implementation. |
MedGANGenerator<T> |
medGAN — generates synthetic patient records by pairing a pre-trained autoencoder with a generative adversarial network whose generator produces points in the autoencoder’s latent space. |
MemoryNetwork<T> |
Represents a Memory Network, a neural network architecture designed with explicit memory components for improved reasoning and question answering capabilities. |
MeshCNN<T> |
Implements the MeshCNN architecture for processing 3D triangle meshes. |
MisGANGenerator<T> |
MisGAN generator for learning from incomplete data using dual generator/discriminator pairs for both data values and missingness patterns. |
MixtureOfExpertsNeuralNetwork<T> |
Represents a Mixture-of-Experts (MoE) neural network that routes inputs through multiple specialist networks. |
MobileNetV2Network<T> |
Implements the MobileNetV2 architecture for efficient mobile inference. |
MobileNetV3Network<T> |
Implements the MobileNetV3 architecture for efficient mobile inference. |
MultinomialDiffusion<T> |
Implements the multinomial diffusion process for categorical features in TabDDPM. |
NEAT<T> |
NeuroEvolution of Augmenting Topologies (Stanley & Miikkulainen 2002): evolves the structure and the weights of feed-forward networks with historical markings, speciation and explicit fitness sharing. |
NODEClassifier<T> |
NODE implementation for classification tasks. |
NODENetwork<T> |
NODE (Neural Oblivious Decision Ensembles) neural network for tabular data. |
NODERegression<T> |
NODE implementation for regression tasks. |
NeuralNetworkArchitecture<T> |
Defines the structure and configuration of a neural network, including its layers, input/output dimensions, and task-specific properties. |
NeuralNetwork<T> |
A neural network implementation that processes data through multiple layers to make predictions. |
NeuralTuringMachine<T> |
Represents a Neural Turing Machine, which is a neural network architecture that combines a neural network with external memory. |
NodeClassificationModel<T> |
Implements a complete neural network model for node classification tasks on graphs. |
OCTGANGenerator<T> |
OCT-GAN (Kim et al. |
OccupancyNeuralNetwork<T> |
Represents a Neural Network specialized for occupancy detection and prediction in spaces. |
OctonionNeuralNetwork<T> |
Represents an Octonion-valued Neural Network for processing data in 8-dimensional hypercomplex space. |
PATEGANGenerator<T> |
PATE-GAN (Jordon, Yoon and van der Schaar, ICLR 2019): a GAN made differentially private by replacing its discriminator with the PATE mechanism. |
Pix2Pix<T> |
Represents a Pix2Pix GAN for paired image-to-image translation tasks. |
PlanCache |
Disk-backed store for compiled inference plans. |
ProgressiveGAN<T> |
Progressive GAN (ProGAN, Karras et al. |
QuantumNeuralNetwork<T> |
Represents a Quantum Neural Network, which combines quantum computing principles with neural network architecture. |
REaLTabFormerGenerator<T> |
REaLTabFormer generator using GPT-2 style autoregressive transformer for synthetic tabular data generation by treating columns as a sequence of tokens. |
RWKV4LanguageModel<T> |
Implements a full RWKV-4 language model: token embedding + N RWKVLayer blocks + layer normalization + LM head. |
RWKV7LanguageModel<T> |
Implements a full RWKV-7 “Goose” language model: token embedding + N RWKV7Blocks + RMS normalization + LM head. |
RadialBasisFunctionNetwork<T> |
Represents a Radial Basis Function Network, which is a type of neural network that uses radial basis functions as activation functions. |
RecurrentGemmaLanguageModel<T> |
Implements a RecurrentGemma language model: embedding + N RGLR blocks + layer norm + LM head. |
RecurrentNeuralNetwork<T> |
Represents a Recurrent Neural Network, which is a type of neural network designed to process sequential data by maintaining an internal state. |
ResNetNetwork<T> |
Represents a ResNet (Residual Network) neural network architecture for image classification. |
ResidualNeuralNetwork<T> |
Represents a Residual Neural Network, which is a type of neural network that uses skip connections to address the vanishing gradient problem in deep networks. |
RestrictedBoltzmannMachine<T> |
Represents a Restricted Boltzmann Machine, which is a type of neural network that learns probability distributions over its inputs. |
RetrievalContext<T> |
Context returned by the retrieval module. |
RetrievalModule<T> |
Retrieval module for TabR (Retrieval-Augmented Tabular Learning). |
SAGAN<T> |
Self-Attention GAN (SAGAN) — a deep convolutional GAN whose generator maps a 1D latent vector to an image and whose discriminator scores images as real or fake. |
SAINTClassifier<T> |
SAINT implementation for classification tasks. |
SAINTNetwork<T> |
SAINT (Self-Attention and Intersample Attention Transformer) neural network for tabular data. |
SAINTRegression<T> |
SAINT implementation for regression tasks. |
SGPT<T> |
SGPT (Sentence GPT) neural network implementation using decoder-only transformer architectures. |
SMOTENCGenerator<T> |
SMOTE-NC generator that creates synthetic minority samples by interpolating between existing minority samples and their k-nearest neighbors, supporting both continuous and categorical features. |
SPLADE<T> |
SPLADE (Sparse Lexical and Expansion Model) neural network implementation. |
SambaLanguageModel<T> |
Implements a Samba language model: embedding + HybridBlockScheduler (Mamba + sliding window attention) + RMS norm + LM head. |
SampledTiedReadoutLocalErrorCreditRule<T> |
Sampled tied-readout local error: zero-auxiliary-parameter sequence local credit assignment that evaluates each hidden layer’s next-token cross-entropy over a compact candidate subset U ≪ V of the network’s shared readout (all unique… |
SelfOrganizingMap<T> |
Represents a Self-Organizing Map (Kohonen map): an unsupervised neural network that maps high-dimensional input onto a 2-D grid of prototype neurons, preserving input topology. |
SiameseNetwork<T> |
Implements a Siamese Neural Network for comparing pairs of inputs and determining their similarity. |
SiameseNeuralNetwork<T> |
Sentence-BERT (SBERT) style shared sentence-encoder tower: a transformer encoder that maps a tokenized input to a fixed-size embedding (default 768-d, BERT vocab 30522, max length 512) for semantic similarity and retrieval. |
SimCSE<T> |
SimCSE (Simple Contrastive Learning of Sentence Embeddings) neural network implementation. |
SparseNeuralNetwork<T> |
Represents a Sparse Neural Network with efficient sparse weight matrices. |
Sparsemax<T> |
Implements the Sparsemax activation function, which projects input onto the probability simplex with sparse outputs (many exact zeros). |
SpikingNeuralNetwork<T> |
Represents a Spiking Neural Network, which is a type of neural network that more closely models biological neurons with temporal dynamics. |
SpiralNet<T> |
Implements the SpiralNet++ architecture for mesh-based deep learning. |
StyleGAN<T> |
Represents a StyleGAN (Style-Based Generator Architecture for GANs) that generates high-quality images with fine-grained control over image style at different levels. |
SuperNet<T> |
SuperNet implementation for gradient-based neural architecture search (DARTS). |
TVAEGenerator<T> |
Tabular Variational Autoencoder (TVAE) for generating synthetic tabular data. |
TabDDPMGenerator<T> |
TabDDPM (Tabular Denoising Diffusion Probabilistic Model) for generating synthetic tabular data. |
TabDPTClassifier<T> |
TabDPT implementation for classification tasks. |
TabDPTNetwork<T> |
TabDPT (Tabular Data Pre-Training) neural network for tabular data. |
TabDPTRegression<T> |
TabDPT implementation for regression tasks. |
TabFlowGenerator<T> |
TabFlow generator using flow matching with optimal transport conditional paths for high-quality, fast synthetic tabular data generation. |
TabLLMGenGenerator<T> |
TabLLM-Gen generator that uses LLM-style schema-aware tokenization and autoregressive transformers to generate realistic tabular data. |
TabMClassifier<T> |
TabM implementation for classification tasks. |
TabMNetwork<T> |
TabM (Parameter-Efficient Ensemble) neural network for tabular data. |
TabMRegression<T> |
TabM implementation for regression tasks. |
TabNetClassifier<T> |
TabNet implementation for classification tasks. |
TabNetNetwork<T> |
TabNet neural network for interpretable tabular learning. |
TabNetRegression<T> |
TabNet implementation for regression tasks. |
TabPFNClassifier<T> |
TabPFN implementation for classification tasks. |
TabPFNNetwork<T> |
TabPFN (Prior-Fitted Network) neural network for tabular data. |
TabPFNRegression<T> |
TabPFN implementation for regression tasks. |
TabRClassifier<T> |
TabR implementation for classification tasks. |
TabRNetwork<T> |
TabR (Retrieval-Augmented) neural network for tabular data. |
TabRRegression<T> |
TabR implementation for regression tasks. |
TabSynGenerator<T> |
TabSyn generator combining VAE pretraining with latent diffusion for state-of-the-art synthetic tabular data generation. |
TabTransformerClassifier<T> |
TabTransformer implementation for classification tasks. |
TabTransformerGenGenerator<T> |
TabTransformer-Gen generator that uses column-wise contextual embeddings and masked prediction to generate realistic tabular data. |
TabTransformerNetwork<T> |
TabTransformer neural network for tabular data with categorical features. |
TabTransformerRegression<T> |
TabTransformer implementation for regression tasks. |
TableGANGenerator<T> |
TableGAN generator using a DCGAN-style architecture with classification and information loss for high-quality synthetic tabular data generation. |
TabularDataTransformer<T> |
Transforms tabular data using Variational Gaussian Mixture (VGM) mode-specific normalization for continuous columns and one-hot encoding for categorical columns. |
TiedReadoutLocalErrorCreditRule<T> |
Tied-readout local error: a sequence-model form of Local Error Signals (Nøkland & Eidnes, 2019) in which every hidden layer’s local classifier is the network’s own shared readout (for a language model: the final norm and the tied ou… |
TimeGANGenerator<T> |
TimeGAN generator for synthesizing realistic time-series tabular data while preserving temporal dynamics using an embedding-supervisor-adversarial training framework. |
TransformerArchitecture<T> |
Defines the architecture configuration for a Transformer neural network. |
TransformerEmbeddingNetwork<T> |
A customizable Transformer-based embedding network. |
Transformer<T> |
Represents a Transformer neural network architecture, which is particularly effective for sequence-based tasks like natural language processing. |
TripleStreamArchitecture<T> |
A neural network architecture for three-stream models (e.g., perceiver/abstractor/resampler-style generative VLMs and cross-modality fusion encoders) that hosts a vision encoder, an auxiliary stream (perceiver / abstractor / resampler / cro… |
UNet3D<T> |
Represents a 3D U-Net neural network for volumetric semantic segmentation. |
UnifiedMultimodalNetwork<T> |
Unified multimodal network that handles text, images, audio, and video in a single architecture with cross-modal attention and any-to-any generation. |
VGGNetwork<T> |
Represents a VGG (Visual Geometry Group) neural network architecture for image classification. |
VariationalAutoencoder<T> |
Represents a Variational Autoencoder (VAE) neural network architecture, which is used for generating new data similar to the training data and learning compressed representations. |
VideoCLIPNeuralNetwork<T> |
VideoCLIP neural network for video-text alignment and temporal understanding. |
VisionMambaModel<T> |
Vision Mamba (Vim) image classification: the official network with a linear classifier on its class token. |
VisionTransformer<T> |
Implements the Vision Transformer (ViT) architecture for image classification tasks. |
VoxelCNN<T> |
Represents a Voxel-based 3D Convolutional Neural Network for processing volumetric data. |
WGANGP<T> |
Represents a Wasserstein GAN with Gradient Penalty (WGAN-GP), an improved version of WGAN that uses gradient penalty instead of weight clipping to enforce the Lipschitz constraint. |
WGAN<T> |
Represents a Wasserstein Generative Adversarial Network (WGAN), which uses the Wasserstein distance (Earth Mover’s distance) to measure the difference between the generated and real data distributions. |
Word2Vec<T> |
Word2Vec neural network implementation supporting both Skip-Gram and CBOW architectures. |
XLSTMLanguageModel<T> |
Implements a full xLSTM language model: token embedding + N ExtendedLSTMLayer blocks + RMS normalization + LM head. |
Zamba2LanguageModel<T> |
Implements a Zamba2 language model: embedding + HybridBlockScheduler (Mamba2 + shared attention with LoRA) + RMS norm + LM head. |
ZambaLanguageModel<T> |
Implements a Zamba language model: embedding + HybridBlockScheduler (Mamba + shared attention) + RMS norm + LM head. |
Layers (332)
| Type | Summary |
|---|---|
ABCLayer<T> |
Implements the ABC (Attention with Bounded-memory Control) layer from Peng et al., 2022. |
ALiBiPositionalBiasLayer<T> |
Implements Attention with Linear Biases (ALiBi) from Press et al., 2022. |
AcousticConditionEncoderLayer<T> |
AdaSpeech’s acoustic condition encoder: two blocks of Conv1D → ReLU → LayerNorm → Dropout, then either a mean pooling over time (one vector per utterance) or a linear projection per position (one vector per phoneme). |
ActNormFlowLayer<T> |
Activation normalization (Kingma & Dhariwal 2018): a per-channel affine map z = b + e^s ⊙ x whose scale and bias are initialized from the first batch so that its output has zero mean and unit variance per channel. |
ActivationLayer<T> |
A layer that applies an activation function to transform the input data. |
AdaptiveAveragePoolingLayer<T> |
Implements adaptive average pooling that outputs a fixed spatial size regardless of input dimensions. |
AddLayer<T> |
A layer that adds multiple input tensors element-wise and optionally applies an activation function. |
AffineCouplingFlowLayer<T> |
An affine coupling layer with a WaveNet-like transform network (WaveGlow; Glow-TTS §3.3): the first half of the channels passes through and parameterizes a shift and log-scale applied to the second half. |
AnomalyDetectorLayer<T> |
Represents a layer that detects anomalies by comparing predictions with actual inputs. |
AttentionLayer<T> |
Represents an Attention Layer for focusing on relevant parts of input sequences. |
AttentionalFeatureFusionLayer<T> |
Attentional feature fusion, AFF (Dai et al., “Attentional Feature Fusion”, WACV 2021), in its 2-D form: merges two feature maps with a per-position, per-channel soft selection computed from both. |
AttentiveTransformerLayer<T> |
Implements the Attentive Transformer block used in TabNet architecture for feature selection. |
AutoregressiveEncoderDecoderLayer<T> |
Owns the two execution branches of an autoregressive encoder-decoder architecture. |
AveragePoolingLayer<T> |
Implements an average pooling layer for neural networks, which reduces the spatial dimensions of the input by taking the average value in each pooling window. |
BASEDLayer<T> |
Implements the BASED (Bidirectional Attention with Sliding-window and Expanded features) layer from “Simple Linear Attention Language Models Balance the Recall-Throughput Tradeoff” (Arora et al., 2024). |
BartEncoderDecoderLayer<T> |
The BART encoder-decoder Florence-2 uses as its language model (Lewis et al. |
BasicBlock<T> |
Implements the BasicBlock used in ResNet18 and ResNet34 architectures. |
BatchEnsembleLayer<T> |
Implements a BatchEnsemble layer that provides parameter-efficient ensembling. |
BatchNormalizationLayer<T> |
Implements batch normalization for neural networks, which normalizes the inputs across a mini-batch. |
BertEmbeddingLayer<T> |
Implements BERT’s parallel word, learned-position, and token-type embedding front-end. |
BiaffineSpanScorerLayer<T> |
Biaffine span scorer per Yu et al., ACL 2020, “Named Entity Recognition as Dependency Parsing” (arXiv:2005.07150). |
BiasFreeLinearLayer<T> |
Trainable linear projection y = xW with no bias term. |
BidirectionalClozeNetworkLayer<T> |
ABINet’s Bidirectional Cloze Network (Fang et al., “Read Like Humans”, CVPR 2021, Sec. |
BidirectionalLayer<T> |
Represents a bidirectional layer that processes input sequences in both forward and backward directions. |
BidirectionalRecurrentLayer<T> |
A bidirectional recurrent layer whose forward and backward outputs are concatenated: [time, 2 · hidden], the convention of PyTorch’s bidirectional=True LSTM and GRU. |
BilinearGLUFeedForwardLayer<T> |
|
BottleneckBlock<T> |
Implements the BottleneckBlock used in ResNet50, ResNet101, and ResNet152 architectures. |
BranchformerBlock<T> |
A Branchformer encoder block per Peng et al., ICML 2022, “Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding” (arXiv:2207.02971). |
CapsuleLayer<T> |
Represents a capsule neural network layer that encapsulates groups of neurons to better preserve spatial information. |
CbhgLayer<T> |
Tacotron’s CBHG module: a 1-D Convolution Bank, Highway network and bidirectional GRU. |
CifAlignmentLayer<T> |
Continuous Integrate-and-Fire (CIF) alignment layer per Gao et al. |
CitrinetBlockLayer<T> |
A single NVIDIA Citrinet residual mega-block over [B, C, T] feature data (Majumdar et al., 2021, “Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for ASR”, https://arxiv.org/abs/2104.01721). |
ClipVisionTransformerLayer<T> |
CLIP’s vision transformer (Radford et al. |
ClozeAttentionLayer<T> |
Bidirectional Cloze Network (BCN) attention per Fang et al., CVPR 2021, “Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition” (arXiv:2103.06495). |
CohereDecoderBlock<T> |
Cohere (Command-R) decoder block with a parallel residual: a single LayerNorm feeds both the attention and the gated-SwiGLU FFN, whose outputs are added together to the residual — x = x + Attn(norm(x)) + FFN(norm(x)). |
ConcatenateLayer<T> |
Represents a neural network layer that concatenates multiple inputs along a specified axis. |
ConditionalLayerNormalizationLayer<T> |
Layer normalization whose scale and bias are computed from a conditioning vector instead of being free parameters: CLN(x, e) = (e W_γ) ⊙ (x − μ) / σ + e W_β. |
ConditionalRandomFieldLayer<T> |
Represents a Conditional Random Field (CRF) layer for sequence labeling tasks. |
ConformerBlockLayer<T> |
A Conformer block — the convolution-augmented transformer block of Gulati et al. |
ConstantScaleLayer<T> |
Multiplies its input by a fixed (non-trainable) scalar. |
ContextNetBlockLayer<T> |
A paper-faithful ContextNet convolution block over channels-first [B, C, T] features. |
ContinuumMemorySystemLayer<T> |
Continuum Memory System (CMS) layer for neural networks. |
Conv1DLayer<T> |
1D convolutional layer for sequence / waveform data, with optional dilation. |
Conv1DTransposeLayer<T> |
1D transposed convolution (“deconvolution”) for sequence / waveform data — the learnable temporal-upsampling primitive used by HiFi-GAN (Kong et al. |
Conv2DLayer<T> |
A 2-D convolution with explicit input and output channels, kernel, stride and zero padding per axis and an optional bias — PyTorch’s nn.Conv2d, with its weight layout [out, in, kernelHeight, kernelWidth]. |
Conv3DLayer<T> |
Represents a 3D convolutional layer for processing volumetric data like voxel grids. |
Conv3DTransposeLayer<T> |
3D transposed convolution (“up-convolution”) for volumetric data — the learnable upsampling primitive the 3D U-Net decoder is built from (Cicek et al. |
ConvBatchNormStackLayer<T> |
A stack of 1-D convolutions, each followed by batch normalization, an activation and dropout: Tacotron 2’s encoder convolutions and post-net. |
ConvLSTMLayer<T> |
Implements a Convolutional Long Short-Term Memory (ConvLSTM) layer for processing sequential spatial data. |
ConvNeXtV2Block<T> |
A ConvNeXt v2 block: depth-wise convolution, layer norm, point-wise expansion, GELU, Global Response Normalization (GRN), point-wise projection, and a residual connection. |
ConvSubsamplingLayer<T> |
The Conformer’s convolution subsampling front end (Gulati et al. |
ConvolutionalLayer<T> |
Represents a convolutional layer in a neural network that applies filters to input data. |
CroppingLayer<T> |
Represents a cropping layer that removes portions of input tensors from the edges. |
CrossAttentionLayer<T> |
Implements cross-attention for conditioning diffusion models on text or other context. |
DaViTLayer<T> |
DaViT, the dual-attention vision transformer that Florence-2 uses as its image encoder (Ding et al. |
DbrxDecoderBlock<T> |
DBRX pre-LN decoder block: y = x + Attn(LayerNorm(x)) then z = y + MoE(LayerNorm(y)). |
DecoderLayer<T> |
Represents a Decoder Layer in a Transformer architecture. |
DeconvolutionalLayer<T> |
Represents a deconvolutional layer (also known as transposed convolution) in a neural network. |
DeformableConvolutionalLayer<T> |
Deformable Convolutional Layer that learns spatial sampling offsets. |
DeltaFormerLayer<T> |
Implements the DeltaFormer layer from “An Associative Memory Perspective on Transformers and DeltaNet” (Li and Papailiopoulos, 2025, arXiv:2505.19488). |
DeltaNetLayer<T> |
Implements the DeltaNet layer from “Linear Transformers with Learnable Kernel Functions” (Yang et al., 2024). |
DeltaProductLayer<T> |
Implements the DeltaProduct layer from “DeltaProduct: Increasing the Expressivity of DeltaNet Through Products of Householders” (Siems et al., 2025). |
DenseBlockLayer<T> |
A single layer within a DenseBlock: BN-ReLU-Conv1x1-BN-ReLU-Conv3x3. |
DenseBlock<T> |
Implements a Dense Block from the DenseNet architecture. |
DenseLayer<T> |
Represents a fully connected (dense) layer in a neural network. |
DepthwiseConv1DLayer<T> |
A depthwise 1-D convolution over [B, C, T] feature/waveform data: each input channel is convolved with its own temporal filter (no cross-channel mixing), producing C * multiplier output channels. |
DepthwiseSeparableConvolutionalLayer<T> |
Represents a depthwise separable convolutional layer that performs convolution as two separate operations. |
DiffusionConvLayer<T> |
Implements diffusion convolution for mesh surface processing using the heat diffusion equation. |
DigitCapsuleLayer<T> |
Represents a digit capsule layer that implements the dynamic routing algorithm between capsules. |
DilatedConvolutionalLayer<T> |
Represents a dilated convolutional layer for neural networks that applies filters with gaps between filter elements. |
DilatedResidualConvBlock<T> |
SpeedySpeech’s residual block: n repetitions of a dilated 1D convolution, ReLU and temporal batch normalization, with a residual connection around them: y = x + f(x). |
DirectionalGraphLayer<T> |
Implements Directional Graph Networks for directed graph processing with separate in/out aggregations. |
DocFormerEncoderLayer<T> |
DocFormer’s multi-modal self-attention encoder (reference DocFormerEncoder + MultiModalAttentionLayer). |
DocFormerSpatialEmbeddingLayer<T> |
DocFormer’s spatial embedding of one stream (Appalaraju et al., ICCV 2021; reference DocFormerEmbeddings). |
DropoutLayer<T> |
Implements a dropout layer for neural networks to prevent overfitting. |
DuelingCombinationLayer<T> |
Dueling DQN combination head (Wang et al. |
EdgeConditionalConvolutionalLayer<T> |
Implements Edge-Conditioned Convolution for incorporating edge features in graph convolutions. |
EmbeddingLayer<T> |
Represents an embedding layer that converts discrete token indices into dense vector representations. |
ExpertLayer<T> |
Represents an expert module in a Mixture-of-Experts architecture, containing a sequence of layers. |
ExtendedLSTMLayer<T> |
Implements the Extended LSTM (xLSTM) layer from Hochreiter et al., 2024. |
FastTextEmbeddingLayer<T> |
Reusable FastText word/subword embedding front end. |
FeatureTokenizerLayer<T> |
Feature tokenizer for tabular transformers: embeds each scalar input feature into its OWN learnable embedding vector, producing a [features, embedding] token sequence that a transformer encoder can attend over. |
FeatureTransformerLayer<T> |
Implements the Feature Transformer block used in TabNet architecture. |
FeedForwardLayer<T> |
Represents a fully connected (dense) feed-forward layer in a neural network. |
FeedForwardTransformerBlock<T> |
The feed-forward Transformer (FFT) block of FastSpeech: multi-head self-attention followed by a two-layer 1D convolutional network, each with a residual connection, dropout and post-layer normalization. |
FlashAttentionLayer<T> |
A multi-head attention layer using the Flash Attention algorithm for memory-efficient computation. |
FlattenLayer<T> |
Represents a flatten layer that reshapes multi-dimensional input data into a 1D vector. |
Florence2ProjectorLayer<T> |
Florence-2’s visual projector (HF Florence2MultiModalProjector). |
FrozenBatchNormLayer<T> |
Batch normalization fixed at its running statistics: y = (x − μ) / √(σ² + ε) · γ + β per channel, with μ, σ², γ and β all held constant — torchvision’s FrozenBatchNorm2d, for a pretrained network used as a fixed feature extractor. |
FullyConnectedLayer<T> |
Represents a fully connected layer in a neural network where every input neuron connects to every output neuron. |
GRUCellLayer<T> |
One step of a gated recurrent unit: h' = GRU(x, h), for recurrent decoders that run a step at a time. |
GRULayer<T> |
Represents a Gated Recurrent Unit (GRU) layer for processing sequential data. |
GandalfGFLULayer<T> |
GANDALF feature backbone: a stack of Gated Feature Learning Units (GFLUs), the defining component of GANDALF (Joseph & Raj 2022, “GANDALF: Gated Adaptive Network for Deep Automated Learning of Features”). |
GatedConvolutionBlockLayer<T> |
Deep Voice 3’s convolution block: dropout, a 1-D convolution to twice the channels, a gated linear unit, and an optional residual connection scaled by √0.5. |
GatedDeltaNetLayer<T> |
Implements the GatedDeltaNet layer from NVIDIA, ICLR 2025. |
GatedDeltaProductLayer<T> |
Implements the Gated DeltaProduct layer from “DeltaProduct: Increasing the Expressivity of DeltaNet Through Products of Householders” (Siems et al., 2025). |
GatedFeatureLearningUnitLayer<T> |
Gated Feature Learning Unit (GFLU) for GANDALF architecture. |
GatedFusionLayer<T> |
Gated fusion of two equally-wide feature streams, as used by ABINet’s fusion stage. |
GatedLinearAttentionLayer<T> |
Implements the Gated Linear Attention (GLA) layer from Yang et al., 2023. |
GatedLinearUnitLayer<T> |
Represents a Gated Linear Unit (GLU) layer in a neural network that combines linear transformation with multiplicative gating. |
GatedSlotAttentionLayer<T> |
Implements the Gated Slot Attention (GSA) layer from Li et al., 2024. |
GaussianMixtureClassifierLayer<T> |
A Gaussian mixture classifier: one diagonal-covariance Gaussian mixture per class, emitting each class’s log-likelihood log p(x | class). |
GaussianNoiseLayer<T> |
A neural network layer that adds random Gaussian noise to inputs during training. |
GeGLUFeedForwardLayer<T> |
|
Gemma2DecoderBlock<T> |
Gemma-2 decoder block with sandwiched RMSNorms — a norm both before and after each sublayer: x = x + postAttnNorm(Attn(inputNorm(x))) then x = x + postFfnNorm(GeGLU(preFfnNorm(x))). |
GlobalPoolingLayer<T> |
Represents a global pooling layer that reduces spatial dimensions to a single value per channel. |
GotOcr2ProjectorLayer<T> |
GOT-OCR2’s vision-to-language projector (HF GotOcr2MultiModalProjector). |
GraFPrintCoordinateAugmentationLayer<T> |
Builds the three-channel GraFPrint peak-extractor input from a single-channel spectrogram. |
GraFPrintFrequencyStrideLayer<T> |
Applies GraFPrint’s frequency-only peak-extractor stride while preserving every time frame. |
GraFPrintGraphBlockLayer<T> |
Implements one GraFPrint Grapher + residual feed-forward block. |
GraphAttentionLayer<T> |
Implements Graph Attention Network (GAT) layer for processing graph-structured data with attention mechanisms. |
GraphConvolutionalLayer<T> |
Represents a Graph Convolutional Network (GCN) layer for processing graph-structured data. |
GraphIsomorphismLayer<T> |
Implements Graph Isomorphism Network (GIN) layer for powerful graph representation learning. |
GraphSAGELayer<T> |
Implements GraphSAGE (Graph Sample and Aggregate) layer for inductive learning on graphs. |
GraphTransformerLayer<T> |
Implements Graph Transformer layer using self-attention mechanisms on graph-structured data. |
GroupNormalizationLayer<T> |
Represents a Group Normalization layer that normalizes inputs across groups of channels. |
GroupedConv1DLayer<T> |
A grouped 1-D convolution (PyTorch Conv1d(groups = g)): the channels are split into g groups and each output group sees only its input group. |
GroupedInvertibleConvFlowLayer<T> |
Glow-TTS’s invertible 1×1 convolution with channel grouping: the channels are split into groups of nSplit and the same nSplit × nSplit matrix mixes each group. |
GroupedQueryAttentionLayer<T> |
Implements Grouped-Query Attention (GQA) from Ainslie et al., 2023. |
HGRN2Layer<T> |
Implements the HGRN2 layer from “HGRN2: Gated Linear RNNs with State Expansion” (Qin et al., 2024). |
HGRNLayer<T> |
Implements the Hierarchically Gated Recurrent Neural Network (HGRN) layer from NeurIPS 2023. |
HedgehogLayer<T> |
Implements the Hedgehog layer from “The Hedgehog and the Porcupine: Expressive Linear Attentions with Softmax Mimicry” (Zhang et al., 2024, ICLR 2024). |
HeterogeneousGraphLayer<T> |
Implements Heterogeneous Graph Neural Network layer for graphs with multiple node and edge types. |
HeterogeneousGraphMetadata |
Represents metadata for heterogeneous graphs with multiple node and edge types. |
HiFiGANResBlockLayer<T> |
HiFi-GAN Multi-Receptive Field (MRF) fusion module (Kong et al. |
HighwayLayer<T> |
Represents a Highway Neural Network layer that allows information to flow unchanged through the network. |
HippoMemoryCellLayer<T> |
Implements the recurrent memory cell from the original HiPPO paper. |
HouseholderSubspaceLayer<T> |
Matrix-free orthogonal projection y = x Q (or y = x Qᵀ) parameterized as a product of k Householder reflections Q = H₀ H₁ ⋯ H_{k−1}, where each Hⱼ = I − (2 / (‖uⱼ‖₂² + ε)) uⱼ uⱼᵀ uses only O(k · D) parameters and 4 k · M · D F… |
HtsatFusionPatchEmbeddingLayer<T> |
HTS-AT’s patch embedding with LAION-CLAP’s 2-D feature fusion (Wu et al. |
HtsatSpectrogramImageLayer<T> |
HTS-AT’s reshape_wav2img (Chen et al. |
HybridBlockScheduler<T> |
Implements a composable hybrid block that schedules SSM and attention layers according to configurable patterns used in modern hybrid architectures (Jamba, Zamba, Samba). |
HyenaLayer<T> |
Implements the Hyena layer from “Hyena Hierarchy: Towards Larger Convolutional Language Models” (Poli et al., 2023, arXiv:2302.10866). |
HyperbolicLinearLayer<T> |
Represents a fully connected layer operating in hyperbolic (Poincare ball) space. |
InputLayer<T> |
Represents an input layer that passes input data through unchanged to the next layer in the neural network. |
InstanceNormalizationLayer<T> |
Represents an Instance Normalization layer that normalizes each channel independently across spatial dimensions. |
InteractingLayer<T> |
Interacting Layer for AutoInt architecture. |
InternImageBlockLayer<T> |
Implements one pre-normalized InternImage block with a deformable spatial mixer and a GELU feed-forward network, each protected by a residual path. |
IntersampleAttentionLayer<T> |
Intersample (Row) Attention for SAINT architecture. |
InvertedResidualBlock<T> |
Implements an Inverted Residual Block (MBConv) used in MobileNetV2 and MobileNetV3. |
KairosMultiSizePatchLayer<T> |
Kairos Mixture-of-Size patch embedder: emits N parallel patch-size paths through the SAME transformer backbone shape (numPatches varies per path but hiddenDim is fixed), then combines them via a learned router that weights each path per-inp… |
KimiLinearAttentionLayer<T> |
Implements the Kimi KDA (Key-Value Driven Gated Linear Attention) layer from the “Kimi-VL Technical Report” (Kimi Team, 2025, arXiv:2510.26692). |
KosmosImageResamplerLayer<T> |
Turns image tokens into a fixed number of image embeddings for the language model. |
LSTMCellLayer<T> |
One step of a long short-term memory cell, (h', c') = LSTM(x, h, c), for recurrent decoders that run a step at a time. |
LSTMLayer<T> |
Represents a Long Short-Term Memory (LSTM) layer for processing sequential data. |
LambdaLayer<T> |
Represents a customizable layer that applies user-defined functions for both forward and backward passes. |
LayerBase<T> |
Represents the base class for all neural network layers, providing common functionality and interfaces. |
LayerCloning |
Cloning for layers, built on the construction state layers already record for serialization. |
LayerGraphContract |
Declares a non-sequential graph boundary inside an iterator-based layer factory. |
LayerNormalizationLayer<T> |
Represents a Layer Normalization layer that normalizes inputs across the feature dimension. |
LayerPort |
Declares a named input or output port on a layer. |
LayoutEmbeddingLayer<T> |
Paper-faithful text + 2D-layout embedding front-end for layout-aware document models (Xu et al., KDD 2020, “LayoutLM: Pre-training of Text and Layout for Document Image Understanding”, §3.2 “Model Architecture”). |
LearnableLogitScaleLayer<T> |
Applies exp(clamp(logScale, 0, log(100))) to similarity logits. |
LearnedPositionalEmbeddingLayer<T> |
Adds a LEARNED per-position vector to a sequence of feature vectors — the position embedding used by BERT, ViT, BEiT and everything descended from them. |
LearnedTokenTypeEmbeddingLayer<T> |
Adds a learned BERT token-type embedding to [S,C] or [B,S,C] states. |
LiltLayoutEmbeddingLayer<T> |
LiLT’s layout-flow embedding (Wang et al., ACL 2022; reference LiltLayoutEmbeddings). |
LinearRecurrentUnitLayer<T> |
Implements the Linear Recurrent Unit (LRU) layer from Orvieto et al., 2023. |
LocallyConnectedLayer<T> |
Represents a Locally Connected layer which applies different filters to different regions of the input, unlike a convolutional layer which shares filters. |
LocationSensitiveAttentionLayer<T> |
Location-sensitive attention (Chorowski et al. |
LogLinearAttentionLayer<T> |
Implements the Log-Linear Attention layer from Zhang et al., 2025. |
LogVarianceLayer<T> |
Represents a layer that computes the logarithm of variance along a specified axis in the input tensor. |
LogitSoftcapLayer<T> |
Applies logit soft-capping — y = cap · tanh(x / cap) — elementwise, bounding values to (-cap, cap). |
LonghornLayer<T> |
Implements the Longhorn layer from “Longhorn: State Space Models are Amortized Online Learners” (Liu et al., 2024). |
MEGALayer<T> |
Implements the MEGA (Moving Average Equipped Gated Attention) layer from Ma et al., 2023. |
MLPMixerBlockLayer<T> |
A single MLP-Mixer block: temporal-axis MLP + channel-axis MLP, each wrapped with pre-norm and residual, per Tolstikhin et al. |
MagnetoDecoderLayer<T> |
KOSMOS’s MAGNETO causal language model (reference Kosmos2TextTransformer + tied lm_head). |
Mamba2Block<T> |
Implements the Mamba-2 block using the State Space Duality (SSD) framework from Dao and Gu, 2024. |
MambaBlock<T> |
Implements a general-purpose Mamba block (Selective State Space Model) from Gu and Dao, 2023. |
MaskingLayer<T> |
Represents a layer that masks specified values in the input tensor, typically used to ignore padding in sequential data. |
MaxPool3DLayer<T> |
Represents a 3D max pooling layer for downsampling volumetric data. |
MaxPoolingLayer<T> |
Implements a max pooling layer for neural networks, which reduces the spatial dimensions of the input by taking the maximum value in each pooling window. |
MeanLayer<T> |
Represents a layer that computes the mean (average) of input values along a specified axis. |
MeasurementLayer<T> |
Represents a layer that performs quantum measurement operations on complex-valued input tensors. |
MegalodonLayer<T> |
Implements the Megalodon layer from “Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length” (Ma et al., 2024, arXiv:2404.08801). |
MemoryReadLayer<T> |
Represents a layer that reads from a memory tensor using an attention mechanism. |
MemoryWriteLayer<T> |
Represents a layer that writes to a memory tensor using an attention mechanism. |
MesaNetLayer<T> |
Implements the MesaNet layer from Grazzi et al., 2025. |
MeshEdgeConvLayer<T> |
Implements edge convolution for mesh-based neural networks (MeshCNN style). |
MeshPoolLayer<T> |
Implements mesh pooling via edge collapse for MeshCNN-style networks. |
MessagePassingLayer<T> |
Implements a general Message Passing Neural Network (MPNN) layer. |
MinGRULayer<T> |
Implements the minGRU layer from “Were RNNs All We Needed?” (Feng et al., 2024). |
MinLSTMLayer<T> |
Implements the minLSTM layer from “Were RNNs All We Needed?” (Feng et al., 2024). |
MixtureOfExpertsBuilder<T> |
A builder class that helps create and configure Mixture-of-Experts layers with sensible defaults. |
MixtureOfExpertsLayer<T> |
Implements a Mixture-of-Experts (MoE) layer that routes inputs through multiple expert networks. |
MixtureOfMambaLayer<T> |
Implements the Mixture-of-Mamba layer from Jiang et al., 2025 (arXiv:2501.16295). |
MixtureOfMemoriesLayer<T> |
Implements the Mixture of Memories (MoM) layer from Chou et al., 2025. |
MoEDecoderBlock<T> |
Pre-LN decoder block whose feed-forward sublayer is a sparse mixture-of-experts (Mixtral-style): y = x + Attn(RMSNorm(x)), then z = y + MoE(RMSNorm(y)). |
MoEFeedForwardLayer<T> |
Sparse mixture-of-experts feed-forward sublayer (Mixtral / sparse-MoE): a linear router selects the top-k of E gated-SwiGLU experts per token and mixes their outputs by the renormalized router weights — `y = Σ_{e∈topk(x)} softmax(router… |
MultiHeadAttentionLayer<T> |
Implements a multi-head attention layer for neural networks, a key component in transformer architectures. |
MultiLatentAttentionLayer<T> |
Implements the Multi-Latent Attention (MLA) layer from DeepSeek-V2 (Aixin Liu et al., 2024). |
MultimodalEditMapperLayer<T> |
Maps joint image/instruction hidden states to a fixed-length diffusion context using learned queries and an encoder-decoder transformer with true cross-attention. |
MultiplyLayer<T> |
Represents a layer that performs element-wise multiplication of multiple input tensors. |
NodeEnsembleLayer<T> |
NODE ensemble: a set of differentiable oblivious decision trees run in PARALLEL on the same input, with their outputs concatenated (Popov et al. |
NoisyDenseLayer<T> |
Noisy linear layer for exploration in reinforcement learning (Fortunato et al. |
NormedConv1DLayer<T> |
A 1-D convolution (or transposed convolution) with weight or spectral normalization, stride, dilation and groups — PyTorch’s weight_norm(Conv1d(...)) / spectral_norm(Conv1d(...)) as GAN vocoders use them. |
NormedConv2DLayer<T> |
A 2-D convolution with optional weight or spectral normalization, stride, padding and depthwise grouping — PyTorch’s Conv2d(...), weight_norm(Conv2d(...)) and spectral_norm(Conv2d(...)) as GAN encoders and discriminators use them. |
ObliviousDecisionTreeLayer<T> |
Oblivious Decision Tree (ODT) for NODE architecture. |
OccupancyNetworkDecoder<T> |
Implements the decoder of an Occupancy Network (Mescheder et al., CVPR 2019, arXiv:1812.03828): the fully-connected, conditional-normalization ResNet that maps a queried 3D point p ∈ ℝ³ to an occupancy probability, conditioned on a latent… |
OctonionLinearLayer<T> |
Represents a fully connected layer using octonion-valued weights and inputs. |
PANNsPoolingLayer<T> |
Implements CNN14’s channel-preserving frequency/time aggregation for NCHW features. |
PReLULayer<T> |
Implements a Parametric ReLU (PReLU) layer with learnable negative-slope coefficients. |
PaTHAttentionLayer<T> |
Implements the PaTH Attention (Positional-aware Transformer via Householder) layer from Mao et al., 2025 (arXiv:2505.16381). |
PaddingLayer<T> |
Represents a layer that adds padding to the input tensor. |
ParallelStreamsLayer<T> |
Splits input along the feature axis into two equal halves, processes each half through its own independent sub-network (stream), and concatenates the two stream outputs. |
PatchEmbeddingLayer<T> |
Implements a patch embedding layer for Vision Transformer (ViT) architecture. |
PatchGANDiscriminator<T> |
PatchGAN: the convolutional “Markovian” discriminator introduced by pix2pix (Isola et al., 2017). |
PickGcnLayer<T> |
One graph-convolution step of PICK’s GLCN (Yu et al., ICPR 2020; reference GCNLayer) over node-edge-node triplets: H_ij = ReLU(x_i W_vi + x_j W_vj + alpha_ij + b), x_i' = ReLU((sum_j A_ij H_ij) W_node), `alpha_ij’ = ReLU(H_ij W_alpha)… |
PickGraphLayer<T> |
PICK’s graph learning-convolutional network (reference GLCN): it learns a soft adjacency over the document’s text segments from their embeddings, then runs numLayers``PickGcnLayer steps seeded by an embedding of the six box-relation fea… |
PiecewiseLinearEncodingLayer<T> |
Piecewise linear encoding (PLE) of numerical features, from Gorishniy, Rubachev and Babenko, “On Embeddings for Numerical Features in Tabular Deep Learning” (NeurIPS 2022). |
PixelShuffleLayer<T> |
Pixel shuffle (sub-pixel convolution) layer for efficient spatial upsampling. |
PoolingLayer<T> |
Represents a layer that performs pooling operations on input tensors. |
PortShapeConstraint |
A small, decidable shape language shared by generated contracts, runtime validation and test synthesis. |
PositionalEncodingLayer<T> |
Represents a layer that adds positional encodings to input sequences. |
PostNormTransformerDecoderBlock<T> |
The decoder layer of the original Transformer (Vaswani et al. |
PreLNTransformerBlock<T> |
Pre-Layer-Normalization transformer block with RMSNorm and a caller-supplied self-attention sublayer. |
PrependCLSTokenLayer<T> |
Prepends a learnable [CLS] token to a sequence-of-embeddings input, as introduced by BERT (Devlin et al. |
PrimaryCapsuleLayer<T> |
Represents a primary capsule layer for capsule networks. |
PrincipalNeighbourhoodAggregationLayer<T> |
Implements Principal Neighbourhood Aggregation (PNA) layer for powerful graph representation learning. |
PrototypeAlignmentLayer<T> |
A learned prototype-alignment layer per Sun et al. |
PyramidSpanLayer<T> |
The Pyramid nested-NER decoder (Wang et al. |
QuantumLayer<T> |
Represents a neural network layer that uses quantum computing principles for processing inputs. |
QwenDecoderLayer<T> |
A Qwen-family causal decoder (Bai et al. |
RBFLayer<T> |
Represents a Radial Basis Function (RBF) layer for neural networks. |
RBMLayer<T> |
Represents a Restricted Boltzmann Machine (RBM) layer for neural networks. |
RMSNormalizationLayer<T> |
Root-Mean-Square Layer Normalization (Zhang & Sennrich 2019). |
RRDBLayer<T> |
Residual in Residual Dense Block (RRDB) - the core building block of ESRGAN and Real-ESRGAN generators. |
RRDBNetGenerator<T> |
RRDBNet Generator - the full generator architecture from ESRGAN and Real-ESRGAN. |
RWKV7Block<T> |
Implements a single RWKV-7 “Goose” block with the WKV-7 kernel featuring dynamic state evolution. |
RWKVLayer<T> |
Implements the RWKV (Receptance Weighted Key Value) layer, a linear attention RNN from Peng et al., 2024. |
ReGLUFeedForwardLayer<T> |
|
ReadoutLayer<T> |
Represents a readout layer that performs the final mapping from features to output in a neural network. |
RealGatedLinearRecurrenceLayer<T> |
Implements the Real-Gated Linear Recurrence Unit (RG-LRU) from Google DeepMind’s Griffin architecture. |
RebasedLayer<T> |
Implements the ReBased linear attention layer from “Linearizing Large Language Models” (Bick et al., 2024). |
ReconstructionLayer<T> |
Represents a reconstruction layer that uses multiple fully connected layers to transform inputs into outputs. |
RecurrentLayer<T> |
Represents a recurrent neural network layer that processes sequential data by maintaining a hidden state. |
RelativePositionTransformerBlock<T> |
A post-normalization Transformer encoder layer with relative position representations (Shaw et al. |
RepParameterizationLayer<T> |
Represents a reparameterization layer used in variational autoencoders (VAEs) to enable backpropagation through random sampling. |
RepViTBlockLayer<T> |
Implements the token mixer and channel mixer from a RepViT block. |
ReservoirLayer<T> |
Represents a reservoir layer used in Echo State Networks (ESNs) for processing sequential data with fixed random weights. |
ReshapeLayer<T> |
Represents a reshape layer that transforms the dimensions of input data without changing its content. |
ResidualConvReluNormLayer<T> |
A residual stack of Conv1D → LayerNorm → ReLU → Dropout layers with a zero-initialized 1×1 output projection: y = x + P(f(x)), which starts as the identity. |
ResidualDenseBlock<T> |
Residual Dense Block (RDB) as used in ESRGAN and Real-ESRGAN generators. |
ResidualLayer<T> |
Represents a residual layer that adds the identity mapping (input) to the output of an inner layer. |
RetNetLayer<T> |
Implements the RetNet (Retentive Network) layer from Sun et al., 2023. |
RobertaEncoderLayer<T> |
The RoBERTa text encoder (Liu et al. |
RodimusLayer<T> |
Implements the Rodimus layer from “Rodimus: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions” (He et al., 2025). |
RotaryPositionalEncodingLayer<T> |
Implements Rotary Position Embedding (RoPE) as described by Su et al., 2021. |
Rwkv7Stack<T> |
A stack of RWKV7Block layers that threads the RWKV-7 value residual between them. |
S4DLayer<T> |
Implements a Diagonal State Space (S4D) layer from Gu et al., 2022. |
S5Layer<T> |
Implements the Simplified State Space (S5) layer from Smith et al., 2023. |
S6Scan<T> |
Provides reusable S6 (Selective Structured State Space Sequence) scan operations for Mamba-family architectures. |
SSMQuantizationHelper<T> |
Provides SSM-specific quantization utilities for reducing memory and accelerating inference. |
SSMStateCache<T> |
Manages hidden state caching across autoregressive inference steps for SSM models. |
STCConnectorLayer<T> |
Implements the Spatial-Temporal Convolution (STC) connector from VideoLLaMA 2 (Cheng et al. |
SVTRMixingBlockLayer<T> |
SVTR local/global component-mixing block with the released model’s pre-normalization and MLP. |
SVTRThinPlateSplineLayer<T> |
SVTR’s trainable STN_ON rectifier: a six-block localization CNN predicts control points and a thin-plate-spline grid resamples the source image to 32x100. |
SamViTDetEncoderLayer<T> |
The SAM ViTDet image encoder that GOT-OCR2 uses (Kirillov et al. |
ScaledPositionalEncodingLayer<T> |
Adds sinusoidal positional encodings scaled by a trainable weight: x_i + α · PE(i). |
SelfAttentionLayer<T> |
Represents a self-attention layer that allows a sequence to attend to itself, capturing relationships between elements. |
SeparableConvolutionalLayer<T> |
Represents a separable convolutional layer that decomposes standard convolution into depthwise and pointwise operations. |
SequenceLastLayer<T> |
A layer that extracts the last timestep from a sequence. |
SequenceTokenSliceLayer<T> |
Collapses a transformer encoder’s [batch, seq, dim] hidden states down to [batch, dim] by selecting a single position (last, first, or a fixed middle index). |
SnakeLayer<T> |
The Snake activation (Liu, Hartwig and Ueda 2020): f(x) = x + (1/α) sin²(αx) with a trainable frequency α per channel — a periodic nonlinearity BigVGAN uses to give its generator a periodic inductive bias. |
SoftTreeLayer<T> |
A differentiable soft decision tree layer for GANDALF and similar architectures. |
SpanRepresentationScorerLayer<T> |
Scores every span (start, end) of a sentence against every entity category from a span representation: the endpoint representation of PURE or the max-pooled representation of SpERT, each with a learned width embedding. |
SparseLinearLayer<T> |
Represents a fully connected layer with sparse weight matrix for efficient computation. |
SpatialPoolerLayer<T> |
Represents a spatial pooler layer inspired by hierarchical temporal memory (HTM) principles. |
SpatialTransformerLayer<T> |
Represents a spatial transformer layer that enables spatial manipulations of data via a learnable transformation. |
SpectralNormalizationLayer<T> |
Represents a spectral normalization layer that normalizes the weights of a layer by their spectral norm. |
SpectrogramStripeDropLayer<T> |
SpecAugment-style stripe dropping on a spectrogram during training (Park et al. |
SpikingLayer<T> |
Represents a layer of spiking neurons that model the biological dynamics of neural activity. |
SpikingNetworkCore<T> |
A surrogate-gradient spiking-neural-network core: a stack of Leaky Integrate-and-Fire (LIF) hidden layers feeding a non-spiking leaky-integrator readout, unrolled over timeSteps and trained end-to-end by backpropagation-through-time with… |
SpiralConvLayer<T> |
Implements spiral convolution for mesh vertex processing. |
SplitLayer<T> |
Represents a layer that splits the input tensor along a specific dimension into multiple equal parts. |
SpyNetLayer<T> |
SPyNet (Spatial Pyramid Network) layer for optical flow estimation. |
SqueezeAndExcitationLayer<T> |
Represents a Squeeze-and-Excitation layer that recalibrates channel-wise feature responses adaptively. |
StarCoder2DecoderBlock<T> |
StarCoder2 pre-LN decoder block: y = x + Attn(LayerNorm(x)) then z = y + MLP(LayerNorm(y)), where the norms are true (mean-centered) LayerNorms with bias, the attention projections carry biases, and the FFN is a non-gated two-matrix G… |
StopGradientLayer<T> |
Passes its input through unchanged in the forward direction while blocking gradient flow backwards — the layer form of a stop-gradient / detach barrier. |
SubpixelConvolutionalLayer<T> |
Represents a subpixel convolutional layer that performs convolution followed by pixel shuffling for upsampling. |
SwiGLUFeedForwardLayer<T> |
|
SwinPatchEmbeddingLayer<T> |
Patch embedding layer for Swin Transformer that converts images to patch sequences. |
SwinPatchMergingLayer<T> |
Patch merging layer for Swin Transformer that performs downsampling between stages. |
SwinTransformerBlockLayer<T> |
Swin Transformer block layer with windowed multi-head self-attention. |
SynapticPlasticityLayer<T> |
Represents a synaptic plasticity layer that models biological learning mechanisms through spike-timing-dependent plasticity. |
T5EncoderStack<T> |
A T5 encoder stack whose blocks share one relative position bias table (Raffel et al. |
T5FeedForwardLayer<T> |
T5 feed-forward block (DenseReluDense): bias-free wo(relu(wi(x))). |
T5RelativeBiasAttentionLayer<T> |
T5-style multi-head self-attention with learned relative position bias (Raffel et al., “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer”, JMLR 2020). |
TTTLayer<T> |
Implements the TTT (Test-Time Training) layer from Sun et al., 2024. |
TabMEnsembleLayer<T> |
TabM ensemble MLP (Gorishniy et al. |
TabNetEncoderLayer<T> |
TabNet encoder (Arik & Pfister 2019): a sequential-attention block that produces an aggregated decision representation through several decision steps, each of which selects a sparse subset of features via a learnable mask. |
TemporalConv3DLayer<T> |
A trainable anisotropic 3D convolution for video tensors in NCFHW layout. |
TemporalFrameSplicingLayer<T> |
Concatenates adjacent temporal frames along the feature axis. |
TemporalMemoryLayer<T> |
Represents a temporal memory layer that models sequence learning through hierarchical temporal memory concepts. |
TemporalProcessorModule<T> |
Temporal Processor Module (TPM) from Stream-DiffVSR (Shiu et al., 2025): aligns and fuses the current frame’s features with the previous frame’s warped features to enforce temporal coherence. |
TemporalShiftLayer<T> |
Temporal Shift Module: moves a fraction of the channels one step forward in time and another fraction one step backward, leaving the rest in place. |
TiedEmbeddingHeadLayer<T> |
A language-model output head that shares its weights with the network’s token embedding: the logits are h · Eᵀ, where E is the embedding table [vocabulary, hidden]. |
TiedEmbeddingLayer<T> |
A token embedding whose table also projects hidden states back to token logits (weight tying, Press and Wolf 2017), as T5’s shared input/output embedding does. |
TimeDistributedLayer<T> |
Represents a wrapper layer that applies an inner layer to each time step of a sequence independently. |
TimeEmbeddingLayer<T> |
Represents a time embedding layer that encodes timesteps using sinusoidal embeddings for diffusion models. |
TimeMoEBlockLayer<T> |
A single Time-MoE transformer block: multi-head self-attention + Mixture-of-Experts FFN, each wrapped in pre-norm + residual. |
TimeSformerBlockLayer<T> |
TimeSformer encoder block with divided space-time attention. |
TokenConditionedDecoderLayer<T> |
An autoregressive encoder-decoder whose encoder consumes token IDs. |
TokenDurationPredictorLayer<T> |
Predicts one log-duration per token with same-padded temporal convolutions. |
TorchLstmCellLayer<T> |
A PyTorch-compatible LSTM step retaining independent input and recurrent biases. |
TransNormerLLMLayer<T> |
Implements the TransNormerLLM layer from “TransNormerLLM: A Faster and Better LLM” (Qin et al., 2023). |
TransformerDecoderBlock<T> |
Pre-Layer-Normalization transformer decoder block — self-attention, a second (“cross”) attention, and a position-wise feed-forward network, each wrapped in a residual (skip) connection with layer normalization applied BEFORE the sublayer. |
TransformerDecoderLayer<T> |
Represents a transformer decoder layer that processes sequences using self-attention, cross-attention, and feed-forward networks. |
TransformerEncoderBlock<T> |
Transformer encoder block — multi-head self-attention and a position-wise feed-forward network, each wrapped in a residual (skip) connection with layer normalization applied before the sublayer (Pre-LN, the default) or after the residual su… |
TransformerEncoderLayer<T> |
Represents a transformer encoder layer that processes sequences using self-attention and feed-forward networks. |
TransitionLayer<T> |
Implements a Transition Layer from the DenseNet architecture. |
TransposeLayer<T> |
Reorders the axes of the input tensor according to a fixed permutation. |
TriaffineSpanLayer<T> |
The Triaffine nested-NER classifier (Yuan et al. |
UNetConvBlock<T> |
Convolutional block for U-Net encoder with optional downsampling. |
UNetDiscriminator<T> |
U-Net Discriminator as used in Real-ESRGAN for improved perceptual quality. |
UNetUpBlock<T> |
Upsampling block for U-Net decoder with skip connection concatenation. |
UdopTransformerLayer<T> |
UDOP’s unified vision-text-layout transformer (Tang et al. |
Upsample3DLayer<T> |
Represents a 3D upsampling layer that increases the spatial dimensions of volumetric data using nearest-neighbor interpolation. |
UpsamplingLayer<T> |
Represents an upsampling layer that increases the spatial dimensions of input tensors using nearest-neighbor interpolation. |
VGGishAudioEmbedding<T> |
VGGish audio embedding: the AudioSet convolutional network that turns a log-mel patch into a compact per-segment embedding. |
VGGishMelSpectrogramDefaults |
The log-mel front-end settings the VGGish embedding was trained against. |
VarianceAdaptation<T> |
What the variance adaptor produced for one utterance. |
VarianceAdaptorLayer<T> |
The variance adaptor of FastSpeech 2: duration prediction and length regulation, then pitch and energy prediction, each added to the expanded sequence as a quantized embedding. |
VariancePredictorLayer<T> |
The variance predictor of FastSpeech 2: a 2-layer 1D convolutional network, each layer followed by ReLU, layer normalization and dropout, then a linear projection. |
VarianceTargets<T> |
Ground-truth variance for one utterance (FastSpeech 2 teacher forcing). |
ViTCoMerSegmentationLayer<T> |
Paper-structured ViT-CoMer encoder and dense-prediction head with parallel ViT/CNN branches, multi-receptive-field refinement, and bidirectional cross-branch interaction. |
VideoGigaGANGeneratorLayer<T> |
Native VideoGigaGAN generator with a style-modulated spatial backbone, temporal inflation, anti-aliased flow-guided propagation, and a high-frequency shuttle. |
VisionEncoderDecoderLayer<T> |
An autoregressive encoder-decoder whose encoder consumes continuous features. |
VocosGeneratorLayer<T> |
The native Vocos generator: an isotropic ConvNeXt backbone followed by a differentiable complex-STFT head and inverse STFT synthesis. |
WaveNetResidualBlockLayer<T> |
A single WaveNet / Parallel WaveGAN residual block (van den Oord et al. |
WeightNormConv1DLayer<T> |
A 1-D convolution whose kernel is weight-normalized: w = g · v / ‖v‖, with the norm taken per output channel. |
WhisperConvStemLayer<T> |
Whisper’s audio-encoder stem (Radford et al. |
WordCharEmbeddingLayer<T> |
Paper-faithful word + character embedding front-end for sequence-labeling NER models (Lample et al., NAACL 2016, “Neural Architectures for Named Entity Recognition”, §3). |
WordPairGridLayer<T> |
W2NER’s word-pair relation classifier (Li et al. |
Base Classes (30)
| Type | Summary |
|---|---|
AutoIntBase<T> |
Base class for AutoInt (Automatic Feature Interaction Learning). |
CreditRuleBase<T> |
Public, extensible base class for credit-assignment (learning) rules in the Direct-Feedback-Alignment family. |
DeclaredModelLayoutBase<T> |
Shared constructor surface for model families whose layouts are declared on a measured base. |
DualEncoderArchitecture<T> |
Modality-agnostic base for two-encoder neural network architectures. |
FTTransformerBase<T> |
Base implementation of FT-Transformer (Feature Tokenizer + Transformer) for tabular data. |
GANDALFBase<T> |
Base implementation of GANDALF (Gated Additive Neural Decision Forest). |
GraphModelLayoutBase<T> |
|
ImageClassifierModelLayoutBase<T> |
|
ImageGeneratorModelLayoutBase<T> |
|
ImageTranslationModelLayoutBase<T> |
|
MambularBase<T> |
Base class for Mambular (State Space Models for Tabular Data). |
MultimodalModelLayoutBase<T> |
|
NODEBase<T> |
Base class for NODE (Neural Oblivious Decision Ensembles). |
NeuralNetworkBase<T> |
Base class for all neural network implementations in AiDotNet. |
NeuralSyntheticTabularGeneratorBase<T> |
Shared caller-facing shape contract for neural synthetic-tabular generators. |
SAINTBase<T> |
Base class for SAINT (Self-Attention and Intersample Attention Transformer). |
SequenceModelLayoutBase<T> |
|
SequentialVectorModelLayoutBase<T> |
Base class for vector models whose declared Layers collection is the complete, literal inference graph in execution order. |
SyntheticTabularGeneratorBase<T> |
Abstract base class for synthetic tabular data generators, providing common infrastructure for fitting models on real data and generating synthetic rows. |
TabDPTBase<T> |
Base class for TabDPT (Tabular Data Pre-Training) foundation model. |
TabMBase<T> |
Base implementation of TabM, a parameter-efficient ensemble model for tabular data. |
TabNetBase<T> |
Base implementation of the TabNet architecture for attentive interpretable tabular learning. |
TabPFNBase<T> |
Base class for TabPFN (Prior-Fitted Networks) for tabular data. |
TabRBase<T> |
Base implementation of TabR, a retrieval-augmented model for tabular data. |
TabTransformerBase<T> |
Base implementation of TabTransformer for tabular data. |
TabularNeuralNetworkBase<T> |
Shared public shape law for architecture-driven tabular neural networks. |
TextEmbeddingModelLayoutBase<T> |
|
TokenLanguageModelLayoutBase<T> |
|
VectorModelLayoutBase<T> |
|
VolumetricModelLayoutBase<T> |
Interfaces (2)
| Type | Summary |
|---|---|
IInputDomainProvider |
Supplies the complete executable behavior for a user-defined tensor value domain. |
IInvertibleFlowStep<T> |
One invertible step of a normalizing flow over a [batch, channels, time] sequence. |
Enums (19)
| Type | Summary |
|---|---|
BatchNormDataLayout |
How BatchNormalizationLayer should interpret a rank-3 input. |
CbhgVariant |
Which reference implementation’s CBHG conventions a CbhgLayer follows. |
ColumnDataType |
Specifies the data type of a column in a tabular dataset. |
ConvolutionNormalization |
How a NormedConv1DLayer reparameterizes its kernel. |
FlashAttentionPrecision |
Precision modes for Flash Attention computation. |
GLUGateType |
|
HybridSchedulePattern |
Defines the scheduling pattern for hybrid SSM/attention architectures. |
InputContractReadiness |
Whether a bound input contract can safely execute. |
KosmosPositionEncoding |
How a MagnetoDecoderLayer encodes token positions. |
LayerInputDomainCompatibility |
The result of proving whether one value domain can feed another. |
LayerInputDomainKind |
The kind of values a layer’s input tensor is allowed to hold. |
Position<T> |
Which sequence position the slice selects. |
RecurrentCellType |
The recurrent cell a BidirectionalRecurrentLayer runs in each direction. |
ShapeRelationKind |
How a layer’s output shape is determined by its input shape, declared per LAYER TYPE rather than per model instance. |
SpanRepresentationKind |
How a SpanRepresentationScorerLayer represents a span before classifying it. |
SpatialTransformerDataFormat |
|
SymbolicShapeMode |
Strategy for how CompiledModelHost keys the compile cache. |
TensorPortRole |
Semantic purpose of a tensor port, used in generated diagnostics and tooling. |
TensorPortSource |
Where a tensor port obtains its value. |
Structs (11)
| Type | Summary |
|---|---|
AxisFinding |
What discovery concluded about one output axis. |
ContractShadowResult |
How a layer’s declared shape CONTRACT compared to the shape the imperative walk resolved. |
Inputs<T> |
Generated immutable typed input facade. |
Inputs<T> |
Generated immutable typed input facade. |
Inputs<T> |
Generated immutable typed input facade. |
LayerInputDomain |
Describes what a layer will ACCEPT in its input tensor, so callers can generate or validate conforming data without inspecting the layer’s type. |
LayerShape |
A layer’s shape, in which individual axes may be dynamic and may be named. |
LayoutMismatch |
One layer-to-layer disagreement. |
ModelInputShapeConstraint |
Minimum geometry a generic caller must honor when constructing a model input probe. |
NamedInputs<T> |
Generated immutable typed input facade. |
ValueDomainMismatch |
One incompatible value-domain edge. |
Delegates (3)
| Type | Summary |
|---|---|
AggregationFunction<T> |
Defines the aggregation function type for combining messages. |
MessageFunction<T> |
Defines the message function type for message passing neural networks. |
UpdateFunction<T> |
Defines the update function type for updating node features. |
Options & Configuration (119)
| Type | Summary |
|---|---|
ACGANOptions |
Configuration options for the ACGAN neural network. |
AttentionNetworkOptions |
Configuration options for the AttentionNetwork. |
AudioVisualCorrespondenceOptions |
Configuration options for the AudioVisualCorrespondenceNetwork. |
AudioVisualEventLocalizationOptions |
Configuration options for the AudioVisualEventLocalizationNetwork. |
AutoencoderOptions |
Configuration options for the Autoencoder neural network. |
BGEOptions |
Configuration options for the BGE (BAAI General Embedding) model. |
BigGANOptions |
Configuration options for the BigGAN neural network. |
Blip2Options |
Configuration options for the Blip2NeuralNetwork. |
BlipOptions |
Configuration options for the BlipNeuralNetwork. |
CapsuleNetworkOptions |
Configuration options for the CapsuleNetwork. |
ClipModelConfig |
Configuration for a CLIP model variant. |
ClipOptions |
Input and embedding configuration for the ONNX-only ClipNeuralNetwork. |
ColBERTOptions |
Configuration options for the ColBERT model. |
ConditionalGANOptions |
Configuration options for the Conditional GAN model. |
ConvolutionalNeuralNetworkOptions |
Configuration options for the ConvolutionalNeuralNetwork. |
CycleGANOptions |
Configuration options for the CycleGAN neural network. |
DCGANOptions |
Configuration options for the DCGAN (Deep Convolutional GAN) model. |
DeepBeliefNetworkOptions |
Configuration options for the DeepBeliefNetwork. |
DeepBoltzmannMachineOptions |
Configuration options for the DeepBoltzmannMachine. |
DeepQNetworkOptions |
Configuration options for the DeepQNetwork. |
DenseNetOptions |
Configuration options for the DenseNetNetwork. |
DifferentiableNeuralComputerOptions |
Configuration options for the DifferentiableNeuralComputer. |
EagleOptions |
Configuration options for the EagleLanguageModel. |
EchoStateNetworkOptions |
Configuration options for the EchoStateNetwork. |
EfficientNetOptions |
Configuration options for the EfficientNetNetwork. |
EmbeddingModelOptions |
Shared configuration for text embedding and retrieval models (BGE, ColBERT, SGPT, SPLADE, SimCSE, Instructor, Matryoshka, Word2Vec, GloVe, FastText). |
ExtremeLearningMachineOptions |
Configuration options for the ExtremeLearningMachine. |
FalconMambaOptions |
Configuration options for the FalconMambaLanguageModel. |
FastTextOptions |
Configuration options for the FastText model. |
FeedForwardNeuralNetworkOptions |
Configuration options for the FeedForwardNeuralNetwork. |
FinchOptions |
Configuration options for the FinchLanguageModel. |
FlamingoOptions |
Configuration options for the FlamingoNeuralNetwork. |
FlashAttentionConfig |
Configuration options for Flash Attention algorithm. |
GLAOptions |
Configuration options for the GLALanguageModel. |
GRUOptions |
Configuration options for the GRUNeuralNetwork. |
GanOptions |
Shared configuration for generative adversarial networks (DCGAN, WGAN, WGAN-GP, BigGAN, SAGAN, StyleGAN, ProgressiveGAN, InfoGAN, Pix2Pix, CycleGAN). |
GatedDeltaNetOptions |
Configuration options for the GatedDeltaNetLanguageModel. |
GenerativeAdversarialNetworkOptions |
Configuration options for the GenerativeAdversarialNetwork. |
GloVeOptions |
Configuration options for the GloVe model. |
Gpt4VisionOptions |
Configuration options for the Gpt4VisionNeuralNetwork. |
GraphAttentionNetworkOptions |
Configuration options for the GraphAttentionNetwork. |
GraphClassificationModelOptions |
Configuration options for the GraphClassificationModel. |
GraphClassificationOptions |
Configuration options for GraphClassificationModel. |
GraphGenerationModelOptions |
Configuration options for the GraphGenerationModel. |
GraphIsomorphismNetworkOptions |
Configuration options for the GraphIsomorphismNetwork. |
GraphNeuralNetworkOptions |
Configuration options for the GraphNeuralNetwork. |
GraphSAGEOptions |
Configuration options for GraphSAGENetwork. |
GriffinOptions |
Configuration options for the GriffinLanguageModel. |
HTMNetworkOptions |
Configuration options for the HTMNetwork. |
HawkOptions |
Configuration options for the HawkLanguageModel. |
HopeNetworkOptions |
Configuration options for the HopeNetwork. |
HopfieldNetworkOptions |
Configuration options for the HopfieldNetwork. |
HyperbolicNeuralNetworkOptions |
Configuration options for the HyperbolicNeuralNetwork. |
ImageBindOptions |
Configuration options for the ImageBindNeuralNetwork. |
InferenceArenaSettings |
Global opt-in switch for the inference forward-caching allocator (#1661 / Tensors #661). |
InferenceFusionSettings |
Inference-time graph fusions applied by the layers themselves. |
InfoGANOptions |
Configuration options for the InfoGAN neural network. |
InstructorEmbeddingOptions |
Configuration options for the Instructor Embedding model. |
JambaOptions |
Configures the dimensions and attention spacing of the Jamba language model. |
LLaVAOptions |
Configuration options for the LLaVANeuralNetwork. |
LSTMOptions |
Configuration options for the LSTMNeuralNetwork. |
LinkPredictionModelOptions |
Configuration options for the LinkPredictionModel. |
LinkPredictionOptions |
Configuration options for LinkPredictionModel. |
LiquidStateMachineOptions |
Configuration options for the LiquidStateMachine. |
Mamba2Options |
Configures the dimensions, state width, and head count of the Mamba-2 language model. |
MambaOptions |
Configuration options for MambaLanguageModel. |
MatryoshkaEmbeddingOptions |
Configuration options for the Matryoshka Embedding model. |
MemoryNetworkOptions |
Configuration options for the MemoryNetwork. |
MobileNetV2Options |
Configuration options for the MobileNetV2Network. |
MobileNetV3Options |
Configuration options for the MobileNetV3Network. |
NEATOptions |
Configuration options for the NEAT neural network. |
NeuralNetworkDefaultOptions |
Configuration options for the NeuralNetwork. |
NeuralTuringMachineOptions |
Configuration options for the NeuralTuringMachine. |
NodeClassificationModelOptions |
Configuration options for the NodeClassificationModel. |
NodeClassificationOptions |
Configuration options for NodeClassificationModel. |
OccupancyNeuralNetworkOptions |
Configuration options for the OccupancyNeuralNetwork. |
OctonionNeuralNetworkOptions |
Configuration options for the OctonionNeuralNetwork. |
Pix2PixOptions |
Configuration options for the Pix2Pix neural network. |
ProgressiveGANOptions |
Configuration options for the ProgressiveGAN. |
QuantumNeuralNetworkOptions |
Configuration options for the QuantumNeuralNetwork. |
RWKV4Options |
Configuration options for the RWKV4LanguageModel. |
RWKV7Options |
Configuration options for the RWKV7LanguageModel. |
RadialBasisFunctionNetworkOptions |
Configuration options for the RadialBasisFunctionNetwork. |
RecurrentGemmaOptions |
Configuration options for the RecurrentGemmaLanguageModel. |
RecurrentNeuralNetworkOptions |
Configuration options for the RecurrentNeuralNetwork. |
ResNetOptions |
Configuration options for the ResNetNetwork. |
ResidualNeuralNetworkOptions |
Configuration options for the ResidualNeuralNetwork. |
RestrictedBoltzmannMachineOptions |
Configuration options for the RestrictedBoltzmannMachine. |
SAGANOptions |
Configuration options for the SAGAN neural network. |
SGPTOptions |
Configuration options for the SGPT model. |
SPLADEOptions |
Configuration options for the SPLADE model. |
SambaOptions |
Configures the dimensions and attention spacing of the Samba language model. |
SamplingOptions |
Decoding/sampling configuration for autoregressive text generation (#1632 / #95). |
SelfOrganizingMapNNOptions |
Configuration options for the SelfOrganizingMap neural network. |
SequenceModelOptions |
Shared configuration for sequence and language models (Mamba, RWKV, Jamba, Zamba, Griffin, Hawk, xLSTM, RecurrentGemma and relatives). |
SiameseNetworkOptions |
Configuration options for the SiameseNetwork. |
SiameseNeuralNetworkOptions |
Configuration options for the SiameseNeuralNetwork. |
SimCSEOptions |
Configuration options for the SimCSE model. |
SparseNeuralNetworkOptions |
Configuration options for the SparseNeuralNetwork. |
SpikingNeuralNetworkOptions |
Configuration options for the SpikingNeuralNetwork. |
StyleGANOptions |
Configuration options for the StyleGAN neural network. |
TransformerEmbeddingOptions |
Configuration options for the TransformerEmbeddingNetwork. |
TransformerOptions |
Configuration options for the Transformer neural network. |
UNet3DOptions |
Configuration options for the UNet3D. |
UnifiedMultimodalNetworkOptions |
Configuration options for the UnifiedMultimodalNetwork. |
VGGOptions |
Configuration options for the VGGNetwork. |
VariationalAutoencoderOptions |
Configuration options for the VariationalAutoencoder. |
VideoCLIPOptions |
Configuration options for the VideoCLIPNeuralNetwork. |
VisionLanguageInputOptions |
Shared image, text-context, and output dimensions for vision-language models. |
VisionLanguageModelOptions |
Shared native-tower configuration for vision-language and multimodal models (BLIP, BLIP-2, Flamingo, LLaVA, ImageBind, GPT-4 Vision, VideoCLIP and relatives). |
VisionMambaOptions |
Configuration options for the image-classifying VisionMambaModel. |
VisionTransformerOptions |
Configuration options for the VisionTransformer. |
VoxelCNNOptions |
Configuration options for the VoxelCNN. |
WGANGPOptions |
Configuration options for the WGANGP neural network. |
WGANOptions |
Configuration options for the WGAN neural network. |
Word2VecOptions |
Configuration options for the Word2Vec model. |
XLSTMOptions |
Configures the dimensions and default optimizer learning rate of the xLSTM language model. |
Zamba2Options |
Configuration options for the Zamba2LanguageModel. |
ZambaOptions |
Configures the dimensions and attention spacing of the Zamba language model. |
Helpers & Utilities (14)
| Type | Summary |
|---|---|
AutoregressiveDecoder<T> |
Generic autoregressive decode loop (#1632 / #95): the reusable “generate” driver the codebase lacked — GPT4Vision / Blip / Flamingo each hand-rolled this loop. |
ClipModelLoader |
Loads CLIP models from HuggingFace Hub or local directories. |
CreditRuleFactory<T> |
Maps the public CreditRule selector to a concrete ICreditRule instance. |
CreditRules |
Factory methods for the built-in credit-assignment (learning) rules, returning configurable ICreditRule instances for use with AiModelBuilder.ConfigureCreditRule(ICreditRule<T>). |
InputContractShapeResolver |
Adapts a caller’s preferred probe geometry to a declared input constraint. |
InputContractTensorFactory |
Contract-driven tensor creation for generated tests, examples and custom tooling. |
InputContractValidator |
Central value/shape validation used by model and layer entry points. |
InputDomainProviderRegistry |
Process-wide registry for explicit custom input-domain providers. |
LayerContractValidator |
Checks that a chain of layers agrees on tensor LAYOUT — each layer’s declared output feeding the next layer’s declared input — and reports the first disagreement by layer index and role. |
LayerGraphBuilder<T> |
Fluent construction of a LayerGraph: add each layer and say what feeds it. |
LayerNode<T> |
One layer in a LayerGraph, together with the nodes feeding it and any tensor transformation applied on the way in. |
SegmentationMetrics<T> |
Provides metrics for evaluating segmentation and classification tasks. |
ShapeInference |
Computes output shapes from declared axis roles (TensorLayoutAttribute) plus declared axis relations (IShapeContract), without running a forward pass. |
ShapeRelationDiscovery |
Recovers a layer’s symbolic shape relation by RUNNING it on probe shapes and fitting the simplest relation that reproduces every observation. |
Exceptions (2)
| Type | Summary |
|---|---|
InputContractBindingException |
Thrown when configuration cannot produce an executable input contract. |
InputContractViolationException |
Thrown when a supplied tensor violates a ready input contract. |