NTMOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.MetaLearning.Options

Configuration options for Neural Turing Machine (NTM) algorithm.

Inheritance: ModelOptions → NTMOptions<T, TInput, TOutput>

Implements: IMetaLearnerOptions<T>

For Beginners

NTM is like a neural computer with RAM:

  1. Controller network processes inputs like a CPU
  2. Generates read/write keys for memory access
  3. Attention mechanism determines where to read/write
  4. External memory stores information persistently
  5. Differentiable operations allow end-to-end learning

This allows learning algorithms like sorting, copying, and associative recall!

How It Works

Neural Turing Machines augment neural networks with an external memory matrix and differentiable attention mechanisms for reading and writing. This enables algorithms to be learned and executed within the neural network itself.

Constructors

Constructor Summary
NTMOptions(IFullModel<T, TInput, TOutput> metaModel) Initializes a new instance of the NTMOptions class with the required meta-model.

Properties

Property Summary
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } Gets or sets the episodic data loader for sampling tasks.
IFullModel<T, TInput, TOutput> MetaModel { get; set; } Gets or sets the meta-model (controller network) to be trained.
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built.
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } Optimizer for the meta-update.
ILossFunction<T> LossFunction { get; set; } Loss function for meta-training; null uses cross-entropy over logits.
NTMControllerType ControllerType { get; set; } Gets or sets the type of controller to use.
NTMMemoryInitialization MemoryInitialization { get; set; } Gets or sets how to initialize memory.
bool EnableCheckpointing { get; set; } Gets or sets whether to save model checkpoints.
bool InitializeMemory { get; set; } Gets or sets whether to initialize memory at start of episodes.
bool PresentPreviousLabel { get; set; } Gets or sets whether each input is presented together with the previous step’s label, one-hot encoded (Santoro et al.
bool UseFirstOrder { get; set; } Gets or sets whether to use first-order approximation.
double InnerLearningRate { get; set; } Gets or sets the learning rate for the inner loop (not used in NTM).
double MemorySharpnessRegularization { get; set; } Gets or sets the memory sharpness regularization strength.
double MemoryUsageRegularization { get; set; } Gets or sets the memory usage regularization strength.
double OuterLearningRate { get; set; } Gets or sets the learning rate for the outer loop (controller training).
double? GradientClipThreshold { get; set; } Gets or sets the maximum gradient norm for gradient clipping.
int AdaptationSteps { get; set; } Gets or sets the number of adaptation steps.
int CheckpointFrequency { get; set; } Gets or sets how often to save checkpoints.
int ControllerHiddenSize { get; set; } Gets or sets the hidden size of the controller.
int EvaluationFrequency { get; set; } Gets or sets how often to evaluate the meta-learner.
int EvaluationTasks { get; set; } Gets or sets the number of tasks to use for evaluation.
int MemorySize { get; set; } Gets or sets the number of memory slots.
int MemoryWidth { get; set; } Gets or sets the dimension of each memory slot.
int MetaBatchSize { get; set; } Gets or sets the number of tasks to sample per meta-training iteration.
int NumClasses { get; set; } Gets or sets the number of output classes.
int NumMetaIterations { get; set; } Gets or sets the total number of meta-training iterations.
int NumReadHeads { get; set; } Gets or sets the number of read heads.
int NumWriteHeads { get; set; } Gets or sets the number of write heads.
int ShiftRange { get; set; } Gets or sets the shift range R of location-based addressing: every head emits a shift weighting over the 2R + 1 integer offsets -R..R and convolves its weighting with it (Graves et al.
int? RandomSeed { get; set; } Gets or sets the random seed for reproducibility.

Methods

Method Summary
IMetaLearnerOptions<T> Clone() Creates a deep copy of the NTM options.
bool IsValid() Validates that all NTM configuration options are properly set.