FeatureTransformerLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Implements the Feature Transformer block used in TabNet architecture.
Inheritance: LayerBase<T> → FeatureTransformerLayer<T>
Implements: IShapeContract
For Beginners
The Feature Transformer is like a smart processor that takes selected features and transforms them into useful representations.
Key concepts:
- Shared Layers: Parameters shared across all decision steps (helps learn common patterns)
- Step-Specific Layers: Parameters unique to each step (learns step-specific patterns)
- GLU (Gated Linear Unit): A gating mechanism that controls information flow
- Residual Connections: Helps with gradient flow during training
Think of it as a two-part filter:
- One part decides what information to keep (the “gate”)
- The other part provides the actual information
- The final output is the product of both
This architecture allows TabNet to learn complex feature interactions effectively.
How It Works
The Feature Transformer processes selected features at each decision step using shared and step-specific layers. It employs a GLU (Gated Linear Unit) mechanism for non-linear transformations combined with Ghost Batch Normalization for regularization.
Reference: “TabNet: Attentive Interpretable Tabular Learning” (Arik & Pfister, AAAI 2021)
Constructors
| Constructor | Summary |
|---|---|
FeatureTransformerLayer(int inputDim, int outputDim, List<FullyConnectedLayer<T>> sharedLayers, List<GhostBatchNormalization<T>> sharedBNLayers, int numSharedLayers, int numStepSpecificLayers, int virtualBatchSize, double momentum, double epsilon) |
Initializes a new instance of the FeatureTransformer class. |
Properties
| Property | Summary |
|---|---|
List<FullyConnectedLayer<T>> SharedFCLayers { get; } |
Gets the shared fully connected layers for reuse in other FeatureTransformers. |
List<GhostBatchNormalization<T>> SharedBNLayers { get; } |
Gets the shared batch normalization layers for reuse. |
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
Tensor<T> GetBiases() |
Gets the biases tensor (not applicable for this composite layer). |
Tensor<T> GetWeights() |
Gets the weights tensor (not applicable for this composite layer). |
int[] GetInputShape() |
Gets the input shape for this layer. |
void ClearGradients() |
Clears accumulated gradients. |
void ResetState() |
|
void SetTrainingMode(bool isTraining) |
Sets training mode. |
void UpdateParameters(T learningRate) |
|
void UpdateParameters(Vector<T> parameters) |