TransformerNERBase<T>
Base Classes · AiDotNet.NER.TransformerBased
Base class for transformer-based NER models (BERT-NER, RoBERTa-NER, DeBERTa-NER, etc.).
Inheritance: NeuralNetworkBase<T> → NERNeuralNetworkBase<T> → SequenceLabelingNERBase<T> → TransformerNERBase<T>
Implements: INERModel<T>
Known derived types: BERTNER<T>, BLINKNER<T>, BioBERTNER<T>, ClinicalBERTNER<T>, DeBERTaNER<T>, DistilBERTNER<T>, ELECTRANER<T>, FinBERTNER<T>, InstructionNER<T>, LegalBERTNER<T>, ONNXNER<T>, PromptNER<T>, PubMedBERTNER<T>, RELNER<T>, RoBERTaNER<T>, SECBertNER<T>, SciBERTNER<T>, SpanBERTNER<T>, TemplateNER<T>, TinyBERTNER<T>, XLMRoBERTaNER<T>
For Beginners
Transformer models are the most accurate NER models available. They use “self-attention” to understand each word by looking at all other words in the sentence simultaneously. Different transformer variants (BERT, RoBERTa, etc.) are like different editions of the same textbook - they have the same structure but were trained slightly differently, leading to different strengths.
How It Works
All transformer-based NER models share the same fine-tuning architecture:
The transformer encoder consists of N stacked layers, each containing:
- Multi-head self-attention: Each token attends to all other tokens
- Feed-forward network: Non-linear transformation of each token independently
- Layer normalization and residual connections: Stabilize training
The key differences between BERT, RoBERTa, DeBERTa, etc. are in the pre-training (masked language model, replaced token detection, etc.) and attention mechanism (absolute vs relative position encoding, disentangled attention, etc.). The NER fine-tuning architecture is identical across all variants.
This base class provides the common implementation. Derived classes only need to:
- Pass their model-specific name to the base constructor
- Optionally override CreateDefaultLayers for architecture-specific layer configurations
Properties
| Property | Summary |
|---|---|
int[] ExpectedInputShape { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> PredictLabels(Tensor<T> tokenEmbeddings) |
|
void Train(Tensor<T> input, Tensor<T> expected) |