TransformerNERBase<T>

Base Classes · AiDotNet.NER.TransformerBased

Base class for transformer-based NER models (BERT-NER, RoBERTa-NER, DeBERTa-NER, etc.).

Inheritance: NeuralNetworkBase<T> → NERNeuralNetworkBase<T> → SequenceLabelingNERBase<T> → TransformerNERBase<T>

Implements: INERModel<T>

Known derived types: BERTNER<T>, BLINKNER<T>, BioBERTNER<T>, ClinicalBERTNER<T>, DeBERTaNER<T>, DistilBERTNER<T>, ELECTRANER<T>, FinBERTNER<T>, InstructionNER<T>, LegalBERTNER<T>, ONNXNER<T>, PromptNER<T>, PubMedBERTNER<T>, RELNER<T>, RoBERTaNER<T>, SECBertNER<T>, SciBERTNER<T>, SpanBERTNER<T>, TemplateNER<T>, TinyBERTNER<T>, XLMRoBERTaNER<T>

For Beginners

Transformer models are the most accurate NER models available. They use “self-attention” to understand each word by looking at all other words in the sentence simultaneously. Different transformer variants (BERT, RoBERTa, etc.) are like different editions of the same textbook - they have the same structure but were trained slightly differently, leading to different strengths.

How It Works

All transformer-based NER models share the same fine-tuning architecture:

The transformer encoder consists of N stacked layers, each containing:

  1. Multi-head self-attention: Each token attends to all other tokens
  2. Feed-forward network: Non-linear transformation of each token independently
  3. Layer normalization and residual connections: Stabilize training

The key differences between BERT, RoBERTa, DeBERTa, etc. are in the pre-training (masked language model, replaced token detection, etc.) and attention mechanism (absolute vs relative position encoding, disentangled attention, etc.). The NER fine-tuning architecture is identical across all variants.

This base class provides the common implementation. Derived classes only need to:

  1. Pass their model-specific name to the base constructor
  2. Optionally override CreateDefaultLayers for architecture-specific layer configurations

Properties

Property Summary
int[] ExpectedInputShape { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> PredictLabels(Tensor<T> tokenEmbeddings)
void Train(Tensor<T> input, Tensor<T> expected)