TransformerNEROptions
Options & Configuration · AiDotNet.NER.Options
Base configuration options shared by all transformer-based NER models (BERT-NER, RoBERTa-NER, DeBERTa-NER, ELECTRA-NER, etc.).
Inheritance: ModelOptions → NeuralNetworkOptions → TransformerNEROptions
Known derived types: PromptNEROptions
For Beginners
Transformer models are the most powerful NER architectures available. They read text using “self-attention” - each word looks at every other word in the sentence to understand context. This is more powerful than BiLSTM-CRF because transformers can capture long-range dependencies (e.g., a pronoun referring to an entity mentioned 50 words ago).
The tradeoff is that transformer models are much larger (110M+ parameters for BERT-base vs ~1M for BiLSTM-CRF) and slower to train. However, they achieve state-of-the-art accuracy on all NER benchmarks.
How It Works
Transformer-based NER models share a common architecture: a pre-trained transformer encoder produces contextualized token representations, followed by a token classification head that predicts BIO labels for each token. The key differences between variants are in the pre-training strategy and attention mechanism, not the NER fine-tuning architecture.
The standard transformer NER architecture consists of:
- Transformer encoder: Multi-layer self-attention with feed-forward networks.
Each layer has multi-head attention (captures token-to-token relationships) and a feed-forward network (transforms each token’s representation independently).
- Classification head: A linear projection from hidden_size to num_labels,
optionally with a CRF layer for structured prediction.
- Optional CRF: Can be added on top of the transformer for structured decoding.
This base options class provides the shared parameters. Variant-specific options classes (BERTNEROptions, RoBERTaNEROptions, etc.) can extend this with model-specific settings.
Constructors
| Constructor | Summary |
|---|---|
TransformerNEROptions() |
Initializes a new instance with BERT-base defaults. |
TransformerNEROptions(TransformerNEROptions other) |
Initializes a new instance by deep-copying all settings from another instance. |
Properties
| Property | Summary |
|---|---|
NERModelVariant Variant { get; set; } |
Gets or sets the model size variant. |
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets the ONNX Runtime configuration options. |
bool UseCRF { get; set; } |
Gets or sets whether to use CRF decoding on top of the transformer. |
double DropoutRate { get; set; } |
Gets or sets the dropout rate for regularization. |
double EndLearningRate { get; set; } |
Gets or sets the learning rate reached at the end of linear decay. |
double LearningRate { get; set; } |
Gets or sets the learning rate for fine-tuning. |
double WarmupInitialLearningRate { get; set; } |
Gets or sets the learning rate used for the first warmup update. |
int HiddenDimension { get; set; } |
Gets or sets the hidden dimension of the transformer encoder. |
int IntermediateDimension { get; set; } |
Gets or sets the intermediate (feed-forward) dimension in each transformer layer. |
int MaxSequenceLength { get; set; } |
Gets or sets the maximum input sequence length in tokens. |
int NumAttentionHeads { get; set; } |
Gets or sets the number of self-attention heads in each transformer layer. |
int NumLabels { get; set; } |
Gets or sets the number of entity label classes in the BIO tagging scheme. |
int NumTransformerLayers { get; set; } |
Gets or sets the number of transformer encoder layers. |
int TotalTrainingSteps { get; set; } |
Gets or sets the total number of optimizer updates in the linear warmup/decay schedule. |
int WarmupSteps { get; set; } |
Gets or sets the number of optimizer updates used to linearly warm the learning rate from zero. |
string ModelPath { get; set; } |
Gets or sets the path to a pre-trained ONNX model file. |
string[] LabelNames { get; set; } |
Gets or sets the BIO label names. |