ONNXNER<T>
Models & Types · AiDotNet.NER.TransformerBased
ONNX-NER: Generic ONNX Runtime-based NER model for high-performance inference with any exported NER model.
Inheritance: NeuralNetworkBase<T> → NERNeuralNetworkBase<T> → SequenceLabelingNERBase<T> → TransformerNERBase<T> → ONNXNER<T>
For Beginners
ONNX-NER is not a specific NER architecture, but rather a way to run any NER model that has been exported to the ONNX format. ONNX is like a “universal translator” for neural network models - it lets you train in any framework and run inference efficiently. Use ONNX-NER when you have a pre-trained ONNX model file and want the fastest possible inference, or when you want to deploy NER models without depending on PyTorch/TensorFlow.
How It Works
ONNX-NER provides a model-agnostic wrapper for running any NER model exported to ONNX format through ONNX Runtime. This enables high-performance inference regardless of the original training framework (PyTorch, TensorFlow, JAX, etc.).
ONNX Runtime Optimizations:
- Graph optimizations: Operator fusion (MatMul+Add, Conv+BN), constant folding,
and redundant computation elimination
- Hardware acceleration: CUDA, TensorRT, DirectML, OpenVINO, CoreML execution providers
- Quantization: INT8/FP16 inference for reduced memory and faster execution
- Memory optimization: Memory-efficient attention patterns, memory arena pre-allocation
- Threading: Intra-op and inter-op parallelism configuration
Supported Model Sources:
- HuggingFace Transformers (exported via optimum or torch.onnx.export)
- PyTorch models (exported via torch.onnx.export)
- TensorFlow/Keras models (exported via tf2onnx)
- Custom models exported to ONNX format
Performance (typical speedups over PyTorch):
- CPU: 2-4x faster with graph optimizations
- GPU (CUDA): 1.5-3x faster with TensorRT
- INT8 quantized: 3-5x faster than FP32 with ~0.5% F1 loss
Common ONNX NER Models:
- dslim/bert-base-NER (ONNX): ~92.4% F1, ~5ms/sentence on GPU
- elastic/distilbert-base-cased-finetuned-conll03-english: ~91.1% F1, ~2ms/sentence on GPU
- Jean-Baptiste/camembert-ner (French NER): ~90.5% F1
Example
var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 512, numClasses: 9);
var oNNXNER = new ONNXNER<float>(architecture);
Constructors
| Constructor | Summary |
|---|---|
ONNXNER(NeuralNetworkArchitecture<T> architecture, TransformerNEROptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates an ONNX-NER model in native training mode. |
ONNXNER(NeuralNetworkArchitecture<T> architecture, string modelPath, TransformerNEROptions options) |
Creates an ONNX-NER model in native training mode. |