BioBERTNER<T>
Models & Types · AiDotNet.NER.TransformerBased
BioBERT-NER: Biomedical domain-specific BERT for Named Entity Recognition in biomedical text.
Inheritance: NeuralNetworkBase<T> → NERNeuralNetworkBase<T> → SequenceLabelingNERBase<T> → TransformerNERBase<T> → BioBERTNER<T>
For Beginners
BioBERT is BERT that has been additionally trained on millions of biomedical research papers. It understands medical/scientific terminology better than general BERT. Use BioBERT-NER when you need to extract entities from biomedical text (diseases, drugs, genes, proteins, chemicals). Also consider PubMedBERT for even better biomedical NER.
How It Works
BioBERT-NER (Lee et al., Bioinformatics 2020 - “BioBERT: a pre-trained biomedical language representation model for biomedical text mining”) is BERT pre-trained on large-scale biomedical corpora for domain-specific NER tasks like gene, protein, disease, drug, and species recognition.
Pre-training Data:
- PubMed abstracts: ~4.5B words from 30M+ biomedical abstracts
- PMC full-text articles: ~13.5B words from open-access articles
- Initialized from BERT-base weights, then continued pre-training on biomedical text
Biomedical NER Tasks:
- Gene/Protein NER: BC2GM (87.5% F1), JNLPBA (79.3% F1)
- Disease NER: NCBI Disease (89.4% F1), BC5CDR-Disease (87.2% F1)
- Drug/Chemical NER: BC5CDR-Chemical (93.4% F1), BC4CHEMD (92.4% F1)
- Species NER: Species-800 (74.4% F1), LINNAEUS (88.2% F1)
Why domain-specific pre-training matters: General BERT understands “IL-6” as just tokens, but BioBERT understands it as interleukin-6, a cytokine. This domain knowledge comes from reading millions of biomedical papers during pre-training, enabling BioBERT to recognize biomedical entities that general BERT misses.
Example
var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 512, numClasses: 9);
var bioBERTNER = new BioBERTNER<float>(architecture);
Constructors
| Constructor | Summary |
|---|---|
BioBERTNER(NeuralNetworkArchitecture<T> architecture, TransformerNEROptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a BioBERT-NER model in native training mode. |
BioBERTNER(NeuralNetworkArchitecture<T> architecture, string modelPath, TransformerNEROptions options) |
Creates a BioBERT-NER model in native training mode. |