ClinicalBERTNER<T>
Models & Types · AiDotNet.NER.TransformerBased
ClinicalBERT-NER: Clinical domain BERT for Named Entity Recognition in clinical notes and EHRs.
Inheritance: NeuralNetworkBase<T> → NERNeuralNetworkBase<T> → SequenceLabelingNERBase<T> → TransformerNERBase<T> → ClinicalBERTNER<T>
For Beginners
ClinicalBERT is BERT trained on clinical notes from hospitals. It understands medical abbreviations, drug names, diagnoses, and the unique writing style of doctors and nurses. Use ClinicalBERT-NER for extracting medical entities from clinical notes, discharge summaries, or electronic health records. Note: HIPAA compliance and data privacy are critical when processing real clinical data.
How It Works
ClinicalBERT-NER (Alsentzer et al., NAACL 2019 Clinical NLP Workshop - “Publicly Available Clinical BERT Embeddings”; Huang et al., 2019 - “ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission”) is BERT further pre-trained on clinical text from electronic health records (EHRs) for clinical NLP tasks.
Pre-training Data:
- MIMIC-III clinical notes (~880M words from 2M+ clinical notes)
- Discharge summaries, radiology reports, nursing notes
- Physician progress notes, operative reports
- Initialized from BioBERT weights (which were initialized from BERT)
Clinical NER Entity Types:
- Problem/Diagnosis: Type 2 diabetes mellitus, acute myocardial infarction
- Treatment: Metformin 500mg, coronary artery bypass graft
- Test: Complete blood count, chest X-ray, echocardiogram
- Anatomy: Left ventricle, right lower lobe, anterior cruciate ligament
- Dosage: 500mg BID, 10mg/kg/day
- Duration: 7-day course, q6h for 48 hours
- Temporal: Post-operative day 3, on admission, at discharge
Why Clinical NER is Different: Clinical text is uniquely challenging: heavy abbreviations (SOB = shortness of breath, not an expletive), misspellings, fragmented sentences, negation (patient denies chest pain), and domain-specific jargon (PRN, NPO, PERRLA). ClinicalBERT handles these patterns.
Performance:
- i2b2 2010 Clinical NER: ~88.5% F1 (vs BERT ~84.3%, BioBERT ~86.1%)
- i2b2 2012 Temporal NER: ~85.2% F1
- n2c2 2018 Medication NER: ~91.3% F1
Example
var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 512, numClasses: 9);
var clinicalBERTNER = new ClinicalBERTNER<float>(architecture);
Constructors
| Constructor | Summary |
|---|---|
ClinicalBERTNER(NeuralNetworkArchitecture<T> architecture, TransformerNEROptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a ClinicalBERT-NER model in native training mode. |
ClinicalBERTNER(NeuralNetworkArchitecture<T> architecture, string modelPath, TransformerNEROptions options) |
Creates a ClinicalBERT-NER model in native training mode. |