FinBERT<T>
Models & Types · AiDotNet.Finance.NLP
FinBERT (Financial BERT) model for financial sentiment analysis.
Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → FinancialNLPModelBase<T> → FinBERT<T>
For Beginners
FinBERT solves a key problem in financial NLP:
The Key Insight: General-purpose sentiment models often misinterpret financial language. For example, “shares dropped 5% on earnings miss” is clearly negative for the stock, but a general sentiment model might not understand this. FinBERT is trained specifically on financial text to understand such nuances.
What Problems Does FinBERT Solve?
- Sentiment analysis of financial news articles
- Processing SEC filings (10-K, 10-Q, 8-K) for sentiment signals
- Analyzing earnings call transcripts for tone
- Social media sentiment monitoring for trading signals
- Document classification in financial contexts
How FinBERT Works:
- Tokenization: Text is split into WordPiece tokens (subwords)
- Embedding: Tokens are converted to dense vectors with position information
- Transformer: 12 layers of bidirectional self-attention capture context
- Classification: [CLS] token embedding is used for sentiment prediction
- Output: Softmax over classes: Positive, Negative, Neutral
FinBERT Architecture:
- Input: [CLS] token1 token2 … tokenN [SEP]
- Embeddings: Token + Position + Segment embeddings (768-dim)
- Transformer: 12 layers, 12 heads, 768 hidden dim
- Output: 3-class softmax (negative, neutral, positive)
Key Benefits:
- Understands financial language and terminology
- Captures context across the entire input sequence
- Pre-trained on large financial corpora
- State-of-the-art accuracy on financial sentiment benchmarks
How It Works
FinBERT is a BERT model fine-tuned on financial text for sentiment analysis. It understands financial terminology and context to accurately classify sentiment in financial news, SEC filings, earnings calls, and other financial text.
Reference: Araci, “FinBERT: Financial Sentiment Analysis with Pre-trained Language Models”, 2019.
https://arxiv.org/abs/1908.10063
Example
// Define architecture for financial sentiment classification (512 tokens, 3 classes: pos/neg/neutral)
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.MultiClassClassification,
inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 3);
// Training mode: fine-tune BERT on financial text for sentiment analysis
var model = new FinBERT<double>(architecture);
// ONNX inference mode: load pre-trained FinBERT model
var onnxModel = new FinBERT<double>(architecture, "finbert.onnx");
Constructors
| Constructor | Summary |
|---|---|
FinBERT(NeuralNetworkArchitecture<T> architecture, FinBERTOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance of the FinBERT model in native mode for training. |
FinBERT(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, FinBERTOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance of the FinBERT model in ONNX mode for inference. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
|
bool UseNativeMode { get; } |
|
int HiddenDimension { get; } |
|
int MaxSequenceLength { get; } |
|
int NumAttentionHeads { get; } |
Gets the number of attention heads. |
int NumLayers { get; } |
Gets the number of transformer layers. |
int NumSentimentClasses { get; } |
|
int VocabularySize { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> GetFinancialMetrics() |
Gets financial-specific NLP metrics from the model. |
ModelMetadata<T> GetModelMetadata() |
Gets metadata about the FinBERT model. |
SentimentResult<T>[] AnalyzeSentiment(string[] texts) |
Analyzes sentiment from raw text strings. |
Tensor<T> AnalyzeSentiment(Tensor<T> tokenIds) |
Analyzes sentiment from raw text strings. |
Tensor<T> Forward(Tensor<T> input) |
Performs the forward pass through all layers. |
Tensor<T> ForwardForTraining(Tensor<T> input) |
Tape-aware training forward. |
Tensor<T> GetEmbeddings(Tensor<T> tokenIds) |
Gets embeddings (vector representations) for input tokens. |
Tensor<T> GetSequenceEmbedding(Tensor<T> tokenIds) |
Gets the [CLS] token embedding representing the entire input sequence. |
int[] Tokenize(string text, int? maxLength) |
Tokenizes raw text into token IDs. |
string Detokenize(int[] tokenIds) |
Converts token IDs back to text. |