FinBERT<T>

Models & Types · AiDotNet.Finance.NLP

FinBERT (Financial BERT) model for financial sentiment analysis.

Inheritance: NeuralNetworkBase<T> → FinancialModelBase<T> → FinancialNLPModelBase<T> → FinBERT<T>

For Beginners

FinBERT solves a key problem in financial NLP:

The Key Insight: General-purpose sentiment models often misinterpret financial language. For example, “shares dropped 5% on earnings miss” is clearly negative for the stock, but a general sentiment model might not understand this. FinBERT is trained specifically on financial text to understand such nuances.

What Problems Does FinBERT Solve?

  • Sentiment analysis of financial news articles
  • Processing SEC filings (10-K, 10-Q, 8-K) for sentiment signals
  • Analyzing earnings call transcripts for tone
  • Social media sentiment monitoring for trading signals
  • Document classification in financial contexts

How FinBERT Works:

  1. Tokenization: Text is split into WordPiece tokens (subwords)
  2. Embedding: Tokens are converted to dense vectors with position information
  3. Transformer: 12 layers of bidirectional self-attention capture context
  4. Classification: [CLS] token embedding is used for sentiment prediction
  5. Output: Softmax over classes: Positive, Negative, Neutral

FinBERT Architecture:

  • Input: [CLS] token1 token2 … tokenN [SEP]
  • Embeddings: Token + Position + Segment embeddings (768-dim)
  • Transformer: 12 layers, 12 heads, 768 hidden dim
  • Output: 3-class softmax (negative, neutral, positive)

Key Benefits:

  • Understands financial language and terminology
  • Captures context across the entire input sequence
  • Pre-trained on large financial corpora
  • State-of-the-art accuracy on financial sentiment benchmarks

How It Works

FinBERT is a BERT model fine-tuned on financial text for sentiment analysis. It understands financial terminology and context to accurately classify sentiment in financial news, SEC filings, earnings calls, and other financial text.

Reference: Araci, “FinBERT: Financial Sentiment Analysis with Pre-trained Language Models”, 2019. https://arxiv.org/abs/1908.10063

Example

// Define architecture for financial sentiment classification (512 tokens, 3 classes: pos/neg/neutral)
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.MultiClassClassification,
    inputHeight: 512, inputWidth: 1, inputDepth: 1, outputSize: 3);
            
// Training mode: fine-tune BERT on financial text for sentiment analysis
var model = new FinBERT<double>(architecture);
            
// ONNX inference mode: load pre-trained FinBERT model
var onnxModel = new FinBERT<double>(architecture, "finbert.onnx");

Constructors

Constructor Summary
FinBERT(NeuralNetworkArchitecture<T> architecture, FinBERTOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Initializes a new instance of the FinBERT model in native mode for training.
FinBERT(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, FinBERTOptions<T> options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Initializes a new instance of the FinBERT model in ONNX mode for inference.

Properties

Property Summary
bool SupportsTraining { get; }
bool UseNativeMode { get; }
int HiddenDimension { get; }
int MaxSequenceLength { get; }
int NumAttentionHeads { get; } Gets the number of attention heads.
int NumLayers { get; } Gets the number of transformer layers.
int NumSentimentClasses { get; }
int VocabularySize { get; }

Methods

Method Summary
Dictionary<string, T> GetFinancialMetrics() Gets financial-specific NLP metrics from the model.
ModelMetadata<T> GetModelMetadata() Gets metadata about the FinBERT model.
SentimentResult<T>[] AnalyzeSentiment(string[] texts) Analyzes sentiment from raw text strings.
Tensor<T> AnalyzeSentiment(Tensor<T> tokenIds) Analyzes sentiment from raw text strings.
Tensor<T> Forward(Tensor<T> input) Performs the forward pass through all layers.
Tensor<T> ForwardForTraining(Tensor<T> input) Tape-aware training forward.
Tensor<T> GetEmbeddings(Tensor<T> tokenIds) Gets embeddings (vector representations) for input tokens.
Tensor<T> GetSequenceEmbedding(Tensor<T> tokenIds) Gets the [CLS] token embedding representing the entire input sequence.
int[] Tokenize(string text, int? maxLength) Tokenizes raw text into token IDs.
string Detokenize(int[] tokenIds) Converts token IDs back to text.