AnswerSimilarityMetric<T>
Models & Types · AiDotNet.RetrievalAugmentedGeneration.Evaluation
Evaluates the similarity between the generated answer and ground truth.
Inheritance: RAGMetricBase<T> → AnswerSimilarityMetric<T>
For Beginners
This checks how close the AI’s answer is to the correct answer.
Think of it like grading an exam:
- You have the answer key (ground truth)
- The student’s answer (generated answer)
- This metric gives partial credit based on how much overlaps
For example:
- Ground truth: “Photosynthesis converts sunlight into energy”
- Generated: “Photosynthesis converts sunlight into chemical energy”
- Score: ~0.85 (most words match)
Scoring:
- 1.0: Perfect match
- 0.5-0.8: Partially correct
- 0.0-0.3: Mostly incorrect
Use cases:
- Benchmarking your RAG system against test datasets
- A/B testing different configurations
- Regression testing (ensure changes don’t hurt quality)
Note: This uses simple word overlap. Production systems should use semantic similarity with embeddings or BERTScore for better accuracy.
How It Works
This metric measures how similar the generated answer is to a known correct answer (ground truth). It uses Jaccard similarity (word overlap) to compare the two texts. This is useful for benchmarking and regression testing when you have reference answers.
Constructors
| Constructor | Summary |
|---|---|
AnswerSimilarityMetric(IEmbeddingModel<T> embeddingModel) |
Initializes a new instance of the AnswerSimilarityMetric class. |
Properties
| Property | Summary |
|---|---|
string Description { get; } |
Gets the description of what this metric measures. |
string Name { get; } |
Gets the name of this metric. |