NoiseRobustnessMetric<T>

Models & Types · AiDotNet.RetrievalAugmentedGeneration.Evaluation

Evaluates how well a RAG system handles noisy or irrelevant documents in the retrieved context.

Inheritance: RAGMetricBase<T> → NoiseRobustnessMetric<T>

For Beginners

This checks if your RAG system can ignore “junk” documents.

Think of it like reading a research paper:

  • You have 10 sources
  • 5 are highly relevant to your topic
  • 5 are somewhat related but not very useful
  • A good researcher focuses on the 5 good sources
  • A bad researcher treats all 10 equally

This metric checks if your RAG system acts like a good researcher!

Score interpretation:

  • Score near 1.0: System focuses on relevant documents, ignores noise
  • Score near 0.5: System treats all documents equally (not robust)
  • Score near 0.0: System focuses on wrong documents

For example:

var retrievedDocs = new List<Document<double>>
{
// The metric reads each document's retrieval score, so documents without one are not counted.
new Document<double>("doc1", "Artificial intelligence is the simulation of human intelligence by machines.")
{
RelevanceScore = 0.9,
HasRelevanceScore = true
},
new Document<double>("doc2", "The 2024 football season opened in September.")
{
RelevanceScore = 0.1,
HasRelevanceScore = true
}
};
var metric = new NoiseRobustnessMetric<double>(0.3);
var answer = new GroundedAnswer<double>(
query: "What is AI?",
answer: "AI is artificial intelligence...",
sourceDocuments: retrievedDocs,
citations: new List<string>(),
confidenceScore: 0.8
);
var score = metric.Evaluate(answer);
// Result is available in the returned value

Why this matters:

  • Real-world retrieval often includes some irrelevant documents
  • Robust systems maintain quality despite imperfect retrieval
  • Helps evaluate production reliability

How It Works

This metric measures the robustness of a RAG system when the retrieved document set contains irrelevant or noisy documents. A robust system should focus on high-quality documents and maintain answer quality even when some retrieved documents are not relevant. The metric analyzes the distribution of relevance scores to determine if the system can distinguish signal from noise.

Constructors

Constructor Summary
NoiseRobustnessMetric(T noiseRatio) Initializes a new instance of the NoiseRobustnessMetric class.

Properties

Property Summary
string Description { get; } Gets the description of what this metric measures.
string Name { get; } Gets the name of this metric.