PacBayesMetaBound

Helpers & Utilities · AiDotNet.MetaLearning.Components

SImPa’s PAC-Bayes generalization bounds: the single-task bound (Theorem 1) and the two-level meta-learning bound (Theorem 2) from Nguyen, Do and Carneiro (arXiv:2003.02455).

For Beginners

A PAC-Bayes bound is a promise of the form “the error on data you have not seen is at most the error you measured, plus a penalty”. The penalty grows when the learned distribution moves far from the prior, and shrinks as you get more data or more tasks.

How It Works

Kept as plain arithmetic over doubles, separate from the algorithm, because these are the paper’s theorem statements and are checkable in isolation — the scaling in the denominators and the shape of the log terms are the substance of the result, and burying them inside a training loop is how they drift.

Methods

Method Summary
double MetaLearning(double empiricalValidationLoss, double expectedTaskKL, double metaKL, int validationSampleCount, int taskCount, double epsilon) Theorem 2, the meta-learning bound: the empirical validation loss plus a TASK-level and a META-level complexity term.
double SingleTask(double empiricalLoss, double klDivergence, int sampleCount, double epsilon) Theorem 1, the single-task bound: empiricalLoss + sqrt((KL[q||p] + ln(m / epsilon)) / (2 * (m - 1))).

Fields

Field Summary
double PaperEpsilon The paper’s confidence parameter, epsilon = 0.1 for every task.