RewardFilteredDatasetBuilder
Helpers & Utilities · AiDotNet.Agentic.SelfImproving
Builds a FineTuningDataset from captured trajectories by keeping only those whose reward
meets a threshold and turning each into a (prompt, completion) pair — reward-filtered behavior cloning,
the data-preparation half of online LoRA self-improvement.
For Beginners
Sift the logbook for the runs that scored well, and from each make a question→good-answer pair. Those pairs become the lesson plan for fine-tuning the local model on its own best work.
How It Works
The prompt is the trajectory’s conversation up to (but excluding) its final turn; the completion is the final answer. The resulting dataset is then handed to the repository’s LoRA / fine-tuning trainer to produce an adapter for the local model — the trainer is the model-layer step; this is the agentic step that decides what to learn from (only high-reward runs).
Constructors
| Constructor | Summary |
|---|---|
RewardFilteredDatasetBuilder(double minReward) |
Initializes a new builder. |
Methods
| Method | Summary |
|---|---|
FineTuningDataset Build(IEnumerable<AgentTrajectory> trajectories) |
Builds the dataset from the given trajectories, keeping only graded runs at or above the threshold that have both a prompt context and a non-empty completion. |