CnnDailyMailDataLoaderOptions

Options & Configuration ยท AiDotNet.Data.Text.Benchmarks

Configuration options for the CNN/DailyMail summarization loader (Hermann et al. 2015 / See et al. 2017).

How It Works

CNN/DailyMail v3.0.0 โ€” 287k train / 13.4k val / 11.5k test article-summary pairs. The canonical English news abstractive-summarization benchmark. AutoDownload pulls the HuggingFace parquet shards (abisee/cnn_dailymail).

Constructors

Constructor Summary
CnnDailyMailDataLoaderOptions()

Properties

Property Summary
DatasetSplit Split { get; set; } Gets or sets which split of the dataset is loaded.
bool AutoDownload { get; set; } Gets or sets whether the dataset is downloaded into the data directory when it is not there.
int MaxArticleLength { get; set; } Gets or sets the token length each article is padded or truncated to.
int MaxSummaryLength { get; set; } Gets or sets the token length each summary is padded or truncated to.
int VocabularySize { get; set; } Gets or sets the most distinct tokens in the vocabulary built from the loaded text.
int? MaxSamples { get; set; } Gets or sets the most samples loaded; null loads the whole split.
string DataPath { get; set; } Gets or sets the dataset root directory; null uses the default data directory for this dataset.