XSumDataLoaderOptions
Options & Configuration · AiDotNet.Data.Text.Benchmarks
Configuration options for the XSum extreme summarization loader (Narayan et al. 2018).
How It Works
XSum — 226k BBC articles each paired with a single-sentence summary. The “extreme” abstractive summarization benchmark; targets are highly compressive (≈ 1:30 compression ratio). Splits: 204k train / 11.3k val / 11.3k test. AutoDownload pulls a HuggingFace parquet shard.
Constructors
| Constructor | Summary |
|---|---|
XSumDataLoaderOptions() |
Properties
| Property | Summary |
|---|---|
DatasetSplit Split { get; set; } |
Gets or sets which split of the dataset is loaded. |
bool AutoDownload { get; set; } |
Gets or sets whether the dataset is downloaded into the data directory when it is not there. |
int MaxDocumentLength { get; set; } |
Gets or sets the token length each document is padded or truncated to. |
int MaxSummaryLength { get; set; } |
Gets or sets the token length each summary is padded or truncated to. |
int VocabularySize { get; set; } |
Gets or sets the most distinct tokens in the vocabulary built from the loaded text. |
int? MaxSamples { get; set; } |
Gets or sets the most samples loaded; null loads the whole split. |
string DataPath { get; set; } |
Gets or sets the dataset root directory; null uses the default data directory for this dataset. |