CnnDailyMailDataLoader<T>
Models & Types · AiDotNet.Data.Text.Benchmarks
Loads the CNN/DailyMail abstractive-summarization dataset v3.0.0 (Hermann et al. 2015 / See et al. 2017) via HuggingFace parquet.
Inheritance: DataLoaderBase<T> → InputOutputDataLoaderBase<T, Tensor<T>, Tensor<T>> → CnnDailyMailDataLoader<T>
How It Works
Expects parquet shards under {DataPath}/cnn_dailymail/. Auto-download
fetches a small sentinel set: validation parquet + a single train shard.
For the full 287k-train corpus, manually download all shards from
huggingface.co/datasets/abisee/cnn_dailymail and place them in DataPath.
Constructors
| Constructor | Summary |
|---|---|
CnnDailyMailDataLoader(CnnDailyMailDataLoaderOptions options) |
Properties
| Property | Summary |
|---|---|
int FeatureCount { get; } |
|
int OutputDimension { get; } |
|
int TotalCount { get; } |
|
string Description { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
ValueTuple<IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>, IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>, IInputOutputDataLoader<T, Tensor<T>, Tensor<T>>> Split(double trainRatio, double validationRatio, int? seed) |