Hdf5Dataset<T>

Models & Types · AiDotNet.Data.Formats

Provides read/write access to datasets in a custom binary format for named multidimensional arrays, inspired by the HDF5 data model.

Implements: IDisposable

How It Works

Important: This is NOT a native HDF5 implementation and is NOT compatible with HDF5 files created by h5py, HDFView, or other HDF5 tools. It is a custom binary format designed for storing named tensor datasets within AiDotNet. For native HDF5 interop, use the PureHDF NuGet package.

The on-disk format:

  • Header: [magic: “HDF5” 4 bytes] [version: 4 bytes] [numDatasets: 4 bytes]
  • Dataset table: for each dataset: [nameLen: 4][name: bytes][rank: 4][dims: 4*rank][dataOffset: 8][dataLength: 8]
  • Data section: raw double-precision values for each dataset

Constructors

Constructor Summary
Hdf5Dataset(Hdf5DatasetOptions options) Opens an HDF5 dataset file.

Properties

Property Summary
IReadOnlyCollection<string> DatasetNames { get; } Gets the names of all datasets in the file.

Methods

Method Summary
Tensor<T> ReadDataset(string datasetName) Reads a named dataset as a Tensor.
Tensor<T> ReadSlice(string datasetName, int startRow, int numRows) Reads a slice of rows from a named dataset.
int[] GetShape(string datasetName) Gets the shape of a named dataset.
void Dispose()
void WriteFile(string filePath, IReadOnlyDictionary<string, ValueTuple<T[], int[]>> datasets) Writes multiple named datasets to an HDF5 file.