Class CSVDataset

Inheritance Relationships

Base Type

Class Documentation

class CSVDataset : public mindspore::dataset::Dataset

A source dataset that reads and parses comma-separated values (CSV) datasets.

Public Functions

CSVDataset(const std::vector<std::vector<char>> &dataset_files, char field_delim, const std::vector<std::shared_ptr<CsvBase>> &column_defaults, const std::vector<std::vector<char>> &column_names, int64_t num_samples, ShuffleMode shuffle, int32_t num_shards, int32_t shard_id, const std::shared_ptr<DatasetCache> &cache)

Constructor of CSVDataset.

Parameters
  • dataset_files[in] List of files to be read to search for a pattern of files. The list will be sorted in a lexicographical order.

  • field_delim[in] A char that indicates the delimiter to separate fields (default=’,’).

  • column_defaults[in] List of default values for the CSV field (default={}). Each item in the list is either a valid type (float, int, or string). If this is not provided, treats all columns as string type.

  • column_names[in] List of column names of the dataset (default={}). If this is not provided, infers the column_names from the first row of CSV file.

  • num_samples[in] The number of samples to be included in the dataset. (Default = 0 means all samples).

  • shuffle[in] The mode for shuffling data every epoch (Default=ShuffleMode::kGlobal). Can be any of: ShuffleMode::kFalse - No shuffling is performed. ShuffleMode::kFiles - Shuffle files only. ShuffleMode::kGlobal - Shuffle both the files and samples.

  • num_shards[in] Number of shards that the dataset should be divided into. (Default = 1)

  • shard_id[in] The shard ID within num_shards. This argument should be specified only when num_shards is also specified (Default = 0).

  • cache[in] Tensor cache to use.(default=nullptr which means no cache is used).

~CSVDataset() override = default

Destructor of CSVDataset.