Class DistributedSampler

Inheritance Relationships

Base Type

Class Documentation

class DistributedSampler : public mindspore::dataset::Sampler

A class to represent a Distributed Sampler in the data pipeline.

说明

A Sampler that accesses a shard of the dataset.

Public Functions

DistributedSampler(int64_t num_shards, int64_t shard_id, dataset::ShuffleMode shuffle_mode = dataset::ShuffleMode::kGlobal, int64_t num_samples = 0, uint32_t seed = 1, int64_t offset = -1, bool even_dist = true)

Constructor.

参数

  • num_shards – [in] Number of shards to divide the dataset into.

  • shard_id – [in] Shard ID of the current shard within num_shards.

  • shuffle_mode – [in] If not kFalse, the indices are shuffled (default=kGlobal).

  • num_samples – [in] The number of samples to draw (default=0, return all samples).

  • seed – [in] The seed in use when shuffle_mode is not kFalse (default=1).

  • offset – [in] The starting position where access to elements in the dataset begins (default=-1).

  • even_dist – [in] If true, each shard would return the same number of rows (default=true). If false the total rows returned by all the shards would not have overlap.

样例
/* creates a distributed sampler with 2 shards in total. This shard is shard 0 */
std::string file_path = "/path/to/test.mindrecord";
std::shared_ptr<Dataset> ds = MindData(file_path, {}, std::make_shared<DistributedSampler>(2, 0, false));
~DistributedSampler() override = default

Destructor.