mindspore.dataset.config

The configuration manager.

mindspore.dataset.config.get_monitor_sampling_interval()

Get the default interval of performance monitor sampling.

Returns: interval(ms) of performance monitor sampling.
Return type: Interval

mindspore.dataset.config.get_num_parallel_workers()

Get the default number of parallel workers.

Returns: Int, number of parallel workers to be used as a default for each operation

mindspore.dataset.config.get_prefetch_size()

Get the prefetch size in number of rows.

Returns: Size, total number of rows to be prefetched.

mindspore.dataset.config.get_seed()

Get the seed.

Returns: Int, seed.

mindspore.dataset.config.load(file)

Load configuration from a file.

Parameters: file (str) – path the config file to be loaded.
Raises: RuntimeError – If file is invalid and parsing fails.

Examples

>>> import mindspore.dataset as ds
>>> # sets the default value according to values in configuration file.
>>> ds.config.load("path/to/config/file")
>>> # example config file:
>>> # {
>>> #     "logFilePath": "/tmp",
>>> #     "rowsPerBuffer": 32,
>>> #     "numParallelWorkers": 4,
>>> #     "workerConnectorSize": 16,
>>> #     "opConnectorSize": 16,
>>> #     "seed": 5489,
>>> #     "monitorSamplingInterval": 30
>>> # }

mindspore.dataset.config.set_monitor_sampling_interval(interval)

Set the default interval(ms) of monitor sampling.

Parameters: interval (int) – interval(ms) to be used to performance monitor sampling.
Raises: ValueError – If interval is invalid (<= 0 or > MAX_INT_32).

Examples

>>> import mindspore.dataset as ds
>>> # sets the new interval value.
>>> ds.config.set_monitor_sampling_interval(100)

mindspore.dataset.config.set_num_parallel_workers(num)

Set the default number of parallel workers.

Parameters: num (int) – number of parallel workers to be used as a default for each operation.
Raises: ValueError – If num_parallel_workers is invalid (<= 0 or > MAX_INT_32).

Examples

>>> import mindspore.dataset as ds
>>> # sets the new parallel_workers value, now parallel dataset operators will run with 8 workers.
>>> ds.config.set_num_parallel_workers(8)

mindspore.dataset.config.set_prefetch_size(size)

Set the number of rows to be prefetched.

Parameters: size (int) – total number of rows to be prefetched.
Raises: ValueError – If prefetch_size is invalid (<= 0 or > MAX_INT_32).

Examples

>>> import mindspore.dataset as ds
>>> # sets the new prefetch value.
>>> ds.config.set_prefetch_size(1000)

mindspore.dataset.config.set_seed(seed)

Set the seed to be used in any random generator. This is used to produce deterministic results.

Note

This set_seed function sets the seed in the python random library and numpy.random library for deterministic python augmentations using randomness. This set_seed function should be called with every iterator created to reset the random seed. In our pipeline this does not guarantee deterministic results with num_parallel_workers > 1.

Parameters: seed (int) – seed to be set.
Raises: ValueError – If seed is invalid (< 0 or > MAX_UINT_32).

Examples

>>> import mindspore.dataset as ds
>>> # sets the new seed value, now operators with a random seed will use new seed value.
>>> ds.config.set_seed(1000)