mindspore.experimental.optim.lr_scheduler.SequentialLR

class mindspore.experimental.optim.lr_scheduler.SequentialLR(optimizer, schedulers, milestones, last_epoch=- 1)

Receives the list of schedulers that is expected to be called sequentially during optimization process and milestone points that provides exact intervals to reflect which scheduler is supposed to be called at a given epoch.

Warning

This is an experimental lr scheduler module that is subject to change. This module must be used with optimizers in Experimental Optimizer .

Parameters

• optimizer (mindspore.experimental.optim.Optimizer) – Wrapped optimizer.

• schedulers (list[mindspore.experimental.optim.lr_scheduler.LRScheduler]) – List of learning rate schedulers.

• milestones (list) – List of integers that reflects milestone points.

• last_epoch (int, optional) – The index of the last epoch. Default: -1.

Raises

• ValueError – The optimizer in schedulers is different from the optimizer passed in.

• ValueError – The optimizer in schedulers is different from the optimizer of schedulers[0].

• ValueError – Length of milestones is not equal to length of schedulers minus 1.

Supported Platforms:

Ascend GPU CPU

Examples

>>> from mindspore.experimental import optim
>>> from mindspore import nn
>>> net = nn.Dense(3, 2)
>>> optimizer = optim.Adam(net.trainable_params(), 0.1)
>>> scheduler1 = optim.lr_scheduler.ConstantLR(optimizer, factor=0.1, total_iters=2)
>>> scheduler2 = optim.lr_scheduler.ExponentialLR(optimizer, gamma=0.9)
>>> scheduler = optim.lr_scheduler.SequentialLR(optimizer, schedulers=[scheduler1, scheduler2], milestones=[2])
>>> for i in range(6):
...     scheduler.step()
...     current_lr = scheduler.get_last_lr()
...     print(current_lr)
[Tensor(shape=[], dtype=Float32, value= 0.01)]
[Tensor(shape=[], dtype=Float32, value= 0.1)]
[Tensor(shape=[], dtype=Float32, value= 0.09)]
[Tensor(shape=[], dtype=Float32, value= 0.081)]
[Tensor(shape=[], dtype=Float32, value= 0.0729)]
[Tensor(shape=[], dtype=Float32, value= 0.06561)]

get_last_lr()

Return last computed learning rate by current scheduler.

step()

Get the current learning rate and change the learning rate.