torch.optim

Base class

API名称	API状态	限制与说明
torch.optim.Optimizer	Not Support	N/A
Optimizer.add_param_group	Not Support	N/A
Optimizer.load_state_dict	Not Support	N/A
Optimizer.state_dict	Not Support	N/A
Optimizer.step	Not Support	N/A
Optimizer.zero_grad	Not Support	N/A

Algorithms

API名称	API状态	限制与说明
torch.optim.Adadelta	Not Support	N/A
torch.optim.Adagrad	Not Support	N/A
torch.optim.Adam	Beta	入参不支持foreach、capturable、differentiable、fused；支持数据类型：bf16、fp16、fp32
torch.optim.AdamW	Beta	入参不支持foreach、capturable、differentiable、fused；支持数据类型：bf16、fp16、fp32
torch.optim.SparseAdam	Not Support	N/A
torch.optim.Adamax	Not Support	N/A
torch.optim.ASGD	Not Support	N/A
torch.optim.LBFGS	Not Support	N/A
torch.optim.NAdam	Not Support	N/A
torch.optim.RAdam	Not Support	N/A
torch.optim.RMSprop	Not Support	N/A
torch.optim.Rprop	Not Support	N/A
torch.optim.SGD	Beta	入参不支持foreach、differentiable；参数lr具有默认值1e-3，torch无默认值；支持数据类型：bf16、fp16、fp32

How to adjust learning rate

API名称	API状态	限制与说明
torch.optim.lr_scheduler.LambdaLR	Not Support	N/A
torch.optim.lr_scheduler.MultiplicativeLR	Not Support	N/A
torch.optim.lr_scheduler.StepLR	Not Support	N/A
torch.optim.lr_scheduler.MultiStepLR	Not Support	N/A
torch.optim.lr_scheduler.ConstantLR	Not Support	N/A
torch.optim.lr_scheduler.LinearLR	Not Support	N/A
torch.optim.lr_scheduler.ExponentialLR	Not Support	N/A
torch.optim.lr_scheduler.PolynomialLR	Not Support	N/A
torch.optim.lr_scheduler.CosineAnnealingLR	Not Support	N/A
torch.optim.lr_scheduler.ChainedScheduler	Not Support	N/A
torch.optim.lr_scheduler.SequentialLR	Not Support	N/A
torch.optim.lr_scheduler.ReduceLROnPlateau	Not Support	N/A
torch.optim.lr_scheduler.CyclicLR	Not Support	N/A
torch.optim.lr_scheduler.OneCycleLR	Not Support	N/A
torch.optim.lr_scheduler.CosineAnnealingWarmRestarts	Not Support	N/A