Distributed Parallel

Distributed Parallelism Overview
Distributed Parallel Startup Methods
Data Parallel
Semi-automatic Parallel
Automatic Parallel
Manually Parallelism
Parameter Server
Model Saving and Loading
Fault Recovery
Optimization Techniques
Experimental Characteristics
Distributed High-Level Configuration Case

Custom Operator

Custom Operators (Custom-based)
MindSpore Hybrid Syntax Specification
Custom Operator Registration
Advanced Usage of aot-type Custom Operators

Performance Optimization

Profiling↗
Sinking Mode
Enabling Graph Kernel Fusion
Memory Reuse

Algorithm Optimization

Gradient Accumulation Algorithm
Second-order Optimization
- Introduction to Second-order Optimizer THOR
- Applying Second-Order Optimization Practices on the ResNet-50 Network

High-level Functional Programming

Automatic Vectorization (Vmap)
Computing Jacobian and Hessian Matrices Using Functional Transformations
Per-sample-gradients

Data Processing

Auto Augmentation
Single-Node Data Cache
Optimizing the Data Processing

Model Inference

Inference Model Overview
Model Compression

Complex Problem Debugging

Using Dump in the Graph Mode
Ascend Optimization Engine (AOE)
Running Data Recorder
Fault Recovery
Accuracy-Sensitive Detection

MindSpore

»
Second-order Optimization
View page source

Second-order Optimization

Introduction to Second-order Optimizer THOR
Applying Second-Order Optimization Practices on the ResNet-50 Network

Previous Next

© Copyright MindSpore.

Built with Sphinx using a theme provided by Read the Docs.