Inference on a GPU

Linux GPU Inference Application Beginner Intermediate Expert

Inference Using a Checkpoint File

The inference is the same as that on the Ascend 910 AI processor.

Inference Using an ONNX File

1. Generate a model in ONNX format on the training platform. For details, see Export ONNX Model.

2. Perform inference on a GPU by referring to the runtime or SDK document. For example, use TensorRT to perform inference on the NVIDIA GPU. For details, see TensorRT backend for ONNX.