Frequently Asked Questions

Model-related Issues

Git-LFS Installation

1. Obtain the corresponding git-lfs installation package from the following link.

2. Download and install:

   mkdir git-lfs
   cd git-lfs
   wget https://github.com/git-lfs/git-lfs/releases/download/v3.0.1/git-lfs-linux-arm64-v3.0.1.tar.gz --no-check-certificate
   tar zxvf git-lfs-linux-arm64-v3.0.1.tar.gz
   bash install.sh
   

3. Verify successful installation:

   git lfs install
   

   If Git LFS initialized. is returned, the installation was successful.

Deployment-related Issues

Model Fails to Load During Offline/Online Inference

• Key error message:

  raise ValueError(f"{config.load_checkpoint} is not a valid path to load checkpoint ")
  

• Solution:

  1. Check if the model path exists and is valid;

  2. If the model path exists and the model files are in safetensors format, confirm whether the yaml file contains the load_ckpt_format: "safetensors" field:

     1. Print the path of the yaml file used by the model:

        echo $MINDFORMERS_MODEL_CONFIG
        

     2. Check the yaml file. If the load_ckpt_format field is missing, add it:

        load_ckpt_format: "safetensors"
        

aclnnNonzeroV2 Related Error When Starting Online Inference

• Key error message:

  RuntimeError: Call aclnnNonzeroV2 failed, detail:E39999: Inner Error
  

• Solution: Check whether the CANN and MindSpore versions are correctly matched.

torch Not Found When Importing vllm_mindspore

• Key error message:

  importlib.metadata.PackageNotFoundError: No package metadata was found for torch
  

• Solution: Execute the following commands to uninstall torch-related components:

  pip uninstall torch
  pip uninstall torchvision
  

import vllm_mindspore and raise ImportError

ImportError: cannot import name 'PlacementGroupSchedulingStrategy'

• Key error message:

  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/home/miniconda3/envs/py39/lib/python3.9/site-packages/vllm_mindspore/__init__.py", line 236, in <module>
      from vllm_mindspore.executor.ray_gpu_executor import (
    File "/home/miniconda3/envs/py39/lib/python3.9/site-packages/vllm_mindspore/executor/ray_gpu_executor.py", line 27, in <module>
      from vllm.executor.ray_distributed_executor import (
  ImportError: cannot import name 'PlacementGroupSchedulingStrategy' from 'vllm.executor.ray_distributed_executor' (/home/miniconda3/envs/py39/lib/python3.9/site-packages/vllm_mindspore/executor/ray_gpu_executor.py)
  

• Solution: ray module is needed. Please install ray with pip install ray, or run the following command to install modules.

  pip install -r requirements.txt