Using Huawei MindIE

First, select the PyTorch MindIE image based on your version requirements:

Then refer to Huawei’s documentation: Inference Service Deployment (DeepSeek-R1-Distill-Qwen-7B) - AI Service Deployment - Solution Deployment Guide - RAG Integrated Solution Reference Practice - Kunpeng RAG Development Documentation - Kunpeng Community

Huawei’s documentation streamlines the steps:

1. Prepare the model

2. Edit the /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json file, primarily to modify the model paths and Device Index, among other things. It is also important to set the httpsEnabled field in the config file to false, as certificates are generally not used in this scenario.

3. Load environment variables:

source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh
source /usr/local/Ascend/atb-models/set_env.sh
source /usr/local/Ascend/mindie/set_env.sh
source /usr/local/Ascend/mindie/latest/mindie-service/set_env.sh

1. Start the service

cd /usr/local/Ascend/mindie/latest/mindie-service/bin
./mindieservice_daemon