Huawei Ascend NPU

Please note that the Huawei [Ascend 910B2x Kunpeng 920] currently running on WebCal is equipped with an ARM system; therefore, images you previously built on x86 CPU machines cannot be used on this machine.

Using PyTorch

First, we recommend creating an instance using one of our pre-built base images that includes the PyTorch framework. Then, for NVIDIA-based PyTorch code, just a few simple modifications are needed to run it on the Huawei 910B. Please see the example below:

# NVIDIA PyTorch 代码
import torch

x = torch.randn(10000, 10000).cuda()
y = torch.randn(10000, 10000).cuda()
for _ in range(1000):
    z = x.mm(y)
print(z)


# 改造为华为910B上可运行的代码
# 第一步：import torch后，立即import torch_npu
# 第二步：cuda()函数换为npu()
import torch
import torch_npu

x = torch.randn(10000, 10000).npu()
y = torch.randn(10000, 10000).npu()
for _ in range(1000):
    z = x.mm(y)
print(z)

More Complex Examples

The following is the code for training a NVIDIA PyTorch ResNet network using the MNIST dataset

# 下载代码
wget https://webcal-public.ks3-cn-beijing.ksyuncs.com/npu_chips/huawei/dp_res18_ascend.py

# 执行代码训练，自动会下载MNIST数据集
python dp_res18.py

Below is the code modified from the above to support the Huawei 910B.

# 下载代码
wget https://webcal-public.ks3-cn-beijing.ksyuncs.com/debug/dp_res18_ascend.py

# 执行代码训练，自动会下载MNIST数据集
python dp_res18_ascend.py

Differences between the two code snippets:

# diff dp_res18.py dp_res18_ascend.py
8a9
> import torch_npu
29c30,31
< device = 'cuda' if torch.cuda.is_available() else 'cpu'
---
> device = 'npu' if torch.npu.is_available() else 'cpu'
> print("Detect device:", device)


# 从diff两个文件的结果可以看出，除多了import torch_npu和替换了cuda→npu，其他均相同

Ascend NPU Utilization Monitoring

View NPU utilization, GPU memory usage, and more

Use the npu-smi info command, where AICore(%) is the GPU utilization.

# npu-smi info
+------------------------------------------------------------------------------------------------+
| npu-smi 23.0.3                   Version: 23.0.3                                               |
+---------------------------+---------------+----------------------------------------------------+
| NPU   Name                | Health        | Power(W)    Temp(C)           Hugepages-Usage(page)|
| Chip                      | Bus-Id        | AICore(%)   Memory-Usage(MB)  HBM-Usage(MB)        |
+===========================+===============+====================================================+
| 6     910B2               | OK            | 98.7        46                0    / 0             |
| 0                         | 0000:82:00.0  | 0           0    / 0          3330 / 65536         |
+===========================+===============+====================================================+
+---------------------------+---------------+----------------------------------------------------+
| NPU     Chip              | Process id    | Process name             | Process memory(MB)      |
+===========================+===============+====================================================+
| No running processes found in NPU 6                                                            |
+===========================+===============+====================================================+

Real-time Monitoring of NPU Usage

Use the npu-smi info watch command

# npu-smi info watch
NpuID(Idx)  ChipId(Idx) Pwr(W)      Temp(C)     AI Core(%)  AI Cpu(%)   Ctrl Cpu(%) Memory(%)   Memory BW(%)
6           0           98.0        44          0           0           0           5           0
6           0           97.9        44          0           0           0           5           0

Notes

When using Huawei’s NPU, there is a key difference compared to mounting NVIDIA GPU devices to container instances. For example, if there are 8 GPUs on the host machine,and you rent 2 of them, those 2 GPUs—regardless of which 2 out of the 8 they are—will always be assigned INDEX numbers 0 and 1 (both starting from 0) in the user’s instance. However, when using Huawei NPUs, the INDEX numbers displayed for the cards within the container correspond to those on the host machine (and are not necessarily 0 and 1),However, when using Huawei NPU cards in code, the INDEX is determined by incrementing from 0 to identify which NPU it is—that is, if N cards are leased, each card’s index is 0, 1, 2, ...,N-1. Do not rely on the NPU INDEX displayed in npu-smi info