Multi-machine, multi-GPU parallel processing

Given that GPUs other than the A100 and similar models lack IB network and NVLink hardware support, multi-machine parallel processing is less efficient than single-machine parallel processing; therefore, we no longer support enabling internal IP addresses for multi-machine, multi-card parallel processing

If your computing needs can be met with a single-machine, multi-GPU setup, we highly recommend this approach (multi-machine parallel computing incurs significant network overhead, and its parallel efficiency is far lower than that of a single-machine, multi-GPU setup).For multi-GPU on a single machine, simply rent multiple GPUs within the same instance. For instances that are already running, you can change the number of GPUs by shutting down the instance and then scaling up or down; see Scaling.