Analysis of Server Performance Metrics

The following are common factors that affect server performance in GPU scenarios: 1. CPU clock speed and number of CPU cores 2. Which generation of PCIe the motherboard supports, and what the PCIe bandwidth is (very important, yet often overlooked) 3. Number of memory channels and memory clock speed. These are explained in more detail below:

1. CPU Clock Speed and Number of CPU Cores

First, the higher the clock speed, the better, and the more cores, the better.However, CPUs that balance both of these factors tend to be quite expensive, such as AMD’s Threadripper series and Intel’s W-series (though both fall under the category of desktop CPUs). While these offer excellent performance, they are generally used only in personal desktop computers and workstations; therefore, if you’re building a workstation, you should prioritize these types of CPUs.For servers, server-specific CPUs are generally the preferred choice. Server CPUs typically have lower clock speeds than desktop CPUs, so for many applications that can only utilize a single core, using a desktop CPU will actually be faster. (A brief explanation of Intel server CPUs: For Intel CPUs such as the 8225C, 6330, and 8358P, the second digit indicates which generation of PCIe the CPU supports. A “2” indicates support for PCIe 3.0, while a “3” indicates support for PCIe 4.0.)

2. Which generation of PCIe does the motherboard support, and what is the PCIe bandwidth?

The question “Which generation of PCIe does the motherboard support?” is relatively straightforward; generally, you can determine this by checking the CPU model. For differences in PCIe bandwidth across generations, please refer to: https://en.wikipedia.org/wiki/PCI\_Express The question “What is the PCIe bandwidth?” cannot be easily determined at a glance, nor can it be easily identified even within the operating system. An explanation follows: You can use the following command to check

# lspci -vvv | grep -i nvidia -A 25 | grep LnkSta

LnkSta: Speed 2.5GT/s (downgraded), Width x16 (ok)
LnkSta: Speed 2.5GT/s (downgraded), Width x16 (ok)
LnkSta: Speed 2.5GT/s (downgraded), Width x16 (ok)
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
LnkSta: Speed 2.5GT/s (downgraded), Width x16 (ok)
LnkSta: Speed 2.5GT/s (downgraded), Width x16 (ok)

The information above at Width x16 indicates that the available PCIe bandwidth for this GPU card is x16. The 2.5 GT/s listed before it represents the data rate. According to the PCIe 4.0 x16 specification, the theoretical data rate should be 16 GT/s. The reason several cards listed above are shown as 2.5 GT/s is that they are not in use and are in sleep mode. Based on the above explanation, the following check sequence can be summarized: 1. Check the PCIe generation. For example, the speed of PCIe 3.0 x16 is half that of PCIe 4.0 x16 (you can also determine the PCIe generation based on the speed and bandwidth displayed by the commands above). 2. Check the PCIe bandwidth available to the GPU. X16 is excellent; X8 is typical. If it’s lower than X8, performance will be slower.

In addition to the points mentioned above, there is another scenario that can be difficult to distinguish (a common pitfall).Some barebone systems come with a PCIe switch (which generally allows multiple PCIe devices to exchange data directly through the switch). If this PCIe switch has X16 bandwidth and four GPUs are installed on it (the typical setup), the commands mentioned above will show that each card has excellent PCIe performance—all at X16—but in reality, the four cards are sharing the X16 bandwidth. Consequently, on systems with this configuration, tests using NVIDIA’s P2P utility will also yield excellent results because the P2P test measures bandwidth by having two cards communicate at a time; however, in actual use, all eight cards may be operating in parallel, at which point the bandwidth becomes severely constrained.

3. Number of Memory Channels and Memory Clock Speed

Although the operating system treats the number of memory modules installed as a single, unified address space—and this does not affect normal system operation—it does significantly impact performance.For example, 2 x 32GB of RAM is faster than 1 x 64GB because dual-channel mode is enabled. Similarly, on motherboards with dual CPUs, there is affinity between the CPUs and memory slots, so the specific slot where the memory modules are installed matters.In general, to achieve better performance, it is recommended to install as many memory modules as the motherboard has slots—this will yield optimal performance. The higher the memory clock speed—within the limits set by the motherboard—the better. However, the maximum supported memory clock speed depends on the CPU and motherboard. For example, a PCIe 4.0 server only supports memory up to 3200 MHz. If you want to use memory with a higher clock speed, you’ll need to purchase a CPU and motherboard that support PCIe 5.0.

Summary

In GPU-based scenarios, if you are only concerned with the performance of single-GPU computing (note that this does not refer to a machine with only one GPU, but rather independent computations performed on a single GPU at a time, without data exchange between multiple GPUs), the approach is relatively straightforward: the key is to select a GPU with a high clock speed and a large number of cores.If you frequently use multi-GPU parallel computing, you need to determine whether your GPUs support peer-to-peer (P2P) communication between cards (as a general rule, professional-grade GPUs support this, while gaming GPUs do not).If P2P communication is supported, the GPU’s communication bandwidth can generally achieve PCIe peak performance; if P2P communication is not supported, data will be routed through host memory as an “intermediate,” so in this scenario, special attention must be paid to the memory configuration mentioned above—otherwise, it will be difficult to achieve PCIe peak performance.