We have a small cluster in the lab, around 20 nodes running under a queue system. This is ideal system for analysis and small jobs, meanwhile big jobs, the final step of our projects, run in one of the massive clusters of Compute Canada. Recently, I extended our lab cluster with a few “GPU” machines. This extension covers the gap between prototyping and production of all those jobs accelerated by GPU. However, GPUs are a very expensive hardware, so we want to invest our money wisely to maximize our dollar performance ratio. I feel that provide just a table with the configurations I chose do not have a lot of value because technology has a very fast cycle and any list of hardware is going to be obsolete in a few months, in both value and performance. Consequently, I decide to share what I learned building these computers. At the end TL;DR and 2 configurations to use as a example.
If you are planning to focus in GPU accelerated calculations and your budged is tight, do not invest in a high-end CPU. Obviously is nice to have a powerful CPU in order to speed other day a day process, but in terms of GPU accelerated calculations, specially training a Convoluted neural network, is much important the muscle of your GPU rather the number of cores of your CPU. However, if you plan to build multiple GPUs machine, get a CPU with at least 1 physical core per GPU.
GPU wise, if the money is, again, an important factor to make the final decision, and you plan to do mostly Deep learning then try to get the model with the maximum amount of memory that you can afford. GPU memory will be a limitation factor in terms of size of network and training batches. In terms of performance, multiple GPU are more efficient to run different parameters or different projects simultaneously rather than parallelize the same job using MPI. However, multiple GPU may be necessary if your network and mini-batches are too big for the memory of a single GPU, then you will need to split the job between two or more.
Currently, Nvidia is the standard the facto; they were one of the first to get into the scientific market, and CUDA – Nvidia programming libraries – have more momentum than openCL, the open source alternative. Although RADEON family are a good alternative in terms of performance, they support is , at this moment, very poor and I can not recommend to buy a AMD GPU. Let’s see what will happen in the next years, where other player, like intel , may present alternatives to NVIDIA.
Personally, I prefer to have as much memory I can afford, but if the budget is a problem, try to have at least the same amount as the GPU. Meanwhile the speed of the RAM is not important in terms of accelerated GPU calculations an important bottleneck is the communications of the CPU-RAM and the GPU-RAM, consequently get a motherboard and components (e.g. GPU) supports the latest PCI express version.
A bonus, but not critical, is a SSD disk. This can increase for instance the reading of the mini batches during the training, however, if your budget is tight, you can get a regular disk.
Finally, check the power consumption of your system, GPU are hungry beast in terms of electricity, especially high-end models, a titan Xp can consume almost 250W at full load. The rule of thumb is to get a PSU capable to provide a 20% more than the theoretical usage of your system at full load.
Final tips. PCPartPicker website is quite use full to build computer by parts and keep track in compatibilities and power requirement.
- Get the GPU or GPUs with the highest amount of memory
- Do not invest in CPU or super-fast memory.
- Get one CPU-core per GPU
- Get a modern motherboard; fast PCI bus.
- Try to get a SDD, nut there is life without it.
- 16Gb is enough, but if you can squeeze the budge and afford a little more, do not think it twice, you will not regret it.
- Keep and eye on the power requirements of the system and get a good PSU.
Best budget machine, late-2017
- 1060 6Gb. best value, enough memory to participate in almost all Kaggle challenges)
- AMD Ryzon CPU series have the best value; however, they are not well supported by kernel version earlier than 4.10. A few tips about it, here
- Any gamers mother board that supports PCIe 3.0, e.g. MSI MORTAR
- 16 Gb DDR4 2100M
- 500Gb SSD
- PSU 450W
GPU cluster Node, late-2017
- 3x Nvidia Titan Xp. Multiple GPU to train bigger networks, or run different model/parameters
- Intel Core i7-7700K 4.2Ghz Quad-core
- Asus STRIX Z270-E GAMING ATX LGA1151. Motherboard support up to 4 GPUS
- 64Gb DDR4 2100
- 1Tb SSD
- PSU 1200W