In this document, VMware walks through the steps to enable InfiniBand SR-IOV on a dual-port Mellanox ConnectX-5 VPI adapter card in vSphere 7.x.
They build on the most current VMware and Mellanox documentation and cover the steps from BIOS, ESXi, and vCenter to the functionality test on the VM guest operating system using ibverbs, Intel cluster checker, and the Ohio State University (OSU) microbenchmark suite.
They also introduce how to use the vHPC toolkit, an open-source tool developed by VMware, to speed up the deployment of an HPC cluster in vSphere.
Finally, they present a performance study of five HPC applications across multiple vertical domains, all concerned with dynamic systems (including manufacturing, weather forecasting, and the life sciences).
They conclude that virtual HPC clusters with VMware vSphere perform nearly as well as bare-metal HPC clusters while offering all the advantages of virtualization with vSphere like increased IT agility, flexibility, scalability, and significant cost savings of hardware.
