Overview

The HPC environment became available to MCW researchers in March 2021. The cluster consists of 79 compute nodes, 4,200 CPU cores, and 96 GPUs. The cluster is connected by 7 100 Gbps switches running RoCEv2 (ethernet equivalent to Infiniband). Additionally, a 467 TB NVMe provides scratch storage, and a 2.6 PB scale-out NAS provides persistent storage.

Cluster

Detailed information is available below. Please note, the table is wide and might require side scrolling to view all data.

Nodes	Type	Cores/node	Mem/node (Gb)	Disk/node (Gb)	GPUs/node	Sockets/node	Cores/socket	Threads/core	CPU Vendor	CPU Model	CPU Base Freq (GHz)	CPU Turbo Freq (GHz)	GPU Vendor	GPU Model	GPU Mem (Gb)
60	CPU	48	384	440		2	24	1	Intel	6240R	2.4	4
6	GPU	48	384	440	4	2	24	1	Intel	5220R	2.2	4	NVIDIA	Tesla V100	32
2	GPU	48	512	440	4	2	24	1	Intel	6336Y	2.4	3.6	NVIDIA	Ampere A40	48
1	GPU	40	512	7000	8	2	20	1	Intel	E5-2698 v4	2.2	3.6	NVIDIA	Tesla V100 SXM2	32
2	GPU	128	750	7000	8	2	64	1	AMD	EPYC 9554	3.1	3.75	NVIDIA	Ada Lovelace L40S	48
2	Large Mem	48	1536	440		2	24	1	Intel	6240R	2.4	4

Condo hardware

Condo nodes are factored into the overall cluster metrics, but specific hardware details for condo systems are not listed in the table.