background

so, I'm director of scientific computing at a cancer center and have just received some interesting news: a new lab has between a $300-400K budget to pay for GPU hardware to run their AlphaFold 2 workloads. some of the pertinent characteristics: they need a fair amount of VRAM per job, but not that much; they start to hit diminishing returns after around 50GB. it is very compute intensive but not very data intensive; input data is small and output data is even smaller (literally just some text files). caveat: they end up having millions of tiny files in a fairly varied directory tree, so IOPS for storage will end up being a thing to think about (this is probably fine; we can script around this with prolog and epilog scripts moving data in and out of node-local storage and in/out of our HPC Isilon).

what we currently have

a 70-node cluster of Dell R7525s, 3rd-gen EPYC, 1TB DDR4 RDRAM per node, one (1) A30 per node (can you tell this machine was built shortly before the AI boom?). we do have a project on the docket to have the A30s pulled out of 35 of the nodes and installed next to the other half on the same NUMA node, such that we have 35 nodes with two A30s and 35 empty nodes. each node has 3 available PCIe slots, two on CPU 1 and one on CPU 0. I've run a bunch of tests with Qwen3.8-27B running in TP=2 across two of the cards on CPU 1 and got very nice numbers back, so I know that's a usable strategy for inference. haven't tried Strata yet although it's on the agenda. fairly vanilla system otherwise, Slurm 24.something, Bright 10.something, somewhat shitty networking (2x25Gb LACP to each node). networking is a planned upgrade next year to at least 400GbE.