GPU Concepts and Options

GPU Concepts and Options#

KLC login nodes are CPU-only. For workloads that benefit from GPU acceleration — especially large language models and deep learning — use KLC Reserve GPU nodes via SLURM on the kellogg partition.

This page explains concepts and options. For submission scripts and --gres syntax, see GPU Jobs.

CPU vs. GPU: Key Concepts#

CPU (Central Processing Unit)#

A CPU handles mathematical and logical calculations on a node. CPU cores are powerful but run tasks sequentially — one at a time per core. KLC’s latest nodes have 64 CPU cores and up to 2 TB of shared RAM, which means you can run up to 64 parallel processes on a single node when using Reserve.

GPU (Graphics Processing Unit)#

A GPU is a specialized processor designed for massively parallel mathematical operations. Where you can use 24 CPU cores on KLC login nodes under normal priority, a single A100 GPU contains 6,912 CUDA cores (the H100 has 18,432). Individual GPU cores are less powerful than CPU cores, but their number makes GPUs ideal for the vector and matrix operations at the heart of LLM inference and training.

Note

GPUs are not universally faster. Some tasks cannot be parallelized (serial dependencies), and for small jobs the coordination overhead can make a GPU slower than a CPU. Use GPUs when your workload is inherently parallel — matrix multiplications, batch inference, training neural networks.

CUDA (Compute Unified Device Architecture)#

CUDA is Nvidia’s software platform that routes computations to GPU cores vs. CPU cores. You will not typically write CUDA code directly. Most researchers use high-level libraries — like Hugging Face Transformers or PyTorch — that call CUDA automatically.

The CUDA software stack looks like:

Your code (Python / Hugging Face / Ollama)
       ↓
PyTorch / TensorFlow
       ↓
CUDA
       ↓
GPU Hardware

When setting up your environment, ensure your CUDA version, PyTorch/TensorFlow version, and GPU driver are compatible.

GPU Options at Northwestern#

Kellogg GPU nodes (KLC Reserve)#

  • Dedicated GPU nodes on the kellogg partition with --account=kellogg

  • Three node configurations — L40S, A100, and H100 (see GPU Jobs)

  • Included with a KLC account; no separate Quest allocation required

  • Default path for Kellogg-affiliated researchers — develop and test GPU workflows here before scaling out

Quest General Access GPUs (gengpu)#

  • Northwestern-wide GPU pool on the gengpu partition (A100 and H100; no L40S on this path)

  • Requires a separate Quest allocation , not just a KLC account

  • Use when the job needs GPUs at scale — more cards or more concurrent GPU jobs than the Kellogg Reserve pool can provide

  • Do not start here. Prove the workflow on KLC Reserve first (same Quest file systems and modules), then change partition and account for the scaled run

  • How-to: GPUs on Quest . For Kellogg --gres syntax, see GPU Jobs

Other options#

  • Google Colab — browser-based GPU notebooks

  • AWS, Google Cloud, Microsoft Azure — cloud GPUs (usage-based cost)

Choosing a Path#

Need

Recommended path

Kellogg researcher, develop or test a GPU workflow

KLC Reserve — kellogg partition

Workflow is working; need more GPUs than Reserve can provide

Quest General Access — gengpu partition

Sensitive data must stay on-cluster, no GPU needed

Open Source LLMs on KLC on CPU login nodes

Sensitive data, GPU needed

KLC Reserve GPU nodes

Quick experiment, no cluster setup

Google Colab or cloud (check data governance first)

Test on KLC Reserve, then use gengpu when you need GPUs at scale. Contact rs@kellogg.northwestern.edu if you need guidance on which allocation or partition fits your project.

Open-Source LLMs on CPU vs. GPU#

For the CPU-vs-GPU decision specifically for running open-source LLMs, see the “Choose CPU or GPU” table on Open Source LLMs on KLC.