Skip to content

The lab is in beta. Found something wrong?Beta · Found something wrong?

AI Data Centre · Level 7 of 8

Training job slowdown

Level 7 of 8 in the AI Data Centre: a free data centre exercise at Advanced level. A large training run is slow. Read the utilisation, temperatures and link counters, find the bad optic or the throttling GPU, and fix it without stopping more than you must.

Start this level

Free · no download. Your work is saved on this device. Updated 18 Oct 2026.

3D view of the Training job slowdown level in the AI Data Centre
Training job slowdown

The brief

Client
Northfield Compute
Site
Hall 2, Greenmoor campus
Work order
WO-DC-107
Task code
DC-07
Difficulty
Advanced
Type of work
Fault-finding

A large training run is slow. Read the utilisation, temperatures and link counters, find the bad optic or the throttling GPU, and fix it without stopping more than you must.

Skills you practise

  • GPU utilisation and temperature
  • Link error counters
  • Spine-leaf fault isolation
  • Draining a node for maintenance

What the test checks

The acceptance test marks the skills this level practises.

  • GPU utilisation and temperature
  • Link error counters
  • Spine-leaf fault isolation
  • Draining a node for maintenance

In this lab

  • Rack and cable GPU servers
  • Power budgets and cooling
  • GPU jobs and a DCIM view

Levels in the AI Data Centre

Previous: Replace a failed part from a ticketNext: Power event drill

  1. Safety walk and hall tour
  2. Rack and cable a GPU server
  3. Rack power budget
  4. Cooling hot spot
  5. PUE from a month of meters
  6. Replace a failed part from a ticket
  7. Training job slowdown
  8. Power event drill

Part of the AI Data Centre

Walk an AI data hall, rack and cable a GPU server, and size its power and cooling.

About the AI Data Centre

Feeds these internships: AI and Machine Learning, Data Science and Analytics, AI Agents and Automation, Electrical Design.