Observability Summit North America 2026

As large language models scale across hundreds of GPUs and multi-node AI systems, they’ve become a major operational challenge for infrastructure engineers. Traditional observability tools stop at the node level, leaving GPU health and utilization invisible until workloads fail or budgets spike. Imagine a 25 day training job failing on day 23 because one GPU silently throttled!

In this session, we’ll explore GPU-Scanner, an open-source observability extension for Kubernetes GPU clusters. Built to integrate with Prometheus & OpenTelemetry, GPU-Scanner adds both active and passive GPU health checks, capturing throughput, TFLOPs, memory diagnostics, thermal consistency, and long-run stability metrics.

We’ll demo real-world failure modes like catching a GPU “off the bus” or detecting thermal throttling and show how alerts flow into your existing observability stack. Leave with a practical playbook to proactively validate GPU clusters and maximize reliability and utilization.