Observability Summit North America 2026

As large language models (LLMs) move into production, raw metrics alone aren’t enough. This talk presents an open-source AI observability solution built on Open Data Hub (ODH) that deploys LLMs using vLLM and KServe, scrapes inference metrics using Prometheus, and feeds them into a summarization model to generate actionable insights. We’ll demonstrate a working UI that translates low-level metrics like latency, GPU usage, and token throughput into human-readable summaries—giving platform teams an intelligent way to monitor LLMs at scale. No dashboards to interpret—just straight answers from your models about your models.