Feb 25
Loading video player...
This talk was recorded at NDC London in London, England. #ndclondon #ndcconferences #developer #softwaredeveloper Attend the next NDC conference near you: https://ndcconferences.com https://ndclondon.com/ Subscribe to our YouTube channel and learn every day: / @NDC Follow our Social Media! https://www.facebook.com/ndcconferences https://twitter.com/NDC_Conferences https://www.instagram.com/ndc_conferences/ #ai #cloud #devops Logs and metrics are great—until you're trying to debug an AI agent that just replied “I'm sorry, Dave, I can’t do that.” Observability in AI systems isn't just about uptime—it's about understanding prompts, latencies, retries, hallucinations, token usage, and model behavior under pressure. This session dives into how we used OpenLit, a lightweight CNCF tool for observing LLM and agent-based workloads. We’ll explore how we tracked & debugged prompt chains across orchestration layers, visualize token usage and latency distributions at each step of an AI workflow, monitor agent reasoning patterns and build performance dashboards that go beyond CPU and memory into model selection, response variability, and input size trends. Whether you're building RAG systems, multi-step AI agents, or just trying to figure out why your chatbot is always apologizing, this talk shows how OpenLit brings real observability into your AI stack — without needing a PhD in tracing or LLM internals.