Feb 24
Loading video player...
In this episode, we’re diving into observability for Generative AI apps. AI helps us write code and monitor applications in production - but how do we observe the AI itself? And how do we make sense of complex, non-deterministic AI systems? 🎙️ We’re joined by two great guests: Ishan Jain, working on GenAI observability and Luccas Quadros, working on Grafana Assistant. Together, they bring both platform-level insights and real-world perspectives. 🚀 We’ll explore the fast-evolving space of observability for generative AI apps, including: - How non-deterministic responses change the way we think about observability - Recording prompts and completions in telemetry: when it’s useful, when it’s not, and how to do it safely - Using automated evaluations to understand whether things actually work - Monitoring agentic workflows and multi-step reasoning - What OTel GenAI group works on 💬 Join us live for an open conversation! Bring your questions, opinions, and hot takes - it’s going to be a fun one 🤖 TIMESTAMPS 00:00:00 Introductions and Overview of the Call 00:03:15 Guests Introductions: Ishan and Lucas 00:05:00 Announcement: Final Chance to Register for Otel Unplugged 00:06:30 Discussion: What is Generative AI? 00:10:45 The Evolution of AI and Observability Techniques 00:15:30 Challenges in Observing Generative AI Applications 00:20:00 Importance of Latency in AI Interactions 00:25:00 Demo: Observability for AI Agent Execution 00:32:00 Discussing Costs Associated with AI Model Evaluations 00:40:00 Evaluating AI Responses with LLM as Judge 00:50:00 Closing Thoughts and Resources for Getting Started with Observability