Mar 18
Loading video player...
Stop treating your LLM applications like a black box. In this third installment of our MLflow for GenAI series, Jules Damji (Databricks) dives deep into MLflow Tracing—the essential tool for debugging, monitoring, and operationalizing complex AI workflows. While traditional logging tells you that something happened, MLflow Distributed Tracing tells you how and why it happened, and where it happened.. Whether you are building simple RAG pipelines or complex agentic workflows, visibility into every span or operation is the key to moving from prototype to production. What You’ll Learn: 🔹 MLflow Tracing Fundamentals: Understanding Traces vs. Spans and why traditional logging isn't enough for LLMs. 🔹 The MLflow Trace Data Model: Deep dive into Span types like LLM, Retriever, Tool, , Agent, and Embedding. 🔹 AI Observability: How to use mlflow.openai.autolog() and mlflow.langchain.autolog() to capture latency, token usage, and costs instantly. 🔹 Debugging with AI: Using the MLflow Assistant to automatically diagnose failed API calls, schema errors, and pipeline bottlenecks. 🔹 Best Practices: Why you should enable tracing early in the dev cycle and how to combine it with Experiment Tracking. Resources: 🔗 Notebook 1.3: https://github.com/dmatrix/mlflow-genai-tutorials/blob/main/03_introduction_to_tracing.ipynb 🔗 Full Series Playlist: https://www.youtube.com/playlist?list=PLaoPu6xpLk9EI99TuOjSgy-UuDWowJ_mR 🔗 MLflow Documentation: https://mlflow.org/docs/latest/genai/tracing/