Advanced MLflow Tracing: Manual Spans, RAG, and Agentic Workflows (Notebook 1.4)

Description

In the previous tutorial on MLflow tracing, we explored how MLflow’s auto-logging handles standard OpenAI and LangChain calls. But as your GenAI applications grow into complex RAG pipelines and Agentic workflows, you need more than just what autologging can capture. You need Manual Tracing, with granular ability to capture operations that autologging cannot. In Notebook 1.4, Jules Damji moves beyond the defaults. You will learn how to use the @mlflow.trace decorator to instrument custom business logic, internal API calls, and domain-specific functions or tool usage that auto-logging can't see. By the end of this video, you'll be able to "light up" every step of your AI's agent decision-making process. What You’ll Learn: 🔹 Manual Tracing Strategy: When to move beyond auto-logging for custom functions and external services. 🔹 The MLflow Trace Decorator: How to wrap your Python code to define granular, searchable spans. 🔹 The Common Core Span Types: Mastering the use of Agent, Retriever, Tool, Embedding, Chat, Chain, Reranker, Memory, and Parser labels. 🔹 RAG Observability: Building a hierarchical trace that tracks everything from query parsing to vector retrieval. 🔹 Agentic Instrumentation: Capturing the "why" behind tool selection and parameter usage in autonomous loops. 🔹 Custom Attributes: Attaching metadata like cache hits, versioning, and performance metrics to your traces. Resources: Notebook 1.4: https://github.com/dmatrix/mlflow-genai-tutorials/blob/main/04_manual_tracing_advanced.ipyn Full MLflow for GenAI Playlist: https://www.youtube.com/playlist?list=PLaoPu6xpLk9EI99TuOjSgy-UuDWowJ_mR