The Failed Promises of Data Lineage: Why More Metadata Isn’t the Answer

Description

The data world doesn’t need another lineage graph—it needs understanding. Today’s tools overload teams with metadata but lack context for decisions and action. This session introduces a reasoning layer that unifies OpenLineage, OpenTelemetry, and LLMs to turn raw lineage and telemetry into one continuous, actionable context.​ See how an observability-first Telemetry Lake correlates data movement with system behavior to detect issues, diagnose root causes, and adapt in real time. We’ll cover where lineage fell short, how correlation beats coverage, and why the first 150 characters of your stack’s signal matter most for clarity and response.​ Discover how oleander builds an always-on-call loop that proposes fixes, reduces cost and blast radius, and focuses on the nodes and edges that actually drive impact and business outcomes—without mapping every last dependency. Interested in learning more about Open + AI Lakehouse? Subscribe to the official Luma page: https://luma.com/openlakehouseai 00:00 Introduction to Open Source Data Lineage 03:05 Challenges in Data Lineage 05:24 The Role of OpenLineage 06:52 Understanding Metadata and Its Limitations 10:15 The Evolution of Data Lineage Standards 13:11 Integrating Telemetry with Data Lineage 15:47 The Future of Data Engineering and ML Integration 19:07 Navigating Open Source Projects: Marquez vs Oleander 22:07 Community Involvement and Future Directions