Apr 12
Loading video player...
Arize AX Enablement session on building evaluators! In this hands-on session, we show you how to create LLM-as-a-Judge and code-based evaluators, scope them across spans, traces, and sessions, and apply filters to evaluate on exactly the data that matters. Designed for both technical and non-technical attendees - this session focuses on real examples that make evaluators approachable and actionable for everyone. Learn how OpenTelemetry tracing works for AI apps—spans, traces, sessions, exporters, span processors, and tracer providers—and how OpenInference semantic conventions + auto-instrumentation make LLM/agent workflows observable in Arize AX. We close with advanced topics like context propagation, sampling, and OTEL collectors. 0:00 Intro 0:49 Why observability (trust + silent failures) 4:07 OpenTelemetry overview 7:17 OpenInference overview 8:21 Signals (traces, metrics, & logs) 10:15 Span structure 16:31 OTel components (resource/exporter/processor/provider) 18:17 Resource 21:48 Exporters 31:32 Span processors 35:05 Tracer provider 38:53 Register function (arize.otel) 39:24 ArizeRoutingProcessor 41:25 OpenInference Semantic Conventions 44:59 How to Instrument 45:51 Autoinstrumentation 46:58 Manual instrumentation 51:17 OpenInference Context Managers 52:15 Hybrid instrumentation 53:54 Custom Span Processors 57:26 Context propagation 1:05:36 Sampling 1:12:43 Collector Speakers: Sofia Jakovcevic: https://www.linkedin.com/in/sofia-jakovcevic/ Lucas Moehlenbrock: https://www.linkedin.com/in/lucas-moehlenbrock-b42174130/ Arize docs: Tracing overview: https://arize.com/docs/ax/observe/tracing Quickstart tracing: https://arize.com/docs/ax/quickstarts/quickstart-tracing OpenTelemetry integration (arize-otel): https://arize.com/docs/ax/integrations/opentelemetry/opentelemetry-arize-otel OpenInference helpers: https://arize.com/docs/ax/observe/tracing/configure-tracing-options/instrument-with-openinference-helpers Advanced OTEL examples: https://arize.com/docs/ax/observe/tracing/configure/advanced-tracing-otel-examples Arize community events: https://arize.com/community/