Auditi: Open Source LLM Observability - Trace & Evaluate AI Agents in 2 Lines of Code

Description

Auditi is an open source LLM observability and evaluation platform. Trace, evaluate, and improve your AI agents with just 2 lines of Python. 📦 Quick Start pip install auditi import auditi auditi.init(base_url="http://localhost:8000") auditi.instrument() # Every LLM call is now automatically traced ✅ Key Features • Auto-instrumentation for OpenAI, Anthropic, and Google • Built-in LLM-as-Judge evaluators (relevance, faithfulness, toxicity) • Human annotation workflows with structured scoring • Real-time cost tracking across all providers • Dataset generation from production traces • Self-hostable with Docker Compose - your data stays with you 🆚 Why Auditi? Unlike LangSmith (closed source) or Langfuse (complex), Auditi combines tracing AND evaluation in one open source tool. Get production-ready monitoring without vendor lock-in. #llm #aiagents #opensource #observability #machinelearning #python #openai #anthropic #langchain #devtools