Agentic AI
How to Evaluate Agents Locally with Arize Phoenix
A hands-on guide to running Phoenix locally: trace an agent over OpenTelemetry, then score it with LLM-as-judge and code evaluators.
7 min read
Deep explorations of agentic AI, RAG architectures, and production engineering.
64 articles
A hands-on guide to running Phoenix locally: trace an agent over OpenTelemetry, then score it with LLM-as-judge and code evaluators.
AgentOps is the practice of running agents in production: observability, evaluation, guardrails, cost governance, and human oversight.
OpenTelemetry in plain terms: the data model, the GenAI semantic conventions, and why building agent tracing on OTel avoids lock-in.
What traces and spans actually are, how they map onto an agent run, and how to walk one to find the exact step that broke.
A tour of the reasoning research: how prompting elicits it, how researchers categorize the techniques, and why one prominent position paper says it is not reasoning at all.
Most prompting advice transfers across models. The parts that do not will quietly break your app when you switch providers. A working map of the differences.
A production case study measured fourteen prompt modifications one by one. The results are humbling: naming the model helped, few-shot examples hurt, and templates cost accuracy.
Prompt functions wrap a prompt in a name, inputs, and a rule so you can call and compose it like code. The chat-window version is dated; the idea runs everything now.
Code generation is the most-used LLM capability and the most casually prompted. A tour of the core patterns, plus the checks that catch what the model quietly gets wrong.