Agentic AI
How to Evaluate Agents Locally with Arize Phoenix
A hands-on guide to running Phoenix locally: trace an agent over OpenTelemetry, then score it with LLM-as-judge and code evaluators.
7 min read
39 articles
A hands-on guide to running Phoenix locally: trace an agent over OpenTelemetry, then score it with LLM-as-judge and code evaluators.
OpenTelemetry in plain terms: the data model, the GenAI semantic conventions, and why building agent tracing on OTel avoids lock-in.
What traces and spans actually are, how they map onto an agent run, and how to walk one to find the exact step that broke.
A tour of the reasoning research: how prompting elicits it, how researchers categorize the techniques, and why one prominent position paper says it is not reasoning at all.
Most prompting advice transfers across models. The parts that do not will quietly break your app when you switch providers. A working map of the differences.
A production case study measured fourteen prompt modifications one by one. The results are humbling: naming the model helped, few-shot examples hurt, and templates cost accuracy.
Prompt functions wrap a prompt in a name, inputs, and a rule so you can call and compose it like code. The chat-window version is dated; the idea runs everything now.
Code generation is the most-used LLM capability and the most casually prompted. A tour of the core patterns, plus the checks that catch what the model quietly gets wrong.
Cranking temperature does not make synthetic data diverse. Injecting randomness into the prompt does. Here is the technique, with code.
Most agents fail for boring, fixable reasons. Here are the practices that make an agent reliable: fewer LLM calls, tools that explain themselves, and a debugging ladder to climb when it misbehaves.
You can prompt a model to produce labeled examples on demand: test fixtures, eval sets, cold-start training data. Here is how to do it well and where the sharp edges are.
Your few-shot examples carry hidden biases. The distribution and order of them can nudge the model toward a label before it reads the input. Here is how to spot it and fix it.
Models hallucinate because they are built to sound coherent, not to be right. Here are three prompt-level moves that reduce made-up answers, plus where prompting stops working.
Prompt injection, prompt leaking, and jailbreaking are three different attacks with three different fixes. Here is how each one works and what defence actually holds.
Graph prompting feeds a model structure, not just prose: entities as nodes, relationships as edges. Here is what the idea is, where it comes from, and when it pays off.
smolagents ships two agent classes that act in completely different ways: one writes Python, one emits structured JSON. Here is how each works, what each is good and bad at, and when to reach for which.
Reasoning models do the chain-of-thought for you. That flips a lot of prompting advice on its head, and getting it wrong wastes tokens and quality.
Multimodal CoT extends step-by-step reasoning to text plus vision with a two-stage split: build a rationale from both modalities, then answer from the rationale.
Chain-of-Thought lets a model reason and compute in the same breath, which is where it slips. PAL keeps the reasoning in language but hands the math to a Python interpreter.
Directional Stimulus Prompting trains a small model to produce hints that nudge a big frozen LLM toward what you want. It is prompt optimization when you cannot touch the weights.
Hand-picking chain-of-thought examples is a guess. Active-Prompt uses model disagreement to find the questions where a human annotation actually moves the needle.
ART made a frozen LLM generate its own multi-step reasoning, pause to call tools, and resume. It is 2023 research, but the pattern is everywhere in modern agents.
For questions that need world knowledge, having the model generate relevant facts first, then answer using them, beats answering cold. How it works, and when to use it over RAG.
One chain of thought can take a wrong turn. Self-consistency samples several reasoning paths for the same question and votes on the answer, beating a single greedy pass on reasoning tasks.
Telling a model to reason before it answers genuinely improves accuracy on multi-step problems. How standard and zero-shot chain-of-thought work, when to use them, and the cost.
Examples are the most effective context you can add to a prompt, until they aren't. When few-shot beats zero-shot, how many to use, how to pick them, and where examples won't help.
A good prompt has up to four parts: instruction, context, input data, and output indicator. Here is what each does and the design habits that make them land.
Before you rewrite a prompt, check the dials around it. Temperature, top_p, max tokens, and the penalties each change your output in a specific way. Here is when to touch each.
An agent is a stateless model wrapped in a loop. Here is the ReAct pattern built from scratch: the message list as memory, the exact algorithm, and the rule that generate calls equal tool rounds plus one.
Why model deprecation quietly breaks production apps, and a concrete workflow: one models constant file, a scheduled GitHub Action checking deprecations.info plus a liveness probe, and an alert.
A practical guide to LLM observability: the traces and metrics to log, the handful of alerts worth waking up for, and why traditional monitoring misses what matters.
Prompt caching is solved. The bigger win for agents is caching tool-call results, and that is harder because tools have side effects. Here is how to do it safely.
Prompt caching can cut input cost by up to 90%, but only if you structure prompts so the cache actually hits. Here is how it works and when it pays off.
A practical playbook for making LLM apps cheaper and faster without hurting answers: where the cost and latency actually go, and the levers that work.
A practical guide to keeping production agents safe: where to put input and output rails, how to contain tools, and why prompt injection needs layers, not a fix.
A full walkthrough of streaming LLM output from the provider, through your backend, into a React UI: SSE, backpressure, cancellation, and the gotchas.
A clear split between an agent's working memory and its cross-session memory, with LangGraph code for checkpointers, stores, and the semantic/episodic/procedural breakdown.
Structured outputs guarantee the model's response matches your JSON schema. Here is how it works under the hood, how it relates to function calling, and the patterns that hold up in production.
Embeddings turn text into vectors so meaning becomes geometry. Here is the mental model, the gotchas, and how to choose a model in 2026.