Agentic AI
How to Evaluate Agents Locally with Arize Phoenix
A hands-on guide to running Phoenix locally: trace an agent over OpenTelemetry, then score it with LLM-as-judge and code evaluators.
7 min read
17 articles
A hands-on guide to running Phoenix locally: trace an agent over OpenTelemetry, then score it with LLM-as-judge and code evaluators.
Code generation is the most-used LLM capability and the most casually prompted. A tour of the core patterns, plus the checks that catch what the model quietly gets wrong.
Cranking temperature does not make synthetic data diverse. Injecting randomness into the prompt does. Here is the technique, with code.
Most agents fail for boring, fixable reasons. Here are the practices that make an agent reliable: fewer LLM calls, tools that explain themselves, and a debugging ladder to climb when it misbehaves.
When retrieval fails in your domain or language, you can prompt a large model to generate training data for a custom retriever, matching thousands of hand-labels for tens of dollars.
You can prompt a model to produce labeled examples on demand: test fixtures, eval sets, cold-start training data. Here is how to do it well and where the sharp edges are.
smolagents ships two agent classes that act in completely different ways: one writes Python, one emits structured JSON. Here is how each works, what each is good and bad at, and when to reach for which.
Chain-of-Thought lets a model reason and compute in the same breath, which is where it slips. PAL keeps the reasoning in language but hands the math to a Python interpreter.
Hand-picking chain-of-thought examples is a guess. Active-Prompt uses model disagreement to find the questions where a human annotation actually moves the needle.
An agent is a stateless model wrapped in a loop. Here is the ReAct pattern built from scratch: the message list as memory, the exact algorithm, and the rule that generate calls equal tool rounds plus one.
Why model deprecation quietly breaks production apps, and a concrete workflow: one models constant file, a scheduled GitHub Action checking deprecations.info plus a liveness probe, and an alert.
How to stream LLM tokens, tool progress, and state updates from a LangGraph agent using the v2 stream API, with the custom writer pattern most people miss.
Structured outputs guarantee the model's response matches your JSON schema. Here is how it works under the hood, how it relates to function calling, and the patterns that hold up in production.
Deep Agents bakes planning, context management, subagents, and human approval into the agent loop. Here is why it exists, every main feature, and a production-grade customer support agent built with it.
You cannot improve an agent you cannot measure. A deep, practical guide to evaluating agents with LangSmith: tracing, datasets, evaluators, experiments, and what makes agent evaluation different.
A hands-on guide to building MCP servers with FastMCP, drawn from two I actually shipped: the anatomy, the rules, stdio vs remote HTTP, the pitfalls, and how to test and deploy.
An agent with no tools is just a chatbot. Here is how tool calling really works, what makes a tool the model can use reliably, and how MCP lets you plug in tools you did not write.