Agentic AI
What Is AgentOps? Operating AI Agents in Production
AgentOps is the practice of running agents in production: observability, evaluation, guardrails, cost governance, and human oversight.
6 min read
31 articles
AgentOps is the practice of running agents in production: observability, evaluation, guardrails, cost governance, and human oversight.
OpenTelemetry in plain terms: the data model, the GenAI semantic conventions, and why building agent tracing on OTel avoids lock-in.
What traces and spans actually are, how they map onto an agent run, and how to walk one to find the exact step that broke.
Most prompting advice transfers across models. The parts that do not will quietly break your app when you switch providers. A working map of the differences.
A production case study measured fourteen prompt modifications one by one. The results are humbling: naming the model helped, few-shot examples hurt, and templates cost accuracy.
Code generation is the most-used LLM capability and the most casually prompted. A tour of the core patterns, plus the checks that catch what the model quietly gets wrong.
Prompt injection, prompt leaking, and jailbreaking are three different attacks with three different fixes. Here is how each one works and what defence actually holds.
Reasoning models do the chain-of-thought for you. That flips a lot of prompting advice on its head, and getting it wrong wastes tokens and quality.
Directional Stimulus Prompting trains a small model to produce hints that nudge a big frozen LLM toward what you want. It is prompt optimization when you cannot touch the weights.
A good prompt has up to four parts: instruction, context, input data, and output indicator. Here is what each does and the design habits that make them land.
Before you rewrite a prompt, check the dials around it. Temperature, top_p, max tokens, and the penalties each change your output in a specific way. Here is when to touch each.
Anthropic's Claude Tag drops a shared, persistent agent into Slack channels. The interesting part is not the model, it is the move from one-off chats to a teammate that remembers.
Why model deprecation quietly breaks production apps, and a concrete workflow: one models constant file, a scheduled GitHub Action checking deprecations.info plus a liveness probe, and an alert.
A practical guide to LLM observability: the traces and metrics to log, the handful of alerts worth waking up for, and why traditional monitoring misses what matters.
How to choose how to structure an agent: the plain while loop, the explicit state machine, and the full graph. What each gives you, what each costs, and when to move up.
Prompt caching is solved. The bigger win for agents is caching tool-call results, and that is harder because tools have side effects. Here is how to do it safely.
How to build a RAG system that serves many customers without leaking data between them: silo vs pool vs bridge, namespaces vs metadata filters, and where isolation must hold.
If you can already ship software, the way into AI engineering is not another tutorial. It is building five systems that force you to handle failure, measure quality, and run things in production.
Autonomy is great until the agent deletes the wrong file or emails the wrong list. Here is how DeepAgents pauses on sensitive tool calls and waits for a human, in TypeScript.
DeepAgents treats long-term memory as files the agent reads and writes, with backends deciding where they live. Here is how scoping, episodic memory, and consolidation actually work.
Prompt caching can cut input cost by up to 90%, but only if you structure prompts so the cache actually hits. Here is how it works and when it pays off.
What state, checkpoints, and threads actually are in LangGraph, how to pick a checkpointer, and how persistence powers memory, resume, and human-in-the-loop.
A practical playbook for making LLM apps cheaper and faster without hurting answers: where the cost and latency actually go, and the levers that work.
A practical guide to keeping production agents safe: where to put input and output rails, how to contain tools, and why prompt injection needs layers, not a fix.
A full walkthrough of streaming LLM output from the provider, through your backend, into a React UI: SSE, backpressure, cancellation, and the gotchas.
The field moves faster than any course can. So I learn production AI by gutting and rebuilding the open-source projects that production teams actually run. Here is the path.
A clear split between an agent's working memory and its cross-session memory, with LangGraph code for checkpointers, stores, and the semantic/episodic/procedural breakdown.
Structured outputs guarantee the model's response matches your JSON schema. Here is how it works under the hood, how it relates to function calling, and the patterns that hold up in production.
A practical, opinionated comparison of the four vector databases people actually argue about, with a clear default and the signals that mean you have outgrown it.
Rerankers re-score your top candidates with a model that actually reads the query and the document together. Here is how they work and when they pay off.
Chunking quietly decides how good your RAG system can ever be. Here is how the main strategies compare and a simple way to pick one.