Tag

LLM

53 articles

AI System DesignWhen Building Gets Cheap, Judgment Gets ExpensiveEight moves for shaping an AI product before the first line of code.August 3, 202620 minAugust 3, 202620 min
Agentic AIWhat Agent Skills Are, and Why Progressive Disclosure Changes the MathA skill is a folder of know-how an agent loads only when a task needs it. The mechanism underneath is the whole point.August 2, 20267 minAugust 2, 20267 min
From Demo to Product · Part 12Designing UX for a Confidently-Wrong InternTraditional UX assumes the system is correct and fast. An AI feature is neither guaranteed. Design for that.August 2, 20267 minAugust 2, 20267 min
From Demo to Product · Part 11The Reference Architecture for a Production AI AppThe model call is one box out of ten. The other nine are the engineering.August 2, 202611 minAugust 2, 202611 min
From Demo to Product · Part 10Who Owns Quality: The Benevolent Dictator and AI Incident ResponseOn an AI product, quality is a judgment call that needs one owner, and an incident whose postmortem output is a new eval exampleAugust 2, 20269 minAugust 2, 20269 min
From Demo to Product · Part 9Detect, Contain, Monitor: The Risks That Sink AI ProductsYou do not eliminate AI-product risk. You bound the blast radius and watch the rate.August 2, 20267 minAugust 2, 20267 min
From Demo to Product · Part 8The Unit Economics of an AI ProductEvery request costs real money. Why the unit is cost per task, not per call, and how to keep it profitable.August 2, 20268 minAugust 2, 20268 min
From Demo to Product · Part 7LLMOps and the Data FlywheelLaunch is the start of the work. How you operate an AI product, and how production usage compounds into a moat.August 2, 20268 minAugust 2, 20268 min
From Demo to Product · Part 6Ship on Containment, Not PerfectionHow to expose a non-deterministic feature to real users safely, without waiting for a flawlessness that never comesAugust 2, 202612 minAugust 2, 202612 min
From Demo to Product · Part 5Treat the Model as a Flaky, Expensive DependencyThe vendor changes pricing, retires models, and swaps tokenizers under your live product. Wrap it accordingly.July 25, 20265 minJuly 25, 20265 min
From Demo to Product · Part 4Does Your Problem Even Need an LLM?The fit gate to run before you build, and what to do when the answer is noJuly 25, 20265 minJuly 25, 20265 min
From Demo to Product · Part 3Evals Are the Product Discipline (and Writing Them First Backfires)Why eval-driven development done like TDD fails, and what error-analysis-first evaluation looks like insteadJuly 25, 20266 minJuly 25, 20266 min
From Demo to Product · Part 2AI Is Not a MoatThe model is electricity, not the factory. What actually makes an AI product defensible.July 25, 20266 minJuly 25, 20266 min
From Demo to Product · Part 1The Afternoon and the QuarterWhy an AI demo takes an afternoon and an AI product takes a quarter, and what lives in the gapJuly 25, 20265 minJuly 25, 20265 min
Agentic AIHow to Evaluate Agents Locally with Arize PhoenixTrace and judge your agent on your own machine, no hosted service requiredJuly 12, 20267 minJuly 12, 20267 min
AI InfrastructureWhat Is OpenTelemetry, and Why It Matters for Agent ObservabilityThe vendor-neutral standard your agent traces should be built onJuly 12, 20266 minJuly 12, 20266 min
AI InfrastructureTracing in Agent Observability: What It Is and Why It MattersTraces, spans, and why a readable trace is the fastest way to debug a misbehaving agentJuly 12, 20267 minJuly 12, 20267 min
LLM IntegrationWhat the Research Actually Says About Prompting ReasoningThe taxonomy of reasoning elicitation, and the uncomfortable retrieval-vs-reasoning debate every agent builder should sit withJuly 3, 20264 minJuly 3, 20264 min
LLM IntegrationA Field Guide to Model-Specific PromptingHow prompting actually differs across Claude, GPT, Gemini, Llama, and Mistral, and what transfersJuly 3, 20265 minJuly 3, 20265 min
LLM IntegrationA Real Prompt Engineering Case Study: 65.6 to 91.7 F1 on Job ClassificationWhat a production classification system teaches about which prompt tweaks actually move the needleJuly 3, 20264 minJuly 3, 20264 min
LLM IntegrationPrompt Functions: Treating Prompts as Reusable FunctionsA 2023 chat trick that quietly became how we build LLM systemsJuly 3, 20266 minJuly 3, 20266 min
LLM IntegrationGenerating Code with Prompts: The Fundamentals Still MatterComments to code, SQL from schemas, and the verification habit that separates working code from plausible codeJuly 3, 20266 minJuly 3, 20266 min
LLM IntegrationDataset Diversity: Fixing Repetitive Synthetic GenerationsWhy your generated dataset all sounds the same, and the seeding trick that fixes itJuly 3, 20267 minJuly 3, 20267 min
Agentic AIBuilding Agents That Actually WorkThe practical habits that separate a reliable agent from a flaky demo: simpler workflows, better information flow, and a debugging ladderJuly 2, 202610 minJuly 2, 202610 min
LLM IntegrationGenerating Synthetic Data with PromptsTurn a model into a data factory for tests, evals, and cold-start training, without spending a month labelingJuly 2, 20266 minJuly 2, 20266 min
LLM IntegrationBias in Prompting: How Your Prompt Design Skews the ModelThe distribution and order of your few-shot examples can quietly bias the model. Here is how to test for it.July 2, 20266 minJuly 2, 20266 min
LLM IntegrationFactuality: Prompting to Reduce HallucinationThree prompt-level moves that cut made-up answers, and where prompting stops being enoughJuly 2, 20266 minJuly 2, 20266 min
LLM IntegrationAdversarial Prompting: Injection, Leaking, and JailbreakingThe three attacks that break LLM apps, and the defences that actually hold upJuly 2, 20268 minJuly 2, 20268 min
LLM IntegrationGraph Prompting, ExplainedWhat it means to prompt a model with nodes and edges, and when it actually helpsJuly 2, 20268 minJuly 2, 20268 min
Agentic AICodeAgent vs ToolCallingAgent: Two Ways to Let an Agent ActThe difference between an agent that writes Python and one that emits JSON, with strengths, limits, use cases, and when to pick eachJuly 2, 20269 minJuly 2, 20269 min
Prompt Engineering, Properly · Part 14Prompting Reasoning Models Is Almost the Opposite of Prompting Chat ModelsWhy the step-by-step prompts you learned in this series can hurt an o3 or Claude with thinking onJuly 1, 20267 minJuly 1, 20267 min
LLM IntegrationMultimodal Chain-of-Thought: Reason Over the Picture, Then AnswerA two-stage framework that generates a rationale from text and image first, then infers the answer from that rationaleJuly 1, 20265 minJuly 1, 20265 min
LLM IntegrationPAL: Let the Model Reason in Words, but Let Python Do the MathProgram-Aided Language Models offload the actual computation to an interpreter, so arithmetic bugs stop wrecking your reasoningJuly 1, 20266 minJuly 1, 20266 min
LLM IntegrationDirectional Stimulus Prompting: Train a Tiny Model to Whisper Hints to a Big OneA small tuneable policy model generates hints that steer a frozen black-box LLM, no access to its weights requiredJuly 1, 20265 minJuly 1, 20265 min
LLM IntegrationActive-Prompt: Stop Guessing Which Few-Shot Examples to AnnotateUse the model's own uncertainty to pick which questions are worth a human-written reasoning chainJuly 1, 20266 minJuly 1, 20266 min
LLM IntegrationART: Let the Model Write Its Own Tool-Using ReasoningAutomatic Reasoning and Tool-use, and why it reads like an early sketch of the agents we build todayJuly 1, 20265 minJuly 1, 20265 min
Prompt Engineering, Properly · Part 7Generated-Knowledge Prompting: Surface Facts Before You AnswerHave the model write down what it knows first, then answer using that as contextJune 29, 20266 minJune 29, 20266 min
Prompt Engineering, Properly · Part 6Self-Consistency: Sampling Your Way to Better AnswersRun the same reasoning prompt several times, then take the majority answerJune 29, 20265 minJune 29, 20265 min
Prompt Engineering, Properly · Part 5Chain-of-Thought PromptingStandard and zero-shot CoT, when step-by-step reasoning actually helps, and what it costsJune 29, 20267 minJune 29, 20267 min
Prompt Engineering, Properly · Part 4Zero-Shot vs Few-Shot PromptingWhen examples help, how many to use, how to pick them, and where few-shot quietly breaks downJune 29, 20267 minJune 29, 20267 min
Prompt Engineering, Properly · Part 3The Anatomy of a Good PromptInstruction, context, input, and output indicator, plus the design habits that make them workJune 29, 20267 minJune 29, 20267 min
Prompt Engineering, Properly · Part 2The Settings That Change Your OutputTemperature, top_p, max tokens, stop sequences, and the penalties, and when to touch eachJune 29, 20266 minJune 29, 20266 min
Agentic AIThe Agent Loop: Building ReAct From ScratchReason, act, observe, repeat. The whole engine is about 20 lines, and one counting rule explains all of it.June 28, 202614 minJune 28, 202614 min
AI InfrastructureMonitoring Model Deprecation in ProductionThe model you pinned will be switched off one day, probably with less notice than you would like. Here is how to find out from CI instead of from your users.June 24, 202616 minJune 24, 202616 min
AI InfrastructureObservability for LLM Apps: What to Log, What to Alert OnYour 500s and latency graphs will look fine while the product quietly gives wrong answers. LLM observability is about catching the failures that do not throw.June 24, 20267 minJune 24, 20267 min
AI InfrastructureCaching Agent Tool Calls (Not Just Prompts)Prompt caching saves you tokens. Caching tool calls saves you the slow, flaky, expensive thing the tool actually does.June 24, 20269 minJune 24, 20269 min
LLM IntegrationPrompt Caching for LLM Apps: What It Is and When It Pays OffHow providers cache your prompt prefix, the real discounts, and the prompt structure that decides whether you save anythingJune 22, 20266 minJune 22, 20266 min
LLM IntegrationCutting LLM Cost and Latency Without Wrecking QualityMeasure first, then reach for caching, routing, smaller models, and the structural fixes that actually move the numbersJune 22, 20267 minJune 22, 20267 min
Agentic AIGuardrails and Safety for Agents in ProductionDefense in depth for agents: input rails, output checks, tool limits, and the injection problem that will not fully go awayJune 22, 20267 minJune 22, 20267 min
LLM IntegrationStreaming LLM Responses End to End: Backend to UIWhy SSE won, how the token stream flows from the provider to the browser, and where it breaksJune 22, 20269 minJune 22, 20269 min
Agentic AI in Practice · Part 7Agent Memory: Short-Term vs Long-Term, and How to Wire It UpThreads are short-term memory. Stores are long-term memory. Most agent memory bugs come from confusing the two.June 20, 20267 minJune 20, 20267 min
LLM IntegrationStructured Outputs and Function Calling, In DepthHow to make an LLM return data that always fits your schema, and how function calling is the same idea wearing a different hatJune 20, 20267 minJune 20, 20267 min
Production RAG · Part 3Embeddings Explained for EngineersWhat embeddings really are, why cosine similarity works, and how to pick a model without trusting a leaderboard blindlyJune 20, 20265 minJune 20, 20265 min