Notes from production

1. In progressfrom Payments operations agent

   The LLM Never Writes the Email

   A pattern for automating money-adjacent messages: let the model classify and extract, never compose. The reply body is copied verbatim from a template registry, which makes a hallucinated sentence structurally impossible rather than merely unlikely.

   GuardrailsArchitectureFintech
2. In progressfrom Customer status pages

   Five Stages of Fail-Closed

   Publishing LLM-written text to a public page nobody has to log into. The pipeline that sits between the model and the customer, why every stage falls back to a safe line, and why the evaluation harness has to be shown failing on purpose.

   GuardrailsEvaluationProduction
3. In progressfrom Support inbox agent

   A Second Model Decides Whether the First Gets to Send

   Self-reported confidence from a language model is worth nothing. How I replaced it with a separate judge scoring faithfulness, coverage and fit, why the scores are combined in code rather than in a prompt, and why the hard cap matters more than the weights.

   LLM-as-judgeCalibrationEvaluation
4. In progressfrom Onboarding platform

   Shipping Production Software with a Fleet of Coding Agents

   Thirteen parallel agent worktrees, a written playbook, and twenty-six tickets each scoped to be executed by an agent. What actually makes this work is not the agents — it is the test suite and the review gate that make parallelism safe.

   Agent-nativeProcessTesting
5. In progressfrom Internal knowledge assistant

   The Bakeoff That Made Me Rebuild Our Bot

   Seven questions run through a custom RAG pipeline and an MCP-based one, scored side by side. It came out nearly a tie, and the pattern underneath the tie is what decided the rewrite.

   RAGMCPArchitecture
6. In progressfrom Payments operations agent

   Your Test Email Isn't Production

   A byte-identical email body classified differently depending only on who sent it. Sender turned out to be a load-bearing signal, which means self-sent test messages cannot reproduce the behavior you are trying to verify.

   TestingDebuggingProduction

These are titles and arguments, not published posts — they are listed here because the work behind them is real and the pieces are being written. When one goes up, it will appear above with a link rather than a promise.