Fail-closed agent supervision, a large PostgreSQL security release, and traffic replay as a deploy gate
Three AI picks: how Shopify contains attack agents running against production, GitHub's practical structure for agent skills, and Martin Fowler on LLM-generated domain glossaries. Three engineering picks: a PostgreSQL update fixing 52 vulnerabilities with follow-up steps, Zillow's consolidation of notification services onto one workflow layer, and production traffic replay as a deployment gate.
AI & Agent Engineering
Agent securityverified
Shopify runs attack agents against production, with a second agent that can stop each run
Shopify describes Cyber Chaos Monkey, agents that hunt vulnerabilities in production using test shops. Each engagement is a Kubernetes job. The agent gets ephemeral credentials, a proxy swaps placeholders for real keys, and egress is an allowlist. A second agent on a different model reads the session log live and can kill the run; if that judge times out or crashes, the run ends. Shopify reports tens of thousands of engagements with no outage.
Why it matters
This is a worked example of containing an agent you do not fully trust: least privilege, no raw keys, allowlisted egress, and an independent reviewer that fails closed. The same design fits any agent that acts on production, not only security testing. The outcome figures are Shopify's own.
Do this week
For one agent that acts on real systems, check what happens when its supervisor or guardrail check times out or crashes. If the agent keeps running, change the design so that a missing verdict ends the run.
GitHub's guide to agent skills: one job per skill, project facts in a manifest, checks in CI
A GitHub blog post by Andrea Griffiths describes an agent skill as a versioned directory centered on a SKILL.md file. A skill does not grant access; the host provides tools, credentials and permissions. The advice: give each skill one job, keep project names and conventions in a manifest, share one path to the data platform, and package the same source for Claude Code, Cursor and GitHub Copilot with a check mode in CI.
Why it matters
Teams are accumulating skills without a shared way to structure them. The post draws a useful line: skills fit bounded, repeatable work, while workflows that need shared state, an audit trail or permissions enforced outside the model should be an application. Its verifier example rests on the rule that self-verification is not verification.
Do this week
Take the task your team re-explains to agents most often and write it as one skill with a clear input, output and stopping point. Move project-specific names into a separate manifest, and keep merge-blocking rules in CI, not in the skill text.
Martin Fowler: have an LLM draft a domain glossary, let experts correct it, and version it
Martin Fowler describes a technique for LLM work in specialized domains: use an LLM to generate a glossary from the organization's documents, emails and chat logs, including known words used in unfamiliar ways. Domain experts review and amend it, and it is kept in version control and refreshed on a schedule. He advises against a single standard glossary and suggests separate glossaries per bounded context, including one derived from source code.
Why it matters
Specialized vocabulary is outside a model's training data, and Fowler says it is a common difficulty for people using agents in large organizations. A reviewed glossary is cheap context. For large glossaries he suggests loading only terms and links first, so the model reads full definitions only when needed.
Do this week
Pick one bounded context. Have an LLM draft a glossary from its docs and code, including terms that should not be used. Ask one domain expert to correct it, commit it, and add it to your agents' context for that area.
PostgreSQL 18.7 and other minors fix 52 security vulnerabilities; PostgreSQL 14 reaches end of life
The PostgreSQL project released 18.7, 17.12, 16.16, 15.20 and 14.25, plus PostgreSQL 19 RC 2. The update fixes 52 security vulnerabilities, many rated CVSS 8.8 for arbitrary code execution, and over 140 bugs, including GIN and pg_trgm GiST index corruption and incremental backups that could be corrupt. This is the final release of PostgreSQL 14. PostgreSQL 19 general availability is planned for November 19, 2026.
Why it matters
This is an unusually large security release, and applying the binaries is not the whole job. The notes list follow-up steps: reindex affected GIN, pg_trgm GiST and bpchar B-tree indexes, and take a new full backup if you use incremental backups. Anything still on PostgreSQL 14 now gets no further fixes.
Do this week
List every PostgreSQL instance with its minor version, including managed ones. Schedule this update, add the reindex and full-backup steps from the Updating section to the change plan, and set a date to move off PostgreSQL 14.
Zillow replaced more than 10 campaign services with one config-driven workflow layer
Zillow and AWS describe how Zillow sends more than 100 million notifications a day on Step Functions Express Workflows. More than 10 generator microservices became shared, stateless modules behind one orchestrator, and campaigns are declared as schema-validated configuration. It also evaluated Temporal and Flowable. Express gives at-least-once execution, so modules are idempotent. Batching 10 messages is reported as about 90 percent cheaper.
Why it matters
The post shows the reasoning behind a common consolidation: duplicated per-campaign services, a managed engine chosen partly on operating cost, and weaker execution guarantees accepted because steps are idempotent. It also notes that service quotas were part of the design and that a router outside the workflows handles pause, replay and send speed.
Do this week
If several of your services repeat the same trigger, enrich, apply rules, send pattern, write down which steps are truly different per service. Check whether each step is idempotent before choosing an at-least-once engine to consolidate them.
Replaying 5 percent of production traffic as a deployment gate for behavior changes
An AWS Architecture Blog post describes a framework that captures production requests and responses, replays a sample against new code in pre-production, and compares normalized results. The match fraction feeds a CloudWatch alarm, commonly at 99.9 percent, that gates the deployment. Replayed requests carry simulation IDs so stateful services skip writes. The author reports five regressions caught in over 6 months with zero false positives.
Why it matters
Integration tests with unreliable dependencies produce false failures, and unit tests miss real request shapes. Replay checks that refactors, dependency upgrades and code behind feature flags leave existing behavior unchanged. It does not test new logic, and it fits stateless computation best. The results are from one team.
Do this week
Choose one stateless, high-stakes service such as pricing or eligibility. Capture a small sample of real requests and responses, replay them against the next release, and count mismatches after removing timestamps and sorting lists.
Pick one agent that acts on real systems and has a reviewer, policy check or approval step. In a test environment, make that check time out or crash during a run. Record whether the agent stops, continues, or retries without the check.
Why. Shopify's design ends the run when the judge gives no verdict. Many guardrails are only tested on the path where they answer, so the failure path is unknown.
Measure. Count runs where the agent took any action after the check failed. The target is zero. Note how long it took from the check failing to the agent stopping.
Classic / essential read
Ubiquitous Language
Martin Fowler · 2 min
This 2006 note summarizes Eric Evans's practice of building a common, rigorous language between developers and users, based on the domain model. It is the idea behind today's generated glossary story: agents now need that shared language as much as people do.
One lesson. Test the language by using it in conversations with domain experts, and let it evolve as understanding grows. Ambiguity that people tolerate will trip up software design.
Assume the checker can fail. A guardrail should stop the agent when it gives no answer, a patch release should include its follow-up steps, and a refactor should be compared against real traffic before it ships.
Get this every morning.
Free. Same issue for everyone. One click to unsubscribe.