Tests agents cannot quietly rewrite, a leaderless consensus choice, and a local AWS emulator
Two AI picks: keeping the test contract in data files so coding agents cannot quietly weaken it, and how OpenShell sandboxes agents on Kubernetes with per-request policy. Three engineering picks: why Cloudflare chose QuePaxa over Raft for a global control plane, AWS Emulator for local and CI testing, and why doubling coding speed moved one team's lead time by only 4%.
AI & Agent Engineering
Agentic developmentverified
Tests as data: keep the contract with coding agents in files they are not expected to change
Matteo Vaccari of Thoughtworks writes on martinfowler.com that when agents write both code and tests, developers stop trusting the tests. He interviewed 26 developers who said the AI refactors existing tests along with the code. He proposes black-box customer tests whose inputs and expected outputs live in tables or small DSL text files, run by a custom test runner, so the files can be protected or checked in source control.
Why it matters
A test suite that an agent can edit in the same change as the code is a weak check. Moving expectations into readable data makes a weakened test show up as a small diff a person can review. The author notes limits: it suits production systems, not prototypes, and exploratory testing is still needed.
Do this week
Pick one business rule that agents often touch. Move its cases into a table file with input and expected output, have the test runner read that file, and make changes to the file need a named human reviewer.
OpenShell on Kubernetes: agent sandboxes with deny-by-default egress and placeholder credentials
A CNCF blog post by Jim Bugwadia of Nirmata explains OpenShell, a CNCF Sandbox project started by NVIDIA. Each agent runs in a sandbox with Landlock file limits, a non-root user with seccomp, and outbound traffic denied by default. A supervisor outside the sandbox allows traffic per binary, host, port and HTTP method, and swaps a placeholder for the real API key. The demo uses OpenShell 0.1.3.
Why it matters
The post treats every agent as untrusted code holding real credentials. A NetworkPolicy can allow a host and port, but not say that curl may read issues and not create them. The project is v0.1.x, and the post notes a supervisor Pod per agent may not scale well in large environments.
Do this week
For one agent you run with real credentials, write down which hosts and HTTP methods it needs and whether it can read the raw API key. Even without OpenShell, deny egress by default and move the key behind a proxy that injects it.
Cloudflare explains choosing QuePaxa over Raft for a control plane that spans continents
Cloudflare describes why Meerkat, its experimental consensus service for an internal global control plane, uses QuePaxa, a 2023 algorithm that does not rely on a leader to make progress. Instead of an election timeout tied to network latency, other proposers join after a short hedge timeout. Cloudflare reports that over 95% of decisions use the leader fast path, and that a leader failure cut the decision rate to about 65% without a halt.
Why it matters
Raft and Multi-Paxos stall while a failed or slow leader is detected and replaced, and that detection cannot be faster than the network. The post is a clear account of that trade-off. Cloudflare also says QuePaxa is less battle-tested, so it is formally verifying its implementation. The numbers are its own.
Do this week
For each consensus-backed system you run, such as etcd, Consul or ZooKeeper, find the election timeout and the round-trip time between members. Test what clients see when the leader is slow but not dead, not only when it crashes.
AWS Emulator runs AWS SDK and CLI calls locally and in CI, with no account needed
AWS released AWS Emulator for building and testing AWS applications on a computer or in CI. Existing SDK and CLI calls go to the emulator with credentials it provides. Unsupported operations return 501 Not Implemented. State is kept in memory until it stops, and each copy has its own port, keys and state. Services are generated from the published AWS API models. There is no service fee and the source is released under CC0 1.0.
Why it matters
An official emulator shortens the feedback loop for people and coding agents and removes shared test accounts from many CI jobs. AWS says supported operations return the same results and errors as AWS; that is a vendor claim, and AWS publishes a fidelity report to check it against.
Do this week
Pick one integration test suite that needs a real AWS account today. Run it against the emulator in CI, list the calls that return 501 or behave differently, and keep a smaller set of tests against real AWS for those.
Coding got about twice as fast, but one team's lead time moved only 4%
Bharat Sharma writes on LeadDev that he compared an AI-assisted initiative with similar work done without AI. Coding was about 10% of total lead time, so close to 2x faster coding cut lead time by 4%. He stresses this is one comparison in one organization. He advises reporting lead time and where work waited next to adoption rate and merged PRs, and tracking wait time, active work time and rework time with a reason for each wait.
Why it matters
Adoption rate, PRs merged and suggestion acceptance measure creation. If review, decisions, dependencies or approval gates are the constraint, those numbers can rise while delivery stays flat, and the budget goes to the wrong step.
Do this week
Take two similar pieces of work, one from before AI adoption and one after. Split each into wait, active and rework time, and write a specific reason for every wait. Report the largest queue next to your adoption numbers.
Make one test expectation file hard for an agent to change unnoticed
Choose one feature with clear business rules. Put its cases in a table file of inputs and expected outputs that the tests read. Add a CI check or code owner rule so any change to that file needs a named human reviewer. Then give an agent a refactoring task in that area.
Why. It shows whether moving expectations into data makes a weakened test visible in review, which is the claim of the tests as data article.
Measure. Count agent changes that touched the expectation file, and how many of those a reviewer noticed without being told. Compare review time with a similar change to tests written as code.
Classic / essential read
Specification By Example
Martin Fowler · 4 min
This 2004 note explains why examples that can be clearly passed or failed are often easier to write than general specifications, and why they still need to be backed by other mechanisms. It is the tradition today's tests as data article builds on.
One lesson. Say what you want twice, in different forms: once as examples and once as code. The double check finds errors, but examples are never the whole specification.
Speed in one step is not speed overall. Protect the checks that agents can edit, limit what an agent can reach, test consensus with a slow leader, and measure where work waits before buying more coding speed.
Get this every morning.
Free. Same issue for everyone. One click to unsubscribe.