leadbybuild.ingArchive

LeadByBuilding Newsletter · 8 min read

Patch the runtimes under your services, and measure what an agent needs

Node.js and PHP both shipped urgent security fixes, with Node also pulling in the eMudhra CA change, so runtimes lead today. GitHub shows how scoping a write lock to one ref lifted throughput, while Shopify and a CNCF test show where agent cost and time actually go.

AI & Agent Engineering

Model operationsvendor-reported

Shopify moves token-heavy subagent work to post-trained open models, reporting about 90% lower serving cost

Shopify says Sidekick keeps a frontier model for planning and hands its GraphQL subagent, which reads about 40 tokens for each one it writes, to a post-trained open model. It reports a judge score of 74.5 against 73 for the frontier model, 0.87 s median latency against 6.50 s, and about 90% lower serving cost.

Why it matters

The loop matters more than the model: a calibrated LLM judge scores every change, and nightly self-healing turned failed production chats into training data, repairing 69% of attempted failures. Figures are Shopify's own, from one task, with NVIDIA tooling.

Do this week

Find the one agent step in your system that reads far more tokens than it writes. Before considering a smaller model, build a judge for that step and check it agrees with human labels.

Shopify Engineering →
AI for operationsvendor-reported

CNCF post: giving a coding agent cluster context cut median diagnosis time from 87 to 44 seconds

Radar's maintainer ran Claude Code against 50 SREGym faults on EKS, once with a shell and kubectl, once with read-only MCP tools returning current failures and a change timeline. Correct diagnoses were 45 against 46, while correct within two minutes rose from 60% to 86% and mean tool calls fell from 16.3 to 9.8.

Why it matters

Accuracy barely moved and both runs missed the same three faults; the gain was speed. The author maintains Radar, so read it as a vendor test. Permissions also differed between arms, so it does not isolate which context helped.

Do this week

Before buying an AI SRE product, run your current coding agent on a few past incidents with and without read-only access to failures and recent changes, then compare time to a correct diagnosis.

CNCF Blog →

Broader Engineering

Security releaseverified

Node.js ships security releases for 26.x, 24.x and 22.x, including four high-severity fixes and the eMudhra CA update

The Node.js project released v26.12.1, v24.22.1 and v22.23.4. Four issues are rated high: a dns.resolve NUL truncation, two TLS hostname bypasses involving IP addresses, and a Brotli decoder out-of-bounds write that permits remote code execution. The release also updates the bundled CA certificates for the eMudhra intermediate CA compromise.

Why it matters

This is the Node-side fix for yesterday's eMudhra trust-store incident, plus TLS hostname-check flaws. The post says end-of-life versions are always affected, so anything off a supported line has no fix.

Do this week

Check which Node versions run in production and CI base images, move them to the patched releases, and rebuild images instead of patching running containers.

Node.js Blog →
Security incidentverified

php2shell: two PHP runtime flaws affect PHP 5.4 and later, and updating your plugins will not fix them

Cloudflare says CVE-2026-109618 and CVE-2026-109619 let crafted HTTP traffic cause denial of service, code execution or data exposure through unsafe memory handling after an error. It deployed WAF rules ahead of disclosure, and says the leak brought attack traffic against customers before the official release.

Why it matters

Both flaws sit in the PHP runtime, so application code or plugin updates do not help; the runtime must be updated. Cloudflare says its WAF rules are defense in depth and a patch is still required.

Do this week

Inventory every PHP runtime you run or host, including long-running worker modes. Apply the advisory update, then keep any WAF protection enabled and restrict upload sizes while you roll it out.

Cloudflare Blog →
Distributed systemsvendor-reported

GitHub scopes its Git write quorum to individual refs, reporting 3 to 4x higher throughput across a 25 ms link

GitHub says its Spokes quorum waits for every healthy replica and holds a repository-wide lock per write, so a replica about 25 ms away in Azure caps throughput. It is changing serialization to per-reference scope. In internal tests, writes to different references saw 3 to 4x higher throughput at lower latency.

Why it matters

The bottleneck was lock scope, not hardware: Git only needs ordering per reference, so unrelated pushes need not wait. GitHub also plans object storage as a virtual replica to scale reads separately. The 3 to 4x figure is internal testing.

Do this week

Look at your slowest serialized write path and ask what actually needs ordering. If a coarse lock spans a network round trip, test a narrower key such as per tenant, branch or resource.

GitHub Blog →

Worth experimenting with

Time a security-release rollout for one runtime end to end

Take Node.js or PHP. List every place it runs: base images, CI runners, serverless layers, worker modes. Pick one service and move it to the patched version, recording each step and who had to act.

Why. Two runtime fixes landed this week. A timed rollout on one service shows how long your next urgent patch will really take and where it stalls.

Measure. Record minutes from advisory read to patched version serving traffic, the number of manual steps, and how many places you found that nobody had listed.

Classic / essential read

Technical Debt Quadrant

Martin Fowler · 4 min

Fowler argues the useful question is not whether a flaw counts as debt but whether the metaphor helps you decide, and splits debt into reckless or prudent and deliberate or inadvertent.

One lesson. When you take on debt, make it deliberate and prudent: decide the payoff for shipping early exceeds the interest, and record it so it can be repaid.

Read the original →

Commute watch · optional

Agents for production engineering - Alex Palcuie - Anthropicdotconferences · 22:59The 3 Golden Rules of Distributed SystemsInfoQ · 50:50

The common thread today is knowing what actually sets the limit: the runtime under your code, the lock scope on a write, or the context an agent lacks. Find that one thing in your own system this week and measure it.

Get this every morning.

Free. Same issue for everyone. One click to unsubscribe.

Subscribe