leadbybuild.ingArchive

LeadByBuilding Newsletter · 8 min read

Delete agent skills that add no lift, and learn from a three-bug GKE outage

Two AI picks: Google Cloud's five rules for testing agent skills before they cost context, and Cloudflare's new view and policies for personal agent traffic. Three engineering picks: Google's three-layer GKE timeout investigation, AWS Lambda queue mode for Kafka, and date-based versions for the Compute Engine API.

AI & Agent Engineering

Agent evaluationvendor-reported

Google Cloud: five rules for testing agent skills before they bloat context

Google Cloud's developer relations team describes how it evaluates SKILL.md files for its Developer Plugin. Its rules: check trigger collisions first, run evals in clean sandboxes, compare runs with and without the skill over repeated runs, keep correctness, efficiency and security scores separate, and write operational habits instead of copied docs.

Why it matters

Google reports that a skill with zero gain over the baseline is pure context cost, and that over a third of tasks in one retrieval study scored differently across identical runs. Single-run checks hide both problems. These are vendor-reported results.

Do this week

Pick one skill your team relies on. Run the same task three times with and without it in a clean sandbox. If success and time do not improve, delete the skill. Repeat the check when you change models.

Google Cloud Blog →
Agent trafficverified

Cloudflare adds analytics and per-agent access policies for personal AI agents

Cloudflare added a Personal Assistants category to Security Analytics and a widget listing which agents visit a site and how many requests each sends. The Agents policy, now called AI Crawler and Agent access policies, lets owners set Allow, Block on pages with ads, or Block for each supported personal agent.

Why it matters

Agent traffic looks like generic automation unless it is identified. Per-agent policy lets a site welcome agents that bring customers without opening everything. Account Abuse Protection, which links agents to users, is in early access.

Do this week

If you run a public site behind Cloudflare, review the Personal Assistants widget this week. Check which agents reach login and checkout paths, and decide your default policy before agent traffic grows.

Cloudflare Blog →

Engineering

Incident analysisverified

Google traced GKE API server timeouts to three bugs in three layers, ending in a kernel patch

Pods in GKE clusters using Private Service Connect intermittently timed out reaching the API server. Google found a proxy change that dropped TCP challenge ACKs, packet reordering, Envoy half-closing sockets so the FIN_WAIT_2 timeout never started, and a Linux 6.1 bug that discarded a valid FIN. It rolled back the proxy change and the kernel fix ships in GKE 1.33.5 and later.

Why it matters

Each defect was tolerable alone. Together they produced one symptom that fit every layer, and about 800 developers were blocked for weeks. Simultaneous packet captures at several points in the path, matched against an internal metric spike, exposed it.

Do this week

Write a runbook step for timeouts that rule out your own code: capture packets at two points on the path at once and compare. Also ask your platform vendor which recent network change dates match your first error.

Google Cloud Blog →
Serverless messagingverified

AWS Lambda adds a queue mode for Kafka that scales consumers beyond partition count

AWS added a ConsumptionMode setting to Lambda event source mappings for self-managed Kafka. Queue mode uses Kafka Share Groups (KIP-932), so many pollers read the same partition without ordering guarantees. A poller handles up to 10 concurrent invocations versus 5 in stream mode. It needs Kafka 4.2 or later with queue support.

Why it matters

Plain consumer groups cap concurrency at the partition count and let one slow record block those behind it. Teams used to add a separate queue system for unordered work. Stream mode stays the default, so existing mappings do not change.

Do this week

List Kafka topics where ordering does not matter, such as notifications or rendition jobs. If your cluster runs or can move to Kafka 4.2, test one on queue mode in staging and compare backlog drain time.

AWS Compute Blog →
API designverified

Compute Engine adds date-based API versions so defaults can change without breaking /v1

Google made Compute Engine API multi-versioning generally available. Stable versions named YYYY-MM-DD are immutable, start with 2026-09-01 and ship quarterly. Callers pick one with the X-Goog-Api-Version header or the apiVersion query parameter. /v1 stays the default. New versions enable partial success on aggregated lists and structured error payloads.

Why it matters

A single mutable /v1 made even a better default a breaking change. Pinning an immutable snapshot lets platform teams test a new contract by changing one version string. Client libraries, gcloud and Terraform providers bind to versions on their own.

Do this week

Pin the API version in any direct Compute Engine REST calls, and read the release notes of each quarterly version in staging before adopting it. Consider the same snapshot model for your own internal APIs.

Google Cloud Blog →

Worth experimenting with

Run a with-and-without ablation on one agent skill

Choose one skill file. In a clean container with no cached credentials, run one real task three times without the skill and three times with it. Record pass or fail, wall-clock time and tokens for each run.

Why. Shows whether the skill helps or only adds context, and gives you a baseline to rerun when the model changes.

Measure. Pass rate out of three, median time and token count for each arm. Delete the skill if it shows no gain in pass rate or time.

Classic / essential read

Technical Debt

Martin Fowler · 4 min

It frames cruft as debt with interest. The extra effort to add a feature is the interest, and the cleanup is the principal. That gives you a way to argue for cleanup with numbers.

One lesson. Clean up before a feature only if several similar features follow. Fowler's example: five days of cleanup loses on one feature but wins on three.

Read the original →

Commute watch · optional

Agentic Evaluations at Scale, For Everybody — Nicholas Kang & Michael Aaron, Google DeepMindAI Engineer · 20:03The People Behind the Hamsters and Other Stories of DowntimeAntithesis · 53:40

Both the skills article and the GKE case make one point: single checks mislead. Test an agent skill against a baseline over repeated runs, and debug a timeout by capturing evidence at several layers at once before blaming any one of them.

Get this every morning.

Free. Same issue for everyone. One click to unsubscribe.

Subscribe