leadbybuild.ingArchive

LeadByBuilding Newsletter · 8 min read

Review queues grow under AI coding, faster inference starts, and one shared ECS cluster for many brands

Three AI picks: GitHub on the code review bottleneck from AI-generated code, CNCF on what each LLM startup speed-up preserves, and OpenKruise on sandbox egress control. Two engineering picks: Cognizant and AWS on consolidating tenants onto one ECS cluster, and a maturity model for platform investment.

AI and agent engineering

Inference infrastructureverified

CNCF maps what each LLM startup speed-up reuses: files, caches, weights, or a live engine

A CNCF post by Baofa Fan of DaoCloud groups inference startup optimizations by the work they preserve: model files, compiled artifacts, processed weights, a live engine, or an initialized snapshot. vLLM and SGLang have merged weight-cache daemons. vLLM snapshots are experimental and limited to one NVIDIA GPU with tensor parallelism of one.

Why it matters

Scale-out only adds capacity after a replica can serve. Each approach has a narrow scope, so knowing which stage dominates your startup decides which one helps. The post also notes a vLLM snapshot guide validated on v0.27.1 that does not work on v0.28.

Do this week

Time one cold replica start and split it into weight load, engine init, compile and graph capture. Only then pick between pre-staged weights, persistent caches or a weight daemon, and pin framework versions for anything snapshot-based.

CNCF Blog →
AI-assisted developmentvendor-reported

GitHub: review time grew 441 percent as AI adoption rose, so shrink and pre-check pull requests

A GitHub post says October saw more than 1.2 billion commits in one week, over 400 percent more than a year earlier. It cites a Faros AI study finding pull requests spent 441 percent longer in review when teams went from low to high AI adoption. GitHub suggests stacked pull requests, automated quality and security checks, and agents that fix failing checks.

Why it matters

Faster code generation moves the bottleneck to review and verification. The post is a vendor guide for Copilot features, so treat its tool advice as a product pitch. The review-queue point stands on its own.

Do this week

Measure median time from PR open to first review for your team before and after AI adoption. If it grew, set a diff size limit, stack dependent changes, and run routine checks before a human opens the diff.

GitHub Blog →
Agent infrastructureverified

OpenKruise Agents v0.6.0 adds egress policy, credential injection and warm pool autoscaling for sandboxes

A maintainer post says OpenKruise Agents v0.6.0 adds L4 and L7 egress control, MCP tool-level ACLs, injection of credentials from Kubernetes Secrets, a PoolAutoscaler and probe-driven pause and resume. Its CRD count doubled from 6 to 12, with 283 merged pull requests and 27 new contributors since v0.3.0.

Why it matters

Keeping upstream API keys out of agent code and denying egress by default are controls many teams are still building by hand. L4 denials are dropped in the pod before the gateway, and L7 policy cannot override them.

Do this week

If your agents call external APIs, check whether any key is readable by agent code. Try a default-deny egress policy on a test sandbox namespace and see what breaks. Read the TrafficPolicy semantics before adopting the project.

CNCF Blog →

Engineering

Architecturevendor-reported

OrderServ cut compute cost 47 to 64 percent by sharing one ECS cluster across brands

AWS and Cognizant describe moving OrderServ from a dedicated ECS cluster per brand, 21 services each, to one shared cluster with a tenancy library. EC2 warm pools with Bottlerocket cut scaling from 5 to 7 minutes on Fargate to 30 to 60 seconds. Projected monthly cost for 21 services fell from about 1,568 dollars to 832 with a one-year plan or 568 with a three-year plan.

Why it matters

Isolation moved from infrastructure into code, with a backstop: the library fails closed without a tenant prefix and assumes a role with the tenant ID as a session tag. Cache access is also restricted per tenant. These are AWS-published figures from a partner case study.

Do this week

If you run per-tenant clusters, count idle services and sidecars. Before consolidating, test that a missing tenant context fails closed, and that task roles cannot reach another tenant's data. Time your image pull before blaming scaling policy.

AWS Architecture Blog →
Platform engineeringverified

Fowler site: a maturity model for deciding where to invest in a data product platform

Tom De Wolf of ACA Group applies the CNCF Platform Engineering Maturity Model to data mesh platforms. He describes four levels from Provisional to Optimizing, and a loop of mapping your current state per aspect, focusing on a high-friction aspect and iterating. His example is a level 2 dedicated platform team with level 1 operations and measurement.

Why it matters

Platform investment is usually argued in the abstract. A per-aspect grid separates people and process aspects such as investment and adoption from technology aspects such as interfaces and operations, and warns against over-investing in one side.

Do this week

Run the grid for your own internal platform in one hour with two platform users. Mark each aspect 1 to 4, pick the single highest-friction aspect, and fund that before adding technology.

martinfowler.com →

Worth experimenting with

Test that your tenant or agent code fails closed

Pick one service that resolves a tenant or agent identity per request. Write a test that calls it with the identity missing and one that tries to read another identity's key. Both should be refused, not fall back to a default.

Why. OrderServ's design refuses to build resource IDs without a tenant prefix, so a missing context fails closed. A fail-open default is easy to miss in review.

Measure. Count code paths that return data when identity is absent, and note whether a second layer such as IAM or cache ACLs also blocks the cross-identity read.

Classic / essential read

Anemic Domain Model

Martin Fowler · 5 min

It describes domain objects that are only bags of getters and setters, with all behavior in service classes. It pairs with this week's stories on putting rules in a library or host layer rather than scattering them.

One lesson. Keep business rules next to the data they govern, or accept that you are writing procedural code with object syntax.

Read the original →

This week's common thread is where a guardrail lives. OpenKruise and OrderServ put enforcement outside the code that could get it wrong, and GitHub argues review needs pre-checks. Check where yours sits.

Get this every morning.

Free. Same issue for everyone. One click to unsubscribe.

Subscribe