Theo-brainUnofficial. Not affiliated with Theo or t3.gg.
Sign inYou

Performance ratchet in CI

Technique

An automatic check that runs on every pull request (a proposed code change) and measures one thing you care about, such as how much data a page sends. It posts the number on the change and flags or fails it when that number gets worse, so the gains you made stay in place.

Theo's current stance

Theo says small, innocent-looking changes kept undoing his speed work. The check shows the number in every PR and, most importantly, tells the agent this metric matters. Agents now often fix regressions before reporting done.

Try it

  1. Pick one thing you care about, like data a page sends, and ask your agent to build a check that measures it on realistic examples on every PR.
  2. Have the check post the result in the PR and clearly flag any meaningful regression.
  3. Set the limit a bit above today's good result; Theo used about 30% higher in one case.
  4. When a speedup lands, have the limit lowered so the win is locked in.

First seen

28:01
…a ceiling set here that's like 30% higher than when I got all the optimizations in. So if any changes push us at or over this line, PR fails, I am notified. I come in and say, ‘What the [bleep] are you doing?’ This has already prevented real regressions.

Turn off Claude Code's Memory

Every mention

40:41
…thread transfer impact tool, which is an action that runs on every PR that has a bunch of real event data from real Codex threads and threads, runs it through a fully simulated end-to-end pipe and checks how much data goes through the pipe.

Coding Isn't About Code Anymore

41:19
But the most important thing, this is really silly, it gives the agent the information this is a thing we care about and it should be optimized for.

Coding Isn't About Code Anymore

31:41
From then on, an APR that raised the instruction counts of these paths would fail CI. the daily job lowered each ceiling whenever the count went down. That's another good idea.

Anthropic Made Claude Even Better

29:37
…this because not only will the GitHub action catch these types of regressions by putting it in as a comment, both the agent working on the code as well as the agents doing PR review will notice it and call it out and potentially even fix it autonomously.

Anthropic Made Claude Even Better

28:01
…a ceiling set here that's like 30% higher than when I got all the optimizations in. So if any changes push us at or over this line, PR fails, I am notified. I come in and say, ‘What the [bleep] are you doing?’ This has already prevented real regressions.

Turn off Claude Code's Memory

28:15
I've noticed is my agents don't bug me until they fix them. So now when I make a change that affects the data layer and that change causes a regression, the agent fixes it before it tells me it's…

Turn off Claude Code's Memory

Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.