But Theo, aren't all of the other hosts going to add it, and the providers will race the price to the ground? Let's take a look. Oh, huh. Seems like pretty much every provider is offering it at the same $3 in, 15 out, other than a couple small ones, but only…
…revenue cap and I think it's 10 mil a year rev where the license now requires you to do a deal with Kimmy and it seems like the terms of those deals with moonshot are that you have to match the MSRP rate otherwise these companies…
…you to figure out from a reasonably randomized set how long my average prompt ran for. So from when I sent the prompt to when the response stopped generating, how has the amount of time changed from January or February to now?
He has both 6 Luna and Haiku 5.5 on no reasoning. OpenAI has trained their models to be good at that. Anthropic has trained their models to be bad at that.
…is worth, when I tried out Haiku 55 on my machines with Claude code and my normal Claude code sub, I was seeing around 180 tokens per second for my just traditional prompting working on things…
Higher score, not a tenth the price, almost a hundredth the price. 58x lower costs exactly 1 year later. This is where I realized I was wrong. I thought the number was a pretty consistent 10x. It's not. It's closer to a hundredx year-over-year decrease at a given level of intelligence.
I thought the reason was because they included the 2x limits here and they might not have in the other chart I was in. That is not the case. The reason is much sillier. This version has resets because if you didn't know this, OpenAI resets a lot, 11 times in the last 30 days.
I want an answer. So we're going to figure it out together. Will it give me 50% back on my week? Cuz that's what it should do. Here it shows for the week I have 51% used and for Fable I have 0 percent. So it does indeed give you back half your weekly limit. Cool to see.
The $200 Claude plan is the best value; run heavy work on subscriptions
The $200 Claude subscription gives far more usage than its price at API rates and is the best value for AI money. Run long or heavy work on subscriptions, not full API prices.
I did briefly consider it, but I'm doing thousands of dollars a week on Fable right now. It's only viable because of the sub discounts. Soul is a much better value per dollar, and you get way more out of your sub as…
It is a little bit too easy to burn through your codec plans right now. And since the only options you have are Astra, which is capable but spiky and way too expensive, or Soul, which we'll be talking about in a new video soon, this is a way better value for your 200 bucks right now.
…in the past few days, you get around $2,300 a week of usage on the $200 plan. That puts you at almost 10 grand a month for 30 days with your Cloud Code sub on the $200 plan.
…find the absolute best way to spend your dollars on an AI right now, it's not going to be local GPUs, it's not going to be some weird budget plan from some Chinese lab, and it's certainly not going to be paying for your tokens on OpenRouter. The $200 Claude Go plan is unbelievably good of a value, and I…
You will get more tokens and more work done for $100 in a Claude sub than you would get for $300 on literally any other model in the world because that $100 sub's going to get you like three to five grand of…
Max doesn't raise the ceiling on thinking. It forces heavy thinking even on easy steps, costs many times more tokens and often scores worse. Use high, or xhigh when you need a bit more.
But as I've told you guys a bunch, I don't recommend using max. I recommend high or maybe x high. And as you see here, the score gap is not that big all considered. High uses half as many tokens at 28k and gets nearly the same score.
…to max, which is just a one little bump inside of your slider, it went from 300ish tokens per task to almost 5,000 per task. I'm not exaggerating. That's over a 10x gap for that tiny little jump. And you get almost no improvement in score here at all. It's a 1% bump.
I had a thread go for 6 hours and 30 minutes writing a markdown plan because I had it on max and it just ran in loops forever. I ended up having to stop it and then switch to XH high and…
Max isn't just making it so the model can think more, it is removing its ability to think less. And that's what scares me about max reasoning, and why I specifically think you shouldn't…
They're both fine. I don't have any issue with either, I can barely notice the difference with either. X high feels like it takes a little longer, high sometimes misses like smaller details that X high doesn't.
…where from low to X high you get a 5% increase in token usage. And from X high to max you get, and I'm not exaggerating, a 1,500% increase in token usage. Literally 15 times more tokens for that one little bump at the end because you're no longer letting it be done with easy tasks.
…because if the model is told to keep overthinking the thing, it's going to overthink the hell out of it. It's going to start second guessing itself and is going to start getting wrong answers…
Also, funny, uh 6 Astra on Max scored worse than 6 Astra on X High. Once again, as I've been saying for a while, you should probably avoid the max reasoning levels. They do more damage than good.
Theo has repeatedly underestimated how far models would go. Flat benchmarks meant we needed new benchmarks, not that progress had stopped, and developers who don't adapt risk their jobs.
I genuinely thought we had hit the ceiling of what models would be capable of, like, 2024 into bit in 2025. I was obviously entirely wrong. I never would have guessed agents in the new era of post training would push as far as it…
…has, and I definitely wouldn't have guessed that a new pre-training run that was so much bigger, with things like Fable and Astra, would make the massive difference that it's made. But it has.
That the YouTuber noticed an error in his logs, asked Claude about it, and now has a report that the CTO of Cloudflare hit him up for to get more info about. If you don't think these things are changing, I am sorry, you are going to lose your job in 3 months. Accept where we are or die.
I need to confirm this with y'all. Like every time I think there's like here's the line where models can't do it. There was the line, the models just blew past it. There was nothing special here.
…role is over time because if you haven't changed how you code over the last year as a result of these tools, you might not actually have a job in the next year or two. We are at that point now. I've danced around it for years and I was wrong.
Hands-off work depends on verification. Give the agent checks it can run, ask what it couldn't verify, and accept a check that is merely close enough to what you want; a score agents can push up works best.
Instead of just blindly merging Astra's crazy work here, I asked Fable what its thoughts were. I took a screenshot of the claims Astra had alongside the PR and said, ‘Is this ready to merge?
It said it could partly debug crashes, latency, and resource problems in staging, but it couldn't confidently debug wrong query results, and that's the scary class of bug.
…kind of insane to not look, but as I've talked about in other videos, you should build good verification systems so your agents can check if the thing works, but you should also build good safety nets.
In T3 Code, we have a nightly that goes out every 6 hours, and we get feedback immediately if it breaks something. So, I can fix it before the next stable release goes out.
It gave the model something to pursue and work on and as long as it has something it can check and decide if it's good enough or not and then keep going if it's not, you can get the model running for a while. And as long as the thing it is checking is aligned enough with what your goals are, you're going to end up with something at least somewhat resembling what you asked for.
Programmatic ones are better. And if you can take a problem you have and turn it into a program that gives a score, your agents are going to make that score go up.
Describe the problem or outcome and let the agent find the files and approach. Walking it step by step wastes your attention; if it fails, that tells you the problem deserves your own thinking.
…further in the other direction where I tell it not what solution I want, but I'll give it a bit of the problem I'm having. And if it's vague enough or I'm unsure enough, I'll tell it, hey, I don't know how I want to solve this. Propose some solutions.
Maybe instead of investigating the code base and telling the agent which files to touch, you tell it what the problem is and tell it to find the files and change them itself.
If the model can root cause a bug, fix it, verify that it's fixed, film a video showing the results, put up a pull request, monitor it to address all the review comments as they come in, and then tell you when it's done, if you're going through those steps with the model, you are the same person as the one who would write the type check after the type check in a TypeScript function.
So, now I just hand the screenshot to the agent and say, ‘Fix it.’ And it often does. And if it fails to, I now know this problem is too hard for the agent to do itself. At which point, I know it's time to think more about it.
When I realized that my team doesn't need to be handed solutions, it needs to be handed interesting problems, and they would usually come back with solutions.
The cost of a given level of AI capability is falling fast, so today's expensive experiments will soon be cheap. The machines agents run on, not the models, are becoming the cost constraint.
To everybody who's complaining that AI is getting more expensive, I hope you understand that more intelligence is the thing getting more expensive. At a given level of intelligence, the costs go down exponentially.
So, one of these things goes up by 25 to 30% every year, the other goes down by 10 to 100x every year. We're going to quickly get to a point where inference is cheaper than the VM.
…which point, compute will be a constraint for when we're building with agents. Right now, we're being given the VMs for free, but that is not going to last.
…promise of intelligence 10 times cheaper has been blown out of the water. We absolutely achieved that. The tasks that we used AI for last year are actually that much cheaper.
…much, but even if she legitimately wanted to spend two to three mill a month, if I was the CEO of Cursor, I wouldn't even think twice. I would ask if she wanted more.
She's not doing it cuz Cursor expects every company to spend 10 mill a year on tokens. She's doing it because that 10 mill on tokens is going to cost $10,000 in a year or two.
And remember, intelligence gets four to five times cheaper a month right now, which means that around this time next year, loops like this are going to be pretty cheap except for the compute that they're running on. So if you can find the opportunities to build these skills today, they will be more and more useful over time because the way that we…
Price per token misleads because models use very different numbers of tokens and requests to solve the same problem. A cheap model that burns tokens or fails can cost more than a smart one that finishes.
Notice that he said request here, though. A request is not an accurate way to measure token usage because one prompt can result in a very wide varying number of requests being made.
And the tokens are cheaper. So, cost per task is what we should be looking at and real-world cost per task on something like Terminal Bench 4 is a 13x decrease. So, the dollar value of the sub being half as much ends up still being five times more tasks depending on the work…
…problem, which is the thing I think most people miss is they look at the dollar price per token and they forget that certain models use more and fewer tokens.
…already, as I've said many a time, the token cost does not matter. The cost per task is what matters, especially when you consider how many tokens are being used in a given task. Cash rate efficiency matters a lot too.
I found that in a 5-second window, there was a 20% chance of a cash miss. In a 30-cond window, there was a 34% chance. In 120 second window, it was even more. This results in as much as a 50% increase in requests if they are waiting…
You will ship something broken however strict your process is, so invest in catching and undoing breakage fast rather than shipping slower. Hands-off shipping is only safe once good staging and testing exist.
The fact that a change that breaks things will be called out and noticed before it hits even a thousandth of our users makes it so much easier to hit merge and worry later.
…next example is not meant to be a thing that you do in your day-to-day work on like a real big important code base cuz it's a dangerous. If you have really good systems for staging environments and QA and testing before things go out, maybe play around with these types of workflows cuz they are super super fun.
…you guys have no idea how long I've been harassed for thinking it's more important to be able to undo a bad change than to add thousands of layers in front of the changes to make it slightly harder to make a bad change.
…kind of insane to not look, but as I've talked about in other videos, you should build good verification systems so your agents can check if the thing works, but you should also build good safety nets.
In T3 Code, we have a nightly that goes out every 6 hours, and we get feedback immediately if it breaks something. So, I can fix it before the next stable release goes out.
Teams decided that they'd rather get faster at noticing and fixing problems rather than slow down to prevent them. And I agree there. If your solution to not shipping breaking changes is to just go slower, then you're not actually solving the problem.
Notice what frustrates or slows you and turn it into a fix right away, following each bottleneck one layer deeper. Repeated instructions become skills and slow tools become projects.
If you find where there is friction in your flows, if you pay close attention to what keeps you from merging things, what keeps you from being excited to hit that button, what risks…
…having has been focusing in on where my friction and pain points are. Like what is frustrating me when I use my computer and how do I get rid of it to the best of my ability? How can I offload these things? And one of those things is navigating GitHub because it has gotten miserably slow. It is hell to navigate.
I realized the quality of the code coming out was better than I expected, so I introduced the full send skill to see it all the way to the end. Then I missed that flow a whole bunch, so I made the full stop skill to get as much of it as I could without having to…
So, I asked it how to fix that. It created the queue. I asked it how to speed things up, and it told me it needed more compute. So, I brought all the computers I had around online, gave it SSH access, and let it split the work across those. Helped. I moved over to Blacksmith cuz I wanted the compute back. Helped even more.
But, each of these jobs was still taking way too long. We're talking 20 to 40 minutes for my builds, just to get the code in a state where I can check if it worked or not. That inspired me to do my other insane YOLO project. And that's why Lust exists, because I didn't want TS Rust to take so goddamn long to compile.
…point where it's a whole request that isn't good, that shows an opportunity to improve things, to make it so the next person doesn't make those mistakes. And if you don't take those opportunities as soon as you see them, the quality of everything around you is going to erode…
Benchmarks no longer tell the story; check what was measured
Headline benchmarks and vendor charts can miss important differences between models, so check what a test actually measured and how it was run, and trust your own tests on real work over published scores.
Changed Nov 12
He used to say
Fable's lower score in his pull-request tests was a flaw in its reviews.
I did a deeper audit of the comments that Fable leaves on things compared to Opus and Sonnet. The ones Fable missed, the reports Fable didn't file, as well as the less important things or the incorrect things Fable said, the gap doesn't matter.
Benchmarks don't tell even close to the whole story anymore. If they did, then I would actually use Opus 5 willingly. But, Opus 5 sucks. We all hopefully understand that now.
…because I don't want to make the mistake I made in the Opus 5 video where the model seemed totally good as you were using it in a little bit of test type work and it killed it in the benchmarks, but then when you actually started merging the code, it was nowhere near as nice to use.
But the thing I want to talk about here is how the benchmarks no longer represent reality. And the numbers we look at when we talk about models just aren't the right way to do it anymore.
This is such a weird set of benches to include. You can tell they cherrypicked the hell out of this. To their credit, benchmarks are not very good at representing what makes this model good.
This is the type of chart that you would see and think something went wrong. Not the type of chart that you would publish is the first chart in the blog post, but sure kind of sad that the only place it is outperforming opus for the cost is in the max reasoning effort, which again you shouldn't use.
But again, when you say a model has 100% success in our evals, the eval and its relation to the work you're doing matters and I don't think a lot of people are making a living naming skateboard tricks. I wish they were.
If you build a fancy context management system, instead of just giving the agent the tools it needs to find [bleep] you're behind the curve now. You just are. It's not real.
Maybe, but probably not. But if you're using Frontier models with the $200 a month subscriptions and you're okay with wasting some tokens on how you get the context to the thread, it doesn't end up netting to that big a difference and it will make it way easier to just dump the thing out of…
They pick a number based on the behaviors they see. And Anthropic picked one million tokens of context because that is the point at which they think you might start to see enough degradation that you shouldn't go further. The dumb zone doesn't matter that much.
And I think the fact that I had a 70k token context thing at the front of all my prompts for the last 2 months and didn't even notice shows how little managing your context yourself matters at this point.
Hand each new model the next layer of work, including merging
As models improve, let them cover more of the work, including merging PRs they can verify. With experience, decide per change what the model can check itself and what still needs your look.
…found that I'm telling this model more often to merge the PR when it decides it's ready, and I've yet to be burned by that despite it doing it dozens of times for me already. 5.1 is picky enough about what it thinks is good enough that I trust it to do that, and I haven't…
…the model to autonomously find PRs, confirm that they are good, confirm that they're fixing things that matter, and if they're not making like meaningful product changes I might have opinions on, just let it merge them, especially for bug fixes and performance improvements.
…little nervous of these changes because I've been bit so hard by letting soul yolo merge in the past. So, I thoroughly tested all the changes it made here, and everything was good. A lot of that comes from the model's ability to test its own changes and to coordinate swarms in order to verify the work it's doing more…
I've had Astra merge over a 100 PRs across my projects and I've had Fable merge at least 50. And of the 150 PRs that they merged and wrote and did everything themselves fully autonomously, two kind of had regressions in them. That's a better hit rate than most talented developers have.
…done enough of this type of like agentic coding, you know what the models can and can't do, what they can and can't verify, what needs an extra look, and what doesn't. And once you've built that intuition, it's relatively simple to know this thing can probably merge itself, this thing probably needs another look.
Gemini models invent facts about real codebases, so don't use them for coding or review. If you use one anyway, run everything it produces through a stronger model.
I bet you didn't expect 37 flash on low would do 73,000 tokens and that on high it scores worse than on medium by a little bit and is using 107,000 tokens.
…you do this many output tokens for basic work and have the unreliability of the to the point where you drop points when you increase reasoning, the model is trash. It should be treated accordingly. Just don't use Gemini.
I would argue to an extent that my disdain towards Gemini comes from the floor being so low and so consistent where it just it'll read the same file 26 times before making a change cuz it's a [bleep] model.
Astra gave its findings a two. Of its 78 findings, 56 were wrong. That is a 70% failure [snorts] rate. This model is more likely to gaslight your agents than it is to give useful feedback in reviews. If you do decide to use this model for real work for some reason, for the love of God, please put the things it does through Soul or Opus or some model that doesn't have this egregious…
Gemini 4 was hallucinating columns in my SQLite tables. After reading the file that is my whole schema, it would then fire a query for columns that don't exist.
Keep instructions lean: explain how to think, prune the rules
Over-engineered prompts and piles of 'don't do this' lines hold agents back. Explain how you want the agent to think about the work, strip your instruction file back now and then, and write or audit skill files yourself.
And my advice is often to take one step back and find a way to communicate not just what you don't want the agent to do rather how you want the agent to think about what it would do. And I've been writing my agent and claude MD files like this accordingly because I think it helps a lot with…
This is definitely one of those model releases where you should go look at your Claude MD and see what is deletable. Try deleting everything, see how it behaves, and then add parts back as it makes sense to.
This helps avoid that. Astra still falls into it. Fable falls into it even more. Please write your skills by hand or at least audit them and make some nice changes.
I'm actually here to say the opposite. I think that engineers suck at prompting, but I don't think it's cuz they're not engineering it enough. I think it's cuz they're over-engineering it, and they're not taking the opportunity that prompting offers us to take a step back and rethink how the computer can work…
Keep one default model; have other labs review its work
Stick with one default model for everyday coding (Theo uses Opus 5.5), but bring in models from other labs to review the work. Models with different training catch different things, while models from the same lab mostly repeat each other.
Changed Nov 15
He used to say
Opus and Sonnet weren't good, so his Claude accounts were only worth using for Fable.
I literally just use it for Fable. Opus and Sonnet piss me off. They're just not good right now. I thought Opus would be good. I was wrong. I will keep eating that one for a while.
It is very rare that I swap off of my default model once I've picked what I'm using currently. The main reason I switch between models is because I have tokens to burn from a different subscription. Other than that, I'm usually just sticking with at this point as of filming Opus 5.5.
…calling this out cuz 5.5 is really solid overall and has changed my way of using things. Opus 5 was a useless dumpster fire that I had to work around a lot. 5.5, pretty good model.
…what it brought up was that of the 1.8 million lines of code, 1.3 million were not being used. The reason was because Opus concluded whole of the code from all the other agents was useless slop that had no chance of being recovered, and it chose to rewrite it from scratch itself in another crate.
…still finding the most novel stuff, but it's also throwing a shitload of nits into its findings. Soul is not throwing a bunch of nits in, but it's also not finding anywhere near as much stuff. So, if Soul is your only reviewer, you should pull other things in. Muse was a weirdly good middle ground here.
I personally find that OpenAI models are the most thorough and find things better than others do. Claude models can find like really weird, potentially dangerous things that other stuff will miss. And then Muse and Gemini both can find things here and there, but I find that Gemini hallucinates more than actually finds.
…the 55 models from Anthropic are distilled from the same base model that is likely Fable 55 or whatever equivalent they have internally. They used that model to generate data that they trained Opus, Sonnet, and Haiku with, which means all of these models…
What I have noticed over time is that getting different models from the same family to review things kind of just feels like asking the same person at different times of the day to review something.
It is very rare that I swap off of my default model once I've picked what I'm using currently. The main reason I switch between models is because I have tokens to burn from a different subscription. Other than that, I'm usually just sticking with at this point as of filming Opus 5.5.
Agents grind through big, tedious rewrites, so jobs once not worth the effort, like replacing a rotting app or porting tooling to another language, can now pay off even for a small team.
It took like five prompts to get this to the point where I would want to use it as my main player and like eight to get it genuinely far ahead of the competition. It's silly. All these legacy apps that have been rotting for years can now be replaced in…
The more tedious a task I can come up with, the more likely it is that the models can take advantage of that and do crazy things. Rewriting a 100,000-plus file code base in Rust would be miserable as a human, but an agent doesn't care.
Languages are an implementation detail. They're like a speck on a spec sheet now, and they can be swapped out. They can actually be replaced in a loop in not much time.
Watching a running agent feels productive but doesn't make it finish sooner. Let work run in the background and only surface when it's done or needs you.
…but that thread's not going to complete any faster if you keep it open. And 99% of the time you're not going to see something going on in the thread that you really want to stop. So, just go do something else.
You guys need to stop paying so much attention to your agents. I've been saying this for a while now. You guys should push your limits in your trust and see where it goes. I like this. I only watch if the first output is horribly wrong.
…you get in this mindset of like run the thing in the background and then let it pop up when it's ready, you realize that it doesn't matter that much how long things take as long as you don't expect it or need it immediately.
…don't want things in my face unless they need my attention and I found that every other one of these AI building and coding tools is trying too hard to put everything in your face and make everything accessible. I don't want that. If the thing is working, get it out of my face.
Spending heavily on tokens is a good deal when it saves engineers' time or multiplies a strong engineer's output. A cheaper model that needs more of your effort before and after isn't a saving.
Maybe, but probably not. But if you're using Frontier models with the $200 a month subscriptions and you're okay with wasting some tokens on how you get the context to the thread, it doesn't end up netting to that big a difference and it will make it way easier to just dump the thing out of…
Has it burned more tokens? Yeah. I don't care though because despite the tokens being expensive, so are my engineers. So is our time. If Julius can be three times more productive by spending his salary in tokens, that's a no-brainer.
But my time is more valuable than Fable's. So I downgrade to a cheaper model, I have to put more time in before it fires, and more time in when it stops.
…much, but even if she legitimately wanted to spend two to three mill a month, if I was the CEO of Cursor, I wouldn't even think twice. I would ask if she wanted more.
She's not doing it cuz Cursor expects every company to spend 10 mill a year on tokens. She's doing it because that 10 mill on tokens is going to cost $10,000 in a year or two.
…had one thread that had spun up like 600 plus sub agents in order to go through at least 20 rounds of review like this, and it just wasn't making progress. It took what was supposed to be like a 1,000 line of code PR and it bumped it up to like 20,000 plus trying to address every single edge case one of the agents…
Neither subscription value nor API price shows the labs' real costs. API prices carry huge margins that effectively fund cheap subscriptions, which exist to get developers used to heavy usage.
The reality is that not only is this $200 number made up, so is the API price on the right. Both of these numbers are fake, and neither of them represent the actual costs to the companies when they're running these things.
…it's also worth noting that these $200 subs don't make the labs much, if any, money. The point's to get us addicted to using 12 grand of tokens a month so that when we go to our employer, we're expecting to spend a similar amount there and they'll have to pay those API…
Everyone looks at this as the crazy subsidization of the subscriptions and it is that. But on the other hand, the margins that they charge are so insane on the API prices that they get away with it. Think of it as some part us paying less on the sub, some part enterprises paying a huge tax on top of…
Agent harnesses are good now, so don't rebuild them or tweak settings you don't understand. Pick the option with the capability you want at the price you want.
…actually think it's a good idea to use a non-reasoning classifier model for compaction for your context, then I would plead that you never ever ever stray from the defaults in those tools because you just fundamentally don't understand yet.
We're not just going through your history and selectively deleting lines from it. We are synthesizing a summary based on everything that's happened so far.
The harnesses are pretty dang good now. You should lean into the fact that billions upon billions of dollars are being spent on that and not reinvent the wheel constantly.
The best thing you can do if these things are confusing or concerning to you is don't change the defaults. The only settings I have changed in my Claude code config on my machine are I have it full screen by default, and I have it route through my proxy layer. That is it.
Okay, memory's a fair point here actually. I did turn off memory. I do not like the memory in Claude code. I'm sure it is useful in some places in some ways. It is not useful for me.
…everyone spamming me that I have to check out their harness because it benched so much better than Claude code, congrats on your 3% win with a last-gen model or with an open-weight model that wasn't trained on Claude code. I'm really happy for you. I'm happy that you can spend more money than my subscription to get fewer tokens and less work done.
Which means that we shouldn't be thinking about the harness the same way we don't think about the kitchen when we go to a restaurant. We should just go to the place that has the food we want. We should just select the option that has the capability we want at the price we want.
If agents keep failing in your code, a new hire would too. Fix the code's structure first, then add lint and type checks, and only then instruction files; piling on more instruction files is not the fix.
And I really like that cuz you really should go through these top to bottom and try to apply them out of each layer until the problem goes away. The best thing you can do always is categorically eliminate the problem through better architecture or choice of data structures.
You should adjust your prompts a little, but you should adjust your code base a lot. If agents are screwing up in your code base, then a new dev would, too.
Like I don't want to just keep giving the model more instructions before it goes in the door. I want to make it so the room it's navigating in is as easy for it to navigate as possible and I don't think more and more of these review MD files are the solution at all.
When you have to correct your agent, you should look for the place closest to the top to fix what caused the agent to do things wrong. The top spot is the code base. If it's making mistakes in the code, see if you can fix the code to make it less likely to make mistakes. Don't do this by putting comments everywhere.
Agents stall when unsure they're allowed. Pre-approve what they'll need, tell them they may keep going, and explicitly allow deleting code or scrapping the PR.
Like if you add to the claw MD something like assume you have my permission to keep going. If I state what I want the outcome to be, he'll climb till you get there. It's much more likely to actually get there.
This is important because agents need the permission to do things that they're scared of and deleting code for whatever reason agents are very scared to do.
Readable reasoning is worth more than compressed reasoning
Making models think in fewer tokens or hiding their reasoning makes them harder to monitor. Being able to see what the model is actually doing matters, even when the output is good.
Since OpenAI's reasoning traces are so efficient, they're a lot less readable. I talked about this before in my video all about how OpenAI models are so efficient, and it just didn't really dawn on me at the moment that this would hurt the monitorability as much as it is.
The problem there is that if you try to communicate the same thing, the same ideas, the same concepts, the same plans and work in a half as many tokens, you're going to make it harder to understand and monitor.
But, OpenAI uniquely encrypts the sub-agent prompts, which sucks. Mostly it's cuz I can't show them in the UI and learn from them. It's not even like I can't distill. I don't care. I'm not distilling. It's that I cannot see what the model is doing.
macOS bottlenecks running many agents at once, while a cheap Linux machine handles dozens. Put agent work on an inexpensive Linux box rather than a Mac or a rented server.
But I will just say out right now, Apple's file system does not have compromises that make sense for us as developers really at all. The performance is really bad.
I've had points where on this machine, I was running 40-plus agents, and it wasn't breaking a sweat. And it's not even cuz this computer's super powerful. Obviously, 32 cores is a lot.
…running Linux, not macOS. macOS has a ton of problems that make it really hard to reliably parallelize your work to the point where I just don't like doing it on macOS anymore. Generally speaking, I treat my Macs as one or two tasks at a time, and I treat my Linux boxes as…
Need 32 gigs of RAM and 16 threads? You can get it on Hetzner. It's only $275 a month. That's a great deal. Or you can waste all your money spending $700 on a box that has 32 gigs of RAM and a 1 TB drive. That would be terrible. That's such a waste of money. That's like 2 and 1/2 whole months of renting a worse computer on an IP address that'll get you banned from Claude.
You're right, your MacBook is overheating because your MacBook has a file system and an even shittier security policy that is hard bottlenecking how much you can run at once.
You need to pick up a crappy little Linux box. You need to take some old computer that you haven't used in a while, install Ubuntu on it, set it up with Tailscale, install CLI proxy on it so that it's always coming from your residential IP, and then you use it as a box that your agents can work with.
The most budget option by far is to talk to your friends and family and find somebody with an old laptop or desktop that they'll give you for free that has at least 8 gigs of RAM and at least what or at least four cores, and you'll be able to do some real work there.
The amount of stupidity a thing has and the amount of intelligence a thing has can coexist. Look at GBD-6 Astra. It is both the smartest model I've ever used and without question one of the stupidest I've used this year.
Opus is even more smart and way less dumb. I would say Opus is like three out of 10 dumb and 10 out of 10 smart feels right now. This model takes a hit on the smart. The ceiling's a bit lower. It's like an eight or a nine out of 10. But the dumb is even lower somehow. It's like a one or two out of 10.
Plan your spending around how each subscription resets. When a reset is close, burn the usage you have left on extra or experimental work, because unused usage simply vanishes.
The difference is the dates. When Codex does a reset, it resets your weekly entirely. So, if your weekly was going to renew in a day or 10 minutes or whatever else anyways, your reset is now 7 days off.
I'm using Opus for my serious work anyways, my Codex account being at zero is fine. It's not going to block me a whole lot. So, as soon as there is a whiff that a reset is coming, I burn that account to the ground.
…the quickest way to burn a bunch of tokens and get nothing out of it is to add /goal to the front of your prompt. I do still mostly believe that as a piece of advice, but mostly because if you remove the slash goal and you write the right prompt, it should do the same thing.
Use your Claude subscription only inside Claude Code, and other subscriptions only where their maker permits. Anything serving your own users must go through the paid API.
But if you are trying to serve public traffic, where users can submit a request and it runs on your inference, that is entirely against the rules with the accounts. If you're using this as an alternative to the API to serve user traffic, you deserve your ban. That's not what these accounts are for at all.
Do not, under any circumstance, use this to use your Claude Code sub in something other than Claude Code unless you really, really like dealing with Anthropic support and getting accounts banned all the time. You will be banned, you will be upset. I highly recommend you only ever use Claude models through your Claude subs in Claude Code over the…
So, tip number one, I love VPSs, we'll talk a lot about VPSs. I highly advise you do not sign in to Claude Coder Codex, but especially Claude Code, on a VPS. Anthropic in particular is very, very nervous around people reselling their Claude Code subs in VPSs serving…
So, you should only use this with Claude Code if you're using your Claude subs. I use my Codex subs with Codex. I use my Claude subs with Claude, but I also use my Codex subs in other things like Claude through this because OpenAI doesn't care where you use your tokens effectively. Anthropic cares a lot. OpenAI doesn't.
…been a lot of skepticism around using your Gemini sub and other tools. And I'll be direct, you shouldn't do it without Google giving the tool permission.
Engage with what someone actually said, not who they are. People who answer facts with motives, like 'you're paid by the labs' or 'it's marketing', are dodging, and you can't have a conversation without agreeing on facts.
…good friend with David Cramer, and say you should not listen to a thing he says about AI, even though I did a panel with him 2 weeks ago about AI, because I legitimately think the way he talks about these things are actively harmful to the space that we are in.
The folks I'm warning you about here are going to do the opposite. They will never engage with the points. They're going to steer around them and say something else, like, ‘Well, yeah, you're paid by the labs.’ Or, ‘Well, yeah, they're doing this for marketing.’
Ask for sub-agents outright; don't wait for the tool to offer
Models won't always reach for helper agents on their own, and you don't need a menu to choose them. For big audits or piles of reviews, just tell the agent to use sub-agents, and say which ones if you care.
…saying here is that I've noticed Opus 55 seems less willing to spin up sub agents unless you tell it that it can. So I find myself often telling at the end of prompts, ‘Use sub agents and workflows however you choose. I trust your judgment there.’
…to I don't know 61 Soul on medium fast, I can ask it spin up two Opus 5.5 delegates to explore performance wins in T3 code, summarize their thoughts, and then start a new thread using Grok 4.7…
Talking to your terminal is hell. So all of the strong stances I already had about terminals not being the right place for agentic dev. Oh man, you have no idea. I'm 100x stronger in my convictions there.
I do not get how people live in their terminals. I just voice to text it a paragraph and I can't read it. Get out of your [bleep] terminals, guys. Get over yourselves. You're just wrong. It like I I was such a big terminal person.
Tools and platforms aren't fixed; the people who make them can be worked with. When a tool misbehaves, report it or raise it with its makers instead of building a complex workaround of your own.
Can you hunt through open issues on Claude code and see if others are reporting this? It'd be really nice if we could call it out directly and try and get them to fix it instead of having to architect something crazy in our app directly.
…things are malleable, all of these things can be changed and improved, all of these things are made by people no more special than you or I. And you can just talk to them about things and work with them on things and get things…
Agents now review code better than most humans and never tire of checking again. Use AI review on your PRs, never open one your agent hasn't reviewed, and let agents sort real findings from noise before people look.
…know if any of them are sponsoring this video, but I've said many a time I couldn't live without the AI review spots. And this is another great example of why it had real findings.
I also have the hot take that most agents are now better at code review than most humans and if they're not better in your code base, that's because your code base sucks not because the agent sucks, but that's a…
Humans and agents are both lossy. The main difference here between the smartest AI and the smartest human is that the smartest AI isn't going to get burnt out if you tell it to check over and over and over and over again.
Get the goddamn prompt suggestions out of the code review. That is 2024 core. We're in 2026. I'm not copy-pasting from the code review back to my agent. My agent's babysitting the PR. My agent is deciding if your feedback is good or not.
That shouldn't be most, that should be all. I was saying this for a bit, if you're putting up PR's that your agent hasn't even reviewed yet, you are wasting the time of your team and it upsets me.
Labs' safety concerns are sincere; take them seriously
Safety warnings from labs are not a marketing trick: bad press hurts them and they have financial reasons to care. The dangers are real, and alignment has to be right the first time.
All press is good press is not true. It just isn't. Take it from someone who's gotten a decent bit of good press and a hell of a lot of bad press. The bad press does not help my businesses at all. It actively harms them.
If we don't get alignment right, we won't be able to in the future. This is not a thing we can ship fast and fix later. It's a thing we need to do correct the first try and the dangers are real and we need to stop pretending they aren't.
…think that Anthropic has lost a lot more than they've gained by talking so much about safety stuff, and I'm really tired of the conspiracy that they're doing this to market. It's just like it's so obviously insane that it's hard for me to fathom that people actually say these…
My conspiracy is that OpenAI and Anthropic have a really good financial incentive to care right now because if this happens in 5 years, humanity is wiped out.
But if it happens right now and we decide to shut down the AI companies because they're unsafe now, we'll never get to that point in 5 years and more importantly, OpenAI and Anthropic go bankrupt.
Newcomers who don't know how things are supposed to work ask agents for things experienced engineers wouldn't, and end up building surprisingly powerful things because nobody told them not to.
…where people who have no idea what they're doing can come in with ideas that do not make sense to us because we've been engineering for 20 plus years and these kids don't know the difference between Java and JavaScript, and they're going to make things that are really cool and surprisingly powerful because they don't know to not ask for it.
There is a unique advantage to not knowing how everything already works because it'll give you the opportunity to try things no one else would have tried.
Models retrained to remove refusals are far more willing to do real hacking work. Once a model loses its caution, handle it very differently and with great care.
And this isn't a theoretical risk, either. There's a company called Obliteration AI that I learned about today that just released a new model, which is based on GLM-53, specifically post-trained and RL'd to get out the refusal behaviors when you ask it to hack things. And now they made a model that is way more willing to do real hacking work.
And yeah, some amount this article is meant to get us scared about open-weight stuff, but it is also a genuine experiment to see what happens in these scenarios, and I find this…
And when you take away the model's trepidation and reservations, you now have to treat it very differently. I have like two separate friends that got their home network poned by just playing with an obliterated model because they're scary. So, like be careful.
Agent memory adds little in code, where you can trace exactly what something touches, so turn it off. Look at what it actually saves before trusting it.
Th this has broken me. I was already kind of skeptical, but I hadn't taken the time to look at what it was doing. I'm going to go turn off memory on cloud code across all of my machines because this is [bleep] useless. Should I ask how often are memories actually being used in sessions? I'll ask that next because you have my curiosity.
Like in a codebase, if I click a button and it doesn't do what I expect, you can programmatically trace from where the button is that I clicked to every other thing that gets touched.
There is no direct path from one thing to the other in these more human problems where there is a very direct one in the code bases. So, I have found that memory as a way of tagging relevant information that is user specific can be a little useful.
The best thing you can do if these things are confusing or concerning to you is don't change the defaults. The only settings I have changed in my Claude code config on my machine are I have it full screen by default, and I have it route through my proxy layer. That is it.
Okay, memory's a fair point here actually. I did turn off memory. I do not like the memory in Claude code. I'm sure it is useful in some places in some ways. It is not useful for me.
Ask the agent what led it astray; it can point to the signals, tools and files that misled it. Then fix that source, such as a skill, so it doesn't happen again.
You can say, I expected you to do this, but you did this other thing instead. What led you there? It won't get it properly. The models rarely know exactly why they did a thing, but they will usually indicate what signals they found and what tools they called and what files they read that led them in…
If I notice things not working how I want, I ask the model, ‘Why did you do that?’ And it says, ‘I thought this because of this.’ And then I go update the skill to prevent the thing I didn't like. So, these bullet points here are all because OpenAI models were doing things I didn't like.
Important code is understood in full by nobody. Know roughly how the big pieces fit, and focus on understanding the agent's responses rather than reading all the code.
I got a hot take for you guys. If you truly fully understand all of the code in the projects that you're working on, you're probably not working on things that are that important or serious.
I almost entirely agree here. I wouldn't necessarily word it this way as a partial understanding rather like the depth of your understanding should change depending on where you are. I think you should generally understand how the pieces come together in a big code base before you make changes in…
You got to read the responses from claude and understand what it is doing when it does it. If you're actually reading all the code when you do these types of tests, you're not going to get very far.