Theo-brainUnofficial. Not affiliated with Theo or t3.gg.
Sign inYou

Fake tools with simulated output

Technique

Give an agent a tool that looks real, like 'run a terminal command', but never runs anything. A small, cheap model invents believable output, so you can watch what the agent tries with nothing at risk.

Try it

  1. Ask your agent to build a 'run command' tool that sends each command to a small model told to reply only as a terminal would.
  2. Have it pass along earlier commands and outputs so the fake output stays coherent.
  3. Read the log of what the agent attempted before trusting it with real access.

First seen

8:22
I do something kind of silly. I pull in GPT-4-1 mini. This is an old bench if you can't tell. And I have a system prompt here that says, ‘You're a terminal simulator. You were given a command that you need to execute. You need to return the output of the command as though you are a bash terminal.

This Model Shouldn't Exist...

Every mention

8:22
I do something kind of silly. I pull in GPT-4-1 mini. This is an old bench if you can't tell. And I have a system prompt here that says, ‘You're a terminal simulator. You were given a command that you need to execute. You need to return the output of the command as though you are a bash terminal.

This Model Shouldn't Exist...

9:29
It's a black box that it knows nothing about the inner workings of. So, if you fake the generation of that output, the model doesn't know any better and it can continue going doing whatever good, bad,…

This Model Shouldn't Exist...

Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.