# I deleted 90% of my harness (And It Got Better) — Philipp Schmid — session 2026-11-11T19:20:00.000Z → 2026-11-11T19:40:00.000Z _44 transcript lines · 38 slides_ ## Transcript Harness in the less code we need. And if we look at What we have been doing at Google DeepMind for the last six months. So at May, we had Google IO, and today is November 11, and we have been shipping relentlessly, basically. Just earlier today, we announced our first decision model with Gemini Decision one. Yesterday, we made Gemini four Carbon available. And then if we just look at the last six months, we basically ship now models every few weeks. As already said, Gemini Carbon is now available for you. So if you are a Google AI Ultra subscriber. You can use it in anti gravity, you can use it in the Gemini app, you can use it in AI Studio. Carbon is number one on the CursorBench, eval, and also number one on the perplexity benchmark. And internally at Google, we have been using it already a lot. And much more exciting is that I'm happy to share that we launched the Gemini decisions API roughly one hour ago with our new Gemini decision one model. So it's a similar model to Jeff. You can provide all of the existing Gemini inputs, meaning it's It's multi modal, so you can provide text, audio, images, video, and also PDFs natively. And then provide up to 1,000 questions in a single request. The average end to end latency, including all of the network overhead, is between one hundred and fifty to two fifty milliseconds. Milliseconds and it costs 10¢ per 1,000,000 tokens. So how do models get good at agentic work? And that's a rough overview on how post training works. So we have a task, we have our model. The model has a Set of tools and then we have some kind of compute environment and at the end we have some verifier. And the model tries to basically solve that task to use different tools and then based on the score from the verifier, we update the model. But no one is telling the model which tools to use and basically it only receives that reward feedback. So if we look at two examples on how a model can do a certain tasks with tools, we have the path A, where we have dedicated tools for file system controls. So you all I've probably used coding agents and you've seen a read file call or an edit file call. So that's one way of doing it. And the other way of doing it would be just using a single inline best script for all of the changes. So on the left side, we roughly need around 10 turns to make the same changes to a coding, repository, compared to the two chain, terms on the right side just using bash. And it also reflects in the amount of tokens we need. So the left path takes around 10 x more token. Stand the life, the right path, and it's doing the exact same thing, at the end for the model to achieve their tasks. So in my experiment, every roughly six months, I try to build a new coding agent harness from scratch. And I always ask myself, okay, how much of the harness do I really need to achieve the same task? And what is, overhead to the model and we no longer need? So if we look at the Coding Agent harness, which, from 2025, we had roughly around And to 15 to 20 tools. We had dedicated tools for, file editing. We had dedicated tools for planning, for to do list. And also, we had a lot of dedicated rules in our system prompt, which told the model what to do and what not to do. And that worked, of course, but since then we made a lot of changes. So in my experiments, first one was done roughly now two years ago, in February, where we did not really had agents. Right? We had workflows. We basically had predefined step. RAC applications back then were very popular. We had the planner, we had search, we reflected on whatever was found, and then we generated an answer. Then we started to see our first agents. We had many, many different tools for all sorts of fine kernel actions. And then came the coding agents where we had those roughly 10 to 20 tools. And now in my last experiment, I tried to see, okay, what's going to happen if I provide only one tool to my agent? And the outcome One tool agent, works the same way as every other agent except that it only has one tool, which is bash. And the model can use that tool for any sorts of tasks. And the bash tool is very simple, so it runs in a separate clean sub shell, and it supports the command argument. I also added a description so that it's nice and easier to display to the user what the model is actually doing and then like some async parameters. So basically when the model runs a bash command, the parameter tells the model, okay, if it is Longer running than this, milliseconds. It's being, that the tool call resolves, and the model receives, a local file handle where it can look up later, okay, if the process is still running. That's very interesting if you do, like, all sorts of, like, web development for example. And the single batch tool actually works for all of the things we are using agents. So we can have long running work where we can start on the foreground and then automatically move it to the background. The model receives some file handle and since it As bash available, if we store it to a file system, it can read the logs which are being streamed. If the agent itself is a CLI, it can call itself using bash. So there's practically no need to create sub agent tools for it because it can invoke itself. And then of course, we don't need dedicated planning or memory tools because we have our file system available. We can provide some context to the model so the model can keep track of its own work with the file system and using the best tools. And addition of course For our tools, we have system prompts. And similar to tools, system prompts evolved over time. Every dot here is roughly around 100 tokens, and you can see that coding agents mid twenty twenty five had a lot of context we needed to send in every request. Of course, this also improved, and similar story here is like, okay, what do I really need to provide to my agent to make it successful and work, and what of all of the system information and details we had. In the past are now basically part of the model. So how much of the harness helps and how much is in the way? And to find that out, I was running a lot of different evaluations of a lot of different agent harnesses with a lot of different tools. And one example I like to show here, which was done with Gemini four carbon from an eval task from DeepSVE is like, okay, what happens if I provide my existing coding agent tools with read, write, search, crap? And My Bash only tool. And I think you can pretty clearly see that the top line is taking a lot of more tokens, a lot of more turns, and a lot of more times to achieve the same task. And what stood out really well is that, okay, reading is done in parallel in the beginning, but as soon as we start going into making code changes, we see that same behavior that the model is making turn by turn a single edit, which of course, consumes a lot of time and a lot of tokens. So how does this look at a protoscope? So I have my own email set of around three fifty tasks for coding, knowledge work, and terminal commands, all run with Gemini four argon here, and four different agent harnesses. One is the bash only harness and then okay, with bash we still have that issue on how can I read images, right? If I cannot cut screenshot. Pnga, it would return a binary and it will Fill my context and the model cannot see the image. So I added a view media tool which basically returns the image in a way the model can see. Then we have two different agent harnesses with structure tools for file system operations. And if we just just look at the score column, all of them look roughly identical, right? So if we would just evaluate our model in an agent harness looking at the score, it would basically not really matter which one of. Those we would pick of course. There's a slight variance in the scores, but I only ran each task one time. So might be just some noise or maybe something else. So you would probably need to run a few more samples per harness, but the score looks roughly the same. So how would we now know which harness should I choose? If we just look at all of the events being shown, it's always mostly about score. But if we look at the token column or even more at a dollar per task. There's a picture which is changing. So the token per task is getting lower interestingly if we just use bash and then also the cache hit rate increases. So if we just compare like the bash media as the baseline, the tasks are almost two x cheaper compared to more complex, more tools agent harnesses, and we would not see it if we just look at the score. And to show you a few examples, I prepared some. Demo. So here I'm, using the Py harness which makes it super simple to basically define what tools you want to use. I have the the same prompt for 3.5 Flash which we released roughly six months months ago, for all tools. So like whatever Pi ships out of the box. Then we have it for, bash only, and then we have it for carbon and, sorry, for carbon with the default tools and then with with, bash only. This runs now since we If only have roughly around twenty minutes, I recorded all of that and clicked that very nice accelerate button here. And what we can see on the right side is that the same paradigma basically see with the default tools, compared to bash only, we we see with the default tools compared to bash only, we had the same amount of model calls, we had the same amount of tool calls, and we had roughly the same amount of tokens. And if we look at the auto trace. As we can see for bash only, we have some cut commands to read different files. We have some inline scripts to make, changes and then like some git changes. If we look at the default tools, we can see roughly the same. We had, read file commands, we used the edit tool, and then had some git commands to make those changes. If we now look at Carbon with the default tools, it already used way less tool calls and model calls. So it it's basically doing In work in parallel but also trying to reduce it. So we had for example the still same for write read calls and also the edit calls. But what starts to get interesting if we are going to look into bash? And in bash we had even less tool calls and model calls and the interesting part here is that the model writes a single Python script to make all of the changes in one go and no longer needs to call multiple tools. So if we go further, what does it means in And for for Carbon here and how Bash pays Office that, the better the model gets or the more reasoning and like capability it gets, Bash helps the model to be able to chain more commands in the single call. For Calvin, it was roughly 2.2 to 3.7 commands per bash call and for flash it was only 1.3 to 1.8. So we are seeing up to three x less tokens by carbon to achieve. The same task compared to having the same tools we need to to use. So two more examples to look at it as well is like we have on the left side, Carbon with the default tools. So we see our coding agent tools with write, with edit, and we see, okay, it does some parallel work in turn two, some parallel work in turn three and four, and then we have all of the changes. And on the right side where it just has the bash tool, it uses Python to write an inline script to make all of That changes at the same time with much less token. And the same counts for if we compare Gemini 3.5 Flash to Gemini four Carbon. Flash with bash only is not chaining those commands. So So here it is not really, it doesn't matter if I'm having a retool, a right tool, edit tool, and a bash tool, or if I have just a bash tool because 3.5 flash used to use bash for single term commands. So it had a single term for reading, it had a single Turn for writing. But if we look at carbon, which in here in this example is the exact same task, it needs less way turns. So the reading is not needed to read all of the different files and completely, it just uses script prep and like some intelligence searching to find where we need to make changes. And then it writes a single Python inline bash command to make all of those edits in one go, saving us a lot of tokens. So what does it bring us? Models will keep eating the harness, and I'm Pretty sure that 2027 will also be a lot more about how many tokens do we need to solve a task. Even for the few improvements we made between Argon and Carbon is the better model needs less tokens per task, which of course is very exciting for all of us because then we can use it more. And also it matters a lot what the harness provides. And we will might not see it if we just look at the accuracy. It's also very important that we start measuring the token efficiency because harnesses can look The same on accuracy because our models are now very good. So if your harness, is more complex to use, the model will work around it, will find solutions around it, but needs more turns and more tokens to achieve the same results. And then also what comes afterwards, so I will try to do the same experiment six months from now. And currently the trend we see slowly, rising at least on like social media and like the industry is that we might go back to some code. Mode tool that we instead of writing a bash commands that we write TypeScript or Python directly that MCP will come back. I'm a big fan of MCP and, like, they are making great changes to make it more, easy... Easily usable, Especially, interesting for all of that code mode work as, MCP servers now return allow you to return structured outputs, so the model can write better scripts to take more actions in the single term. And it also allows us to do some better prox. See around the security because bash calls are very hard to understand, but if I have some obstructive layer in TypeScript, I can, watch all of the commands being, executed and make sure those are secure. And for you, build to delete your code before the model does. Give your model, to a computer, not a toolbox. And when a new model ships, I definitely recommend you, trying to think a little bit outside of the box, be more ambitious, try We really see, okay, what if I remove my complete harness and just give it one tool? And then also, please scan the QR code. You will get free credits. We will send them out after the AI engineering event to try out Carbon and also to try out the new Gemini Decisions API. So if you have some time today, I really can recommend checking out the new Decisions model, Carbon, and thank you for your time. ## Slides ### 00:00:36 ## AI Engineer # Code Summit ### 00:01:10 # AI Engineer Code Summit ## I deleted 90% of my harness (And It Got Better) ### Philipp Schmid / Member of the Technical Staff Google DeepMind [QR code for live translation] ### 00:01:39 # I deleted 90% of my harness (And It Got Better) Philipp Schmid / Member of the Technical Staff Google DeepMind > Rich Sutton - The Bitter Lesson - 2 ### 00:02:10 # Six months of shipping, May to November ## Legend - Models - API - GenMedia models - Audio models ## Timeline (2026) ### May - ### 00:02:42 # Gemini 4 Carbon is now available Our strongest model for long-horizon agentic work ## #1 - CursorBench 4.0, Perplexity WANDR and Zapier ### 00:03:06 # Launched today: the Gemini Decisions API Gemini Decision 1 - fast, calibrated decisions in one forward pass - **Bounded choices** - tool selection, model routing, policy checks, ### 00:03:39 # How do models get this good at agentic work? Post-training: a set of tools, a sandbox and a reward No one tells the model which tool to pick. What ### 00:04:07 # Two paths to the same reward > "Rename amount to amount_cents in charge_customer and update every caller." ## Path A - file tools - 10 turns - `grep ### 00:04:39 # My experiment: how much harness does the model still need? The latest rebuild: one tool, a short prompt, and a computer. ## I deleted 90% of my ### 00:05:07 # A coding agent in 2025: ~16 tools and the rules to use them ## 16 tools - Task - BashOutput - Glob - LS ### 00:05:38 # Four rebuilds: each time the model ate another layer ## Feb 2025 - **4** fixed steps - **Workflow** - Query, search, ### 00:06:11 # A 1-tool agent: a computer instead of a toolbox Reading, searching, editing and testing are just commands. ```text # the whole tool list - bash name: ### 00:06:39 # A 1-tool agent: a computer instead of a toolbox Reading, searching, editing and testing are just commands. ``` the whole tool list • bash name: bash description ### 00:07:10 # Subagents, background work, memory: bash and files ## Long-running work ```bash bash - background after 120 s $ pnpm build # still running after 120 s backgrounded • pid 4182 log: .ga/tasks/bash-6a74.log $ tail -n 20 .ga/tasks/bash-6a74.log ``` ## Subagents ```bash bash - another agent $ ga run "Review src/auth for race conditions" > notes/review.md backgrounded • pid 4190 $ cat notes/review.md ``` ## Plans and memory ```bash bash - markdown files $ cat >> notes/plan.md <<'EOF' - [x] reproduce the bug - [ ] fix tax rounding EOF $ cat notes/plan.md ``` --- ### I deleted 90% of my harness (And It Got Better) Philipp Schmid / Member of the Technical Staff Google DeepMind [Three code blocks demonstrating bash commands for background processes, subagents, and markdown file manipulation] [QR code for live translation] ### 00:07:35 # Every dot is 100 tokens the model reads before your prompt - **Coding agent** (mid-2025 - 16 tools): 12.4 ### 00:08:06 # Same model, same task: 47 model calls or 18 > "aiomonitor lacks the ability to capture and compare task state over time. Add snapshots to Monitor ### 00:09:06 # Scores land within 6 points. **Tokens differ by 39%.** | Harness | Tools | Score | Tokens / task | Tokens vs base | Cache hit | $/task ### 00:09:39 # Scores land within 6 points. **Tokens differ by 39%.** | Harness | Tools | Score | Tokens / task | Tokens vs base | Cache hit | $ / ### 00:10:07 # Scores land within 6 points. **Tokens differ by 39%.** | Harness | Tools | Score | Tokens / task | Tokens vs base | Cache hit | $ / ### 00:10:39 # Scores land within 6 points. **Tokens differ by 39%.** | Harness | Tools | Score | Tokens / task | Tokens vs base | Cache hit | ### 00:11:07 # I deleted 90% of my harness (And It Got Better) ### Speaker Philipp Schmid / Member of the Technical Staff Google DeepMind ### Command Line Interface - **Model Configurations ### 00:11:39 # I deleted 90% of my harness (And It Got Better) ## Philipp Schmid / Member of the Technical Staff Google DeepMind ```bash zsh ~/billing-service/ ### 00:12:07 # I deleted 90% of my harness (And It Got Better) ## Code Editor View ```bash zsh ~/billing-service/ pi --model google/ ### 00:12:39 # I deleted 90% of my harness (And It Got Better) Philipp Schmid / Member of the Technical Staff Google DeepMind ## `billing-service/` ### Comparison of ### 00:13:10 # How bash pays off with 4 Carbon ## 3.5 Flash | | model calls | tokens | gain | |---|---|---|---| | Rename a parameter | 22 ### 00:13:39 # Default tools: a call per file. Bash: one program. 4 Carbon - rename task - both recorded verbatim from the runs in the demo. 1. `bash rg -n ### 00:14:11 # Same tool, same task: 4 Carbon chains, 3.5 Flash steps ## gemini-3.5-flash bash only - 22 model calls - 11 ### 00:14:40 # Same tool, same task: 4 Carbon chains, 3.5 Flash steps `gemini-3.5-flash` bash only - 22 model calls - ### 00:15:11 # Models will keep eating the harness. 2027 is about tokens per solved task. **-34%** tokens per task from a better model same bash-only harness, Argon to Carbon **+39%** tokens per task from the harness alone same model, heaviest vs leanest harness ## I deleted 90% of my harness (And It Got Better) Philipp Schmid / Member of the Technical Staff Google DeepMind [QR code for live translation] ### 00:15:39 # Models will keep eating the harness. ## 2027 is about tokens per solved task. - **-34%** - tokens per task from a better model - same bash-only harness, Argon to Carbon - **+39%** - tokens per task from the harness alone - same model, heaviest vs leanest harness ### I deleted 90% of my harness (And It Got Better) Philipp Schmid / Member of the Technical Staff Google DeepMind ### 00:16:10 # What comes after bash? One tool that runs code. - **1-tool harnesses evolve** bash for files and processes, exec(code) for everything else - ### 00:16:39 # What comes after bash? One tool that runs code. - 1-tool harnesses evolve - bash for files and processes, `exec(code)` for everything else - MCP comes ### 00:17:11 # Build to delete your code before the model does. 1. Give the model a computer, not a toolbox: bash and a file system. 2. When a new ### 00:17:39 # AI Engineer Code Summit PRESENTED BY Google DeepMind [QR code for live translation] ### 00:18:10 # AI Engineer Code Summit ## Presented by Google DeepMind [QR code linking to https://events.google.com/code-summit/] ### 00:18:35 # LIVE TRANSLATION ## //AVAILABLE Scan the QR code to access real-time translation. [https://translate.google.com/](https://translate.google.com/) ### 00:19:08 # AI Engineer Code Summit ## STAY IN THE LOOP SCAN THE QR CODE OR VIEW THE SCHEDULE AT AI.ENGINEER/SCHEDULE [QR code linking to https://ai.engineer/schedule] ### 00:19:35 [Abstract background of dark grey dots on black, with a central diamond-shaped cluster of lighter dots.]