# The Software Atelier — Armin Ronacher — session 2026-11-12T17:30:00.000Z → 2026-11-12T17:50:00.000Z _55 transcript lines · 40 slides_ ## Transcript The the the agent, with the latest model than I did, in December last year. The same thing with, like, how often am I boarding it. And I think this to me was the most interesting way in which I sort of analyze my sessions, which is the average... So this is the p ninetieth percentile, but on the ninetieth percentile of how long an agent was running in response to a task that I gave it, with Opus 4.6 in December last year was about two and a half minutes. And now with the latest models, they're on the ninetieth percentile One. For fourteen minutes and more. And that does actually change something about how you wanna work with a harness, but really, again, nothing in pi has changed and still I'm I'm seeing this sort of different behavior of of how I'm using this. And this probably you have noticed in one form or another if you're still using an agent we're gonna see what it's doing. You'll probably notice that the present day agents are much more likely to use bash than any of the other tools that are being provided. But again, nothing has changed in how we configure the harness. Just that there's a lot more emergent behavior coming from the models. What's missing here is Astra and Sol. They actually look very similar to Opus for my testing. I just didn't have enough data in my test set, to show this nicely. I... You will notice that there's a little purple thing here. I want to talk about this in the end which is sort of code mode is sort of slowly creeping up. But the the core of this is, even if you have a piece of infrastructure nowadays that is sort of there and is Doing a good job. It's still changing as a result of the models. What has not changed in the last year, at least this is the case for me and I think this is the case for a lot of people, is that we're still humans, we're still limited by our fundamental capabilities. And as we are being more ambitious and we're doing more of these machines, we're... I'm not... I don't wanna say like we're the bottleneck, but we are the thing that pays us what they can do. And in particular, human attention is, for me at least, the limiting factor. And I'm noticing this With Largely in how, coding agents deployed in teams, that this is sort of a stable, amount of, like, attention that we have that we have to work with. And, starting new project is getting incredibly easy with these agents. And I think like particularly when a new model comes out, we're all super excited of trying the latest and greatest thing you can do from three d games to new ways in which we automate things. But maintaining them has not really gotten easier. But we are also increasingly tech Seeing a lot of external systems that actually in many ways are not supposed to be part of this revolution. I think we all know that GitHub has a lot more commits being pushed up there and a lot more repositories. But I know quite a few people who have well established all the websites on the Internet that found a new life for agents, and they are totally not ready for the new volume that's coming from from us kinda directly hitting them. So I I just wanna caution a little bit here in saying that when we are pushing things, we also accidentally simultaneously all of Pushing things, which we're not really ready for for what agents are doing. And this to me, I think is an interesting graph because, the number of prompts prompts that I'm actively pushing over the year is is sort of flat. I know that they're, like, in some months, it's a little bit higher, in some months, it's a little bit lower, but they're they're alternating between a 120, a 190, something like this prompts, that I'm pushing per day. But you can see that the Time that the agent was active for my use went from like less than my working day to many times my working day. And this November is not even over. Right? So you can see that this is taking an exponential curve up. And this here is the number of projects. This is git repositories that I've created over the last year. And this is like obviously cumulatively going up. So I have 141 different projects that I started. But the number of projects I'm committing on a regular cadence to, which means like I'm committing for more than five days to and I'm Returning to them also, is is pretty much flat with with between 10, 15, something like this. And I have I have done this sort of an informal survey with a bunch of my friends as well and this seems to be a common trend trend that actually maintaining something regularly is tricky. And, I had this discussion with a friend of mine, in sort of in relation to this graph and he said, like, well, actually, I have an open source project that is now fully automated. So, there's a issue tracker coming in. There's a there's an agent that's set up to auto You commit and send it back. And that is maybe a way to scale up so you take the human out of the whole thing. But actually, he also turned it off because eventually, he didn't care what the project is much anymore and he wanted his token specs to use them for other things. So our attention to an open source project matters even if maybe we have automated entirely. So if we stop using something where we have a tendency to no longer actually to actively keep... Take care of it. So if we're pushing towards and I would make this argument First figuring out how to make a really good workshop to build really high quality software, there's a question of what matters. And I think for me, the three most important things to matter is on the one hand to to really know when to push back against what an agent is doing, telling it that this is the wrong way of doing it. And I think that this is a crucial skill because an agent is really rewarded in just completing a goal. And it is very easy to look at if it has succeeded in this goal or not. And this... We can sort of see it in the same way as the the RL environment maybe is rewarding the the model during training. But that doesn't mean that what is being actually delivered is something that we are intrinsically happy with for a longer period of time. And this becomes tricky the more and more commits we're doing, the larger and larger the software is going. It's very easy to basically create the SIM SIM card of all the features in a software, but actually having something that intrinsically fits together where the internal complexity of the system does not as like Explode, still requires us to, to to work with it as it's working. And I I like to call this like true skill versus agent skill. There is clearly a difference between someone who is an experienced software engineer working with an agent and someone who is not an experienced software engineer. And, I think for me, the the truest form of this is that, my son is now sending me pull requests on a project that we're doing together. And there's a Very clear quality difference in what his contributions are because he is not a software engineer, but we're using the same system to prompt it. Right? And I have I've noticed this myself in situations, but I've I've heard this also particularly from a lot of others. It is actually a very frustrating experience if you're in a situation that really matters and other people in the room are basically just going to the agents to answer a question and no longer have a really under... Good understanding of either what the system Is doing or that the fundamental problem that they're working on, they actually comprehend. So there is a lot of value in still understanding what you're doing. That also means that you kinda have to moderate the speed in which you're doing things to still work within your understanding. And one of the ways in which I started doing this as my commit started going through the roof, basically, was I I looked into how are my trajectories of sessions going and can I hook myself into a Part before the end of it to find a consistent way for me to evaluate what's going on. And so for instance, I started, basically prompting the agent that before it's going towards the verification, which is tip... Typically around 70% of it, to start producing a really nice short executive summaries and a nice typeset HTML page. They can really understand what it's doing as it's finishing. Particularly with newer models, it's no longer quite as horrible to read the text that comes out of it. But this is One way in which I found to balance out that I'm basically no longer reading the code in a lot of cases. And this I think is something that increasingly is more important to me is to to to also show up with a little bit of a guidance towards, not writing all the software over and over. I'm a huge believer in using fewer dependencies and writing more code yourself. So I really like the agent revolution for for that. Element of it. But in German, we have this what called Wildwugs, which basically means uncontrolled growth. And uncontrolled growth in software that is supposed to work together can have a lot of really negative consequences. You might have seen a blog post that was going around recently about a TAR archive that is... Has been crafted in a way that different TAR implementations read it in different ways. And, I think this is a bit of a problem now. We... For us to work together, we we really wanna have standards. We wanna have, common specifications. We wanna have common file formats. And we now can basically make an agent implement any format we want. But there are going to be these differences. And what really makes it hard for us to agree on standards and what what makes us work together on something often is not a technical challenge as much as it is a problem of how do people agree on what is supposed to be in this file format and how is it supposed to be implemented and how is it supposed Be interpreted because there can be multiple interpretations of it. And I really think that these open protocols and file formats are really important aspect. You can imagine there is a future where we all have our own little Photoshop clones, but we'll no longer be able to send file formats to each other because we have diverged too much. And this really comes down to this part that collaboration between humans is as hard as it has ever been and this is slowing us in a way down. Right? And I think it's it's important to Sometimes resist this temptation of just having the agent solve the problem and just taking a step back and figuring out if this is maybe one of those problems where we should be working together with other people. It is, for me, coming from an open source background, fascinating to see how little actual human collaboration is happening in open source projects now and how many more projects end up with gazillions of forks because it's just easier to have a fork happen than to try to upstream changes. So I I want Make a very strong argument for, building software that strives towards excellence. And I understand that this is hard. We have, I think, seen a pretty big explosion recently of verified software. It's also very clear that this is in itself hard. The the the technology stack is not right there yet where this works. But I think it is one of those areas where it's really worth pushing towards and seeing if this is maybe something that can help us over time creating more more stable software. I really Very happy with seeing so many people wanting to build fundamentally faster and more memory efficient software. I think it's a great development. For years, I was very frustrated with, how little my RAM usage actually goes to things that I think are worthwhile. And now there's a significant trend of people trying to rebuild things in a native language rather than, for instance, just building an electron wrapper. So all of those things are really cool developments, but they are in many ways still building up on technology we have Before times. I really like Rust. I like that there is now a trend towards building memory safe software. I think you've probably all been exposed to what happens with, yeah, good models finding security issues. So there's a strong argument towards that. And it it now has also led to people that would never have picked that language before to actually give it a try because it delivers some value that now can in quote come for free. On the last part I wanna talk a little bit about maybe there is something where Harness can do something. And this is less so much for me to say, hey, here's a thing that... A path that we are on. It's to give you some idea of maybe where some of the challenges are as we're pushing this task to be much more long running. The first one is if if if you have a task that only takes two minutes to run, you're kind of willing to babysit it and if it breaks and, it it ends early or it doesn't run to completion or you have to restart the agent to just restart it manually and tell it Continue. Right? So it's like we have we have largely been working in environment for quite a while where that was acceptable. If you now have automated agents running on thousands of issues overnight, you want to have an agent actually be able to restart itself. We're pretty big believers on this concept of durable agents. Also durable agents that create durable workflows when it orchestrates themselves in a way where you can start it, stop it, you can suspend it, you can bring it back. This is not just because we have this longer running task. But also because there's a lot of inefficiency in what we're doing. And, we are all compute constraint. We are all, constrained in terms of the cost of inference. And being able to be a little bit more intelligent about how we're doing this, I think is really interesting and I wish more people would be pushing towards that. For instance, if you pay raw token costs, you get a pretty significant discount on flex scheduling. What that means that maybe for minutes no token is coming. Right? And so then you wanna suspend your agent, you wanna suspend also your sandbox. Actually, not waste the other resources that are also expensive. So this is one part where I do think beyond the quality of the models, because the models are getting so good now, we might wanna rethink a little bit of how we are using these, resources intelligently. And the second thing is, you probably know that when when when Py came out there was a pretty big, motivation to say like bash is everything that you have to run. And this is Largely what we believe. But now we also see that there is, an emergence of desire to orchestrate sub agents, a desire to use models which are not LLMs like decision models or or classifier models like Jeff. And in those cases, you need to start composing workflows. And, this, I think, originally came from the folks from Cloudflare, say, like, in in... Instead of having thousands of tools being put into The code, into the context. What if we just give the agent in addition to JavaScript, in addition to bash JavaScript to run and have all the tool calls that the agent would natively do happen via JavaScript? Then he can orchestrate them better. And this has gone from a curiosity, I think, in many ways to something that is fundamentally shockingly useful. And this means that now the harness in many cases, and this is definitely the case already in codex, you can do it in Py and many other harnesses at this point. You can have the harness Orchestrate very large invocations of tool calls, without having to go through the inefficient context. And this does require a little bit of a rethinking of how you're doing things. And it also allows you to to use classifier models. And I think this is the other big part that really changed how I was looking at how far I can push certain technology because you're not being penalized for not having an active kv cache, but they're also very cheap and very fast. And they allow you now to Make certain problems that previously were just too expensive or too slow to do to have a decision model do this. And the way I think you should be thinking on this is that when you have code mode, what the harness can do is it can create a little workflow that can have decision points along the way, where if you, for instance, have to crunch through 3,000 incoming customer support requests and you have to make decisions along the way if this is being sent towards having an agent work on it, towards requiring more feedback from a customer, you can have The agent create basically an on demand workflow and then for every single if condition that it's doing, it's just going through Jeff. And then you can crunch through thousands of issues for very little time in no time in your coding agent where previously that would have taken you hours. So with this, I just wanna leave this idea. Now that we can build everything, maybe we should not, but we should really, I think, push towards much more enjoyable and Many ways, much more ambitious software than we have been doing before. Thank you. What if the future of software engineering isn't about writing code, but building the systems that write it? To explore the rise Of AI powered software factories and the evolving role of engineers, please welcome Zach Lloyd, founder and CEO of Warp. Alright. Hi, everyone. Excited to be here. And today, I'm gonna chat about the evolution of software factoring software engineering and how it's becoming factory engineering. So excited to do this. First, a little bit about me. I've been an engineer for a long time. I was a principal engineer at Google. I used to lead engineering on the Google Docs suite, helped scale that product to hundreds of millions of users. I'm currently the founder and CEO of Warp. Warp builds a platform, for comp To build their own internal software factories. I'm gonna show a little bit of that today. WARP is also well known for our open source agentic terminal, has about 65,000 GitHub stars. It's used by almost a million developers. Our mission has always been helping developers ship better software more quickly, and it's it's it's cool to evolve what we're doing as the world changes around us. This is where I think we're still mostly at. Most of That I talk to, most platform teams, they... They're working with individuals who are running their own coding agents locally. Maybe they're running, multiple of them at a time. It could be through their terminal. It could be through an agentic development environment. Just show of hands quickly, actually. How many people in here are mostly working in this fashion, local, multi threading? Okay. So still still most people, but actually not everyone. ## Slides ### 00:00:05 # I step in less with every model ## Stable Harness ### Corrections per 100 Prompts - GPT-5.3 Codex: 8.5 - GPT ### 00:00:37 # How far one prompt carries, by model I. STABLE HARNESS | Model | Minutes of Agent Work per Prompt | Tool Calls per Prompt | | :-------------- | ### 00:01:09 ### I Stable Harness # How far one prompt carries, by model **Minutes of Agent Work per Prompt:** - **Opus 4.6**: 2.4 min - ### 00:01:37 # Tool use by model, same harness ## I. STABLE HARNESS | Model | read tool | edit tool | write tool | codemode\* | bash: sed, cat ### 00:02:08 ### II · WHAT THE FACTORY SCALES # The human did not 1. *Human attention is limited, and we do not have more humans* 2. *Starting got fast, finishing and maintaining did not* 3. *We tax systems that were never taxed: `GitHub`, `npm`, random third-party websites, ...* ### 00:02:41 ## II · WHAT THE FACTORY SCALES # The human did not 1. Human attention is limited, and we do not have more humans 2. Starting got fast ### 00:03:09 ## II · WHAT THE FACTORY SCALES # The human did not 1. **Human attention is limited**, and we do not have **more humans** 2. ** ### 00:03:39 # Agent-hours per day and my prompts per day ## II · WHAT THE FACTORY SCALES ### MY PROMPTS PER DAY ### AGENT-HOURS PER DAY (PAR ### 00:04:08 ## Agent-hours per day and my prompts per day ### II WHAT THE FACTORY SCALES **My Prompts Per Day:** - March: 126 - ### 00:04:37 ## II WHAT THE FACTORY SCALES # Projects started and projects I keep working on [Combined line and bar chart. The line graph, labeled "PROJECTS STARTED, CUMULATIVE ### 00:05:08 II · WHAT THE FACTORY SCALES # Projects started and projects I keep working on [Combined bar and line chart showing "Projects Started, Cumulative" (line) and "Projects I worked on for 5+ days that month" (bars) from March to October. The cumulative projects started increase from 24 in March to 141 in October. Projects worked on for 5+ days per month fluctuate between 6 and 13.] ### 00:05:38 ### III - THE ATELIER ## What the workshop needs 1. Being able to say “not like this” 2. True skill vs. agent skill 3. Moderating speed ### 00:06:08 ## III - THE ATELIER # What the workshop needs 1. Being able to say “not like this” 2. True skill vs. agent skill 3. Moderating speed by one’s understanding ### 00:06:39 ### III - THE ATELIER ## What the workshop needs 1. Being able to say “not like this” 2. True skill vs. agent skill 3. Moderating speed by one’s understanding ### 00:07:05 ### III - THE ATELIER ## What the workshop needs 1. Being able to say "not like this" 2. True skill vs. agent skill 3. Moderating speed by one's understanding ### 00:07:35 ## III - THE ATELIER # What the workshop needs 1. Being able to say "not like this" 2. True skill vs. agent skill 3. Moderating speed by one's understanding ### 00:08:07 ### III - THE ATELIER # What the workshop needs 1. Being able to say "not like this" 2. True skill vs. agent skill 3. Moderating speed by one's understanding ### 00:08:36 # III - THE ATELIER ## Resist the temptation 1. Sometimes building your own is bad for the ecosystem 2. “Wildwuchs” harms interoperability 3. Open protocols, specifications and file formats are useful 4. But collaboration is human slow! [QR code for live translation] ### 00:09:09 ## III - THE ATELIER ### Resist the temptation 1. Sometimes building your own is *bad for the ecosystem* 2. “Wildwuchs” harms *interoperability* 3. Open protocols, specifications and file formats are *useful* 4. But collaboration is *human slow!* ### 00:09:37 ### III - THE ATELIER # Resist the temptation - Sometimes building your own is bad for the ecosystem - “Wildwuchs” harms interoperability - Open protocols, specifications and file ### 00:10:10 # The Software Atelier ## III - The Atelier ### Resist the temptation - Sometimes building your own is bad for the ecosystem - “Wildwuchs” harms interoperability - Open protocols ### 00:10:42 ### Resist the temptation III - THE ATELIER - Sometimes building your own is bad for the ecosystem - “Wildwuchs” harms interoperability - Open protocols, specifications and ### 00:11:04 ### III - THE ATELIER # Spend the power on excellence 1. More software that is **verified** 2. More software that is **faster** 3. More software in **memory-safe languages** [Small abstract logo at the top center of the slide] [QR code] ### 00:11:37 # The Software Atelier ## III - THE ATELIER ### Spend the power on excellence 1. More software that is **verified** 2. More software that is **faster** 3. More software in **memory-safe languages** Armin Ronacher / CTO [Abstract logo at the top of the slide] [QR code linking to https://earend ### 00:12:08 # THE SOFTWARE ATELIER ### III - THE ATELIER ## Spend the power on excellence 1. More software that is **verified** 2. More software that is **faster** 3. More software in **memory-safe languages** https://earendil.inc/ [QR code linking to earendil.inc] ### 00:12:38 ### WHERE THE HARNESS CAN EVOLVE ## Durability: suspend and come back - Runs now outlive the **process** and the **attention** of the human - ### 00:13:07 ### IV · WHERE THE HARNESS CAN EVOLVE ## Durability: suspend and come back 1. Runs now outlive the **process** and the **attention** of the human 2. Humans answer in **hours**, so waiting has to be **durable** 3. Intelligently schedule inference and compute ### 00:13:41 # DURABILITY: suspend and come back ## IV · WHERE THE HARNESS CAN EVOLVE 1. Runs now outlive the *process* and the *attention* of the human ### 00:14:05 ### IV · WHERE THE HARNESS CAN EVOLVE ## Durability: suspend and come back 1. Runs now outlive the **process** and the **attention** of the human 2 ### 00:14:37 ## IV · WHERE THE HARNESS CAN EVOLVE ### Codemode: expressiveness in the tool call 1. Bash composes programs, not the harness's own tools ### 00:15:02 # Codemode: expressiveness in the tool call ## IV · WHERE THE HARNESS CAN EVOLVE - Bash composes programs, not the harness's own tools - Loops and concurrency *without going through the context* - Runs on the harness side, state lives in the *transcript* ### 00:15:36 ### IV WHERE THE HARNESS CAN EVOLVE ## Codemode: expressiveness in the tool call 1. Bash composes programs, not the harness's own tools 2. Loops and concurrency without going through the context 3. Runs on the harness side, state lives in the transcript ### 00:16:09 ### WHERE THE HARNESS CAN EVOLVE ## Classifier models (*Jev*, *Clef*, ...) 1. Crunch through large numbers of *squishy human tasks* 2. ### 00:16:36 ### IV · WHERE THE HARNESS CAN EVOLVE # Classifier models (Jev, Clef, ...) 1. Crunch through large numbers of squishy human tasks 2. Ask questions, get choices and scores back, not prose 3. Called from `codemode` and loops, not through the context [https://the.software.atelier/](https://the.software.atelier/) ### 00:17:07 # AI Engineer Code Summit Presented by Google DeepMind ## The Software Atelier Armin Ronacher / CTO EARENDIL [https://ai.engineer/summit](https://ai.engineer/summit) [QR code linking to ai.engineer/summit] ### 00:17:40 # AI Engineer Code Summit Presented by Google DeepMind [Background wall covered with a repeating pattern of "Code Summit" logos and various sponsor company logos.] ### 00:18:05 # AI Engineer ## Code Summit [Grid of company logos and "Code Summit" text, indicating sponsors and partners, including AWS, Google DeepMind, Databricks, OpenAI, ### 00:18:37 # AI Engineer Code Summit Presented by Google DeepMind [The event title "Code Summit" is enclosed in a white outline box] [Large 3D letters "AIE" are on the stage floor] ### 00:19:10 # AI Engineer Code Summit Partners - FACTORY - Google DeepMind - G2i - exa - Cisco - Devin - Firecrawl - Parasail - Dat ### 00:19:37 # Zach Lloyd - Principal engineer at Google, lead on GDocs - 20 years of engineering - Founder of Warp ## Software Engineering Is Becoming Factory Engineering Zach Lloyd / Founder & CEO Warp [Portrait photo of Zach Lloyd] [QR code for live translation]