# Rise of the Software Factory — Tereza Tížková — session 2026-06-30T18:10:00.000Z → 2026-06-30T18:30:00.000Z _56 transcript lines · 12 slides · source: full recording_ ## Transcript Now let's see this in action. We've been talking about this stock selling agent, which went haywire in production. This is a representation of the same. You have your initial planning step, which takes into account the user input. It can use the place order tool to do the actual selling and buying of the stocks. And then finally, it delegates to the fine. Hi everyone. Hey what's up? I'm Teresa, and today I will be talking about software Factory. Everyone is talking about software factory, but only few people are actually building one, and even fewer people know what it actually takes to build one and how to define it. So I will be talking about what is a software factory, should I build your own or outsource it, what works in production already, what are the main challenges, and how much is this all going to cost. My name is Teresa. I work at a company called factory.com, and we've been Building, this concept of the software factory for quite a long time. But finally, the technology is catching up, and it's possible to build this in production for enterprises like UI or Adobe. I would define the software factory as the whole loop, the whole life cycle of developing software with autonomy, which doesn't mean just coding and generating code. By that I mean Collecting all the signals, reacting to user feedback, to logs, prioritizing what's important, then orchestrating it all, executing, validating, doing really good, testing in production, and then iterating on all this, while also continuously improving in the process and gaining new knowledge and new skills. So this is just our dashboard, how our software factory looks if you if you look at it and it's catching the whole cycle there. This is just some history and by history I mean 2023 in AI. So the Factory wasn't really possible before even though we always had this idea from the beginning of, Cheggpt Lounge and others. We had this idea of the continuous loop. There was auto GPT, baby AGI, already with the concepts of iterating on the software, but it didn't work before because the elements were hallucinating. There was problem with context length, problem with the reasoning quality, different problems like this, or we were missing good environments where the agents can actually work. The isolation. So all this kind of led to why the software factory is the paradigm that's starting to be popular now and actually useful. I like to define things by what they are not. So I want to say Software Factory is not just coding agent, and it's not even a swarm of coding agents, even thousands of agents, because generating code, writing code, that's the easy part compared to all the others. Engineers don't even spend most of The time, just the writing code. The challenges are the rest of it. So this is what I want to enhance. And it's also not just some consultancy or just some abstract strategy that someone will try to sell you and to your organization. I think the approaches are, of course, different, but we believe you should really rebuild your organization from ground up to be ready to become a software factory. Like, you can't invite a consultancy and just throw something. In the middle of your organization, you should really be mindful and rebuild it from scratch. This is the approach I like, showing to building the software factory, and I think it's similar to building the real team of humans because there is a lot of agents. As I said, they will be testing, validating, iterating, and it all can end up as a big chaos. So you need to be mindful and I would say the three important things is being agnostic. In the software factory, so being independent of LLM choices, of how you already work as the organization, then you need to be autonomous. So you need to give the agents the right trust and all this permissions and governance. You need to trust it to run really for a long time because the predictions are that agents will run even a year or more years without the human in the loop. Of course, that sounds ambitious, but, for example, our missions already our long running sessions already build things in around forty weeks. So it's not that crazy. And third thing, always improving. So same as human organization, you need to onboard any new people in your team by giving them good, code base understanding, good structure, documentation, and also let them improve in the process and gain your knowledge and share it share it within the team. So these are the three points, and now let's talk about the agnostic one. I think the important thing to point out, if you are a builder and Building different primitives of the software factory, you need to be ready for how everyone is already working. You need to work across environments like Slack, GitHub, and, connect it to everything. And you need to also allow people to already bring subscriptions they are using. Because there is every every time there is, like, new technology, new frontier models, new benchmarks, and it's so difficult to catch up, but also to predict what we'll be doing. So it's best to be agnostic for everyone. Then there's the topic of being agnostic to models. This is a topic of a lot of discussions recently, and this chart is from this chart is from CEO of Coinbase, who tweeted about how they started, started saving money in their organization on AI but without reducing the token spend. So the black line is how much they spend on tokens. They Continue growing and token maxing, but they stopped spending so much money. How they did it, different tricks like different default default models for people, so stop pushing people to use only the frontier as a default. Then caching, of course, to stop preferring the stuff every time, and then no limits on spending, but just needing to see the results if you spend a lot. And the last thing that's really important topic is routing. So smart routing between LLMs and just token optimizing between, the LLMs instead of Spending so much tokens. We built a thing in factory called, automatic model routing. And there has been a lot of questions around it, but what it does is it automatically routes between different models in your process. The important thing is it decides first what model is most optimal, and then, it can switch to a different model if it It's failing the task or in case of any troubles, but it usually doesn't happen. And the important thing is it doesn't just help, save money, it also helps with reliability or with speed, because open source models are often faster. And, if one alarm provider fails, you can just switch to another one automatically. So it really does more things than just saving the money. This, is our benchmark, which is very conservative. I prefer conservative benchmarks, but I think the direction is clear. You can say, for example 25%, but even more probably. How the routing works, there are four parts. First, you just assign the task, and you don't really need to do anything. But as an organization, you can, for example, give different permissions and different default models to different people, which is very useful. Like, marketing or sales or engineers can, well, have different default models. And then you have the classification, which is the important part. This is the magic of the routing. You need to really, look at Structure of the prompt, of the, code base, of how difficult the task is, what tools are being used, all these factors, and make classification of difficulty of the task. After that, you basically do a threshold of what is enough to accomplish the task, and you choose the cheapest model above the threshold. So cheapest model that is predicted to accomplish your task, and then you go. Okay. So when we launched this, we got a lot of Questions like, does it actually work? What if, the model misroutes? Is it slower? Is it more expensive, actually? What if the model can't accomplish the task? Or what if it needs to upgrade? And how do we handle caching? So I I think these are all valid questions, and that's why it's so difficult to build a good router. Like, you need to basically classify it very well, and the challenge is to not not to need to switch too often. But even if you're switching in the middle of the task to Two more difficult model, you still, overall are faster probably because it's still this is just some overview of caching. A lot of questions we got on this was also how do we handle caching? Do we do discounts for users? Because l l m LLM Labs, of course, save a lot of money by caching and skipping all the context prefill all the time. Open models can do this as well. I think people sometimes Forget this, and you can just host open models as well, on dedicated compute, and you can take the same advantage of the caching. So I want to enhance, the final price for users is just a pricing decision. It's not a technical challenge because everyone can do caching. It's just what price you pass on the users and what deals you make with the API providers. So this is, being agnostic. And now the autonomy, which is the core of the Software factory. Everyone is talking about loops. Loops, I think, have been here the whole time already in different context. They are just now leveling up and moving to the context of agents. You probably heard about the Ralph loop and specifying the tasks and splitting to subtasks for agents. This is a known concept, and, the question is not the loop itself, but the question is how how you define what it means to be done in This is one example from my colleague. He he made a loop, of agents building three d printing of our logo. I think here you can see what is the challenge of the loops because, before in programming, loops had clear criteria of what it means to be done, while now the criteria become open ended because, a lot of the tasks are very non deterministic. It's basically open world. You can do even real stuff like printing something, and it's really difficult to define how do Verify that the loop was done, that the task was accomplished. So this is the difficult part. Yeah. They just need to be verifiable. This I call the scary chart. It basically shows that the tasks and how long they run autonomously with agents have been increasing, but still it's not that reliable. Even if you can run for a very long time, it doesn't mean that it will be reliable in production and You still probably need to need to iterate on it and provide feedback. So it's still not solved and there are problems like cheating. For example, if you write the what it means to be done in a wrong way, the agent can try to pass your tests but not really, like, not really verify what you need to do and accomplish, but instead try to solve just passing your tests. So cheating by that. We have something called factor emissions, and this is like long running sessions of the agents. We just call it missions because we send the agent to the mission. And The missions work in the loop and iterating on task until it's done, and they can do it even for weeks. And, the main agent is orchestrator, and then it's, assigns work to workers agents and then to validators who review the task. So this is one example, real mission from our customers that run sixteen hours. And just just to see how important is the validation as well, it takes even 40% of the whole process. To summarize the orchestrate There just decides and writes the conditions what it needs to be done, then it gives it to the agents called workers. The agents work on it, and what is interesting or worth noting is that they work in a sequence. So they don't work in a swarm or parallel. The agents work in sequence and everyone accomplishes something and passes it to the next one. We actually found that if you do this you end up with more fresh context and kind of fresh heads same with when with humans have, like, other colleagues verify. Their code. So similar thing, we just let the agents pass to the next one. But still every worker of the sequence can have parallel agents doing smaller tasks. So I don't know, researching on the web or building files during that. So they are in sequence, and everyone has sub agents as well. And validators, they review the output and provide feedback and send it back to the beginning. Important thing is, the validators judge code that they Didn't write. There's something from the orchestrator agent called a validation contract, that's written before any code is done. And first type is scrutiny validator. Second is user testing validator. The scrutiny is the one who really verifies how the code base looks, the interest types, tests. It's really rigorous check of the code. But the second, I think is really interesting is user testing validator and that's the one who is really in the arena trying the things. So it's Doesn't care how it was made, it just goes. It works in its virtual computer, and it clicks on the stuff and really checks if everything works. I've seen one engineer migrating codebases with droid with our agent, and it it really needed the agent in the end to click through the stuff because with other products, it just created created the product, but it didn't work and was wasn't interactive. It was just a dummy result. So our agent didn't really click On the staff, allowed to check that it's actually working and not just looking good in the code. I think this is one cool thing that was also allowed by the progress in computer use and the persistent environments of virtual machines for agents, which also wasn't here before or wasn't that great. But now it's really great, so it's allowing to go in the arena as an agent. This is just one example of real mission from our users. I I like to just learn what users are And get feedback, and this is just, how it can look in your dashboard. And this is how I visualize the missions. What is interesting is that it's just a loop with smaller loops because every of the agents are running it in loops as well. But overall, it's like one big loop. Okay. Maybe this is a bit weird, but I just like that it's all loops with smaller loops. Okay. And the third part, always improving, always learning. There is this elephant in the room context with agents. A lot of questions which are valid are like, how do you navigate the context bloat because you work on these long missions in the software factory, and how do you actually keep the context clean. And that's very valid because enterprises, especially, they use hundreds of tools. They use Figma, Notion, Gmail, Drive, Slack, like hundreds Tools on average. So all these have specification in the code. All these have schema and parameters and log descriptions. So, it can cause agent to really bloat with that and pick wrong tools if there are two tools that sound similar or to actually lose context because they fill up the the context window and need to compress. So this is really dangerous. For this we have something within the software factory called, Deferred Context Engine, and we build it such that we just progressively disclose what's in the context and what tools to use. So we basically have a surprise tools, that help later only if they are actually needed. So it first just has short list of the tools and, only a short descriptions, and when it's needed actually in the code, they can call the tool and fully load it. So important thing is nothing is actually removed. It's just hidden and not reachable until needed. So it's just for later. And important thing is it actually saves a lot of tokens, at scale. Once you use more and more tools, the more actually you save, it really scales and you can save 50% of tokens or more. Okay. Another thing which is tricky and was surprising for me for the first time is that when adopting AI, you either succeed big or you can fail big the same way. It's a bit of power law. So if your codebase is not ready, if you don't have structured codebase and all important things, then adopting or turning into software factory can Actually, make you end up worse and make your code degrading. And, there is a big and growing gap in productivity between those who just adopted AI versus those who actually thought about it a bit more. There is another data from Stanford about yeah. Without structured, codes base and being ready and documenting well, the AI can make your code worse. And I think if you are engineers, you probably can agree that sometimes AI really makes a mess. And this can really compound more and more, and it's difficult to go back. We have something as a part of the software factory. We have something called agent readiness, and it's a framework. I don't like the word framework, but, take it as a hygiene check of your code base and things you should do because there is actually a nice correlation between how your code base is looking and all the details there and how good you're gonna add up the AI and end up productive instead of With a messy code base. So there are things like how how reproducible is your developer environment, or if you have written good tests, if you have all things well documented, what's the style of your code, or the test and linters, everything that you would do also to keep code base clean. So these factors actually show to be very useful. And we have our bigger customers actually go through this agent readiness framework and do the out checks, and then they can follow-up with. Recommended actions, how to fix that. I think one more thing about the context and the continuous learning in the software factory is the most of the time when working with AI, or at least from my experience, is that you keep repeating to agents how how do you want the things done, and you feel like the agent doesn't really get you. And I think this is, again, nice parallel with, human organizations and teams because when you're joining your company, there are a lot of Rules that are not really codified anywhere. Like, you just learn and observe how things are done, and there are a lot of things between the lines that you can't really learn anywhere else than just observing. I think this is a big challenge for agents as well. And one thing we launched for that is plugins, which are, like, packaged reusable skills and context and the things behind the scenes that you can really codify. Or auto wiki, which is automatically updating our documentations and ## Slides ### 00:00:22 `@boundary wrapper (LIVE)` `@boundary("place_order", kind="tool")` ← annotate once **INPUT captured** `symbol=ACME quantity=1000 side=sell (args → InputState)` ```python def place_order(symbol: str, quantity: int) -> dict: notional = quantity * SHARE_PRICE return {"status": "filled", "notional_cents": notional} ``` **OUTPUT captured** `{"status": "filled", "notional_cents": 1900000, ...}` `Envelope → store → fixtures/traces/` [Flowchart diagram illustrating the sequence of input capture, function execution, output capture, and storage within a boundary wrapper] ### 00:00:52 # World's Fair ## AI Engineer ### 00:01:23 # NO ONE KNOWS WHAT software factory means 1. What is a software factory? 2. Should I build my own? 3. How? 4. What works ### 00:01:50 ## The software factory is the **whole cycle** of developing software that runs autonomously signal -> triage -> plan -> execute -> validate iterate factory.com Software Factory dashboard ## Engineering the future of AI [Screenshot of a software factory dashboard with multiple panels showing metrics and line graphs] ### 00:02:23 # The software factory is the **whole cycle** of developing software that runs autonomously `signal -> triage -> plan -> execute -> validate` (Iterate loop) `factory.com Software Factory dashboard` ## Engineering the future of AI [Screenshot of a software factory dashboard displaying multiple metrics and line graphs] ### 00:02:55 # The software factory wasn't possible two years ago ## Timeline of AI Code Generation ### autocomplete - **2021** - Copilot 2021 - **2022** - ChatGPT ### code generation - **2023** - GPT-4, Cursor - first **Factory** - **2024** - first Devin - better reasoning - bigger context windows ### **software factory** - **2025** - persistent environments - long-running Factory Missions - Claude Code - **2026** - token cost wars # Engineering the future of AI [Timeline showing key developments in AI code generation from 2021 to 2026, categorized into autocomplete, code generation, and software factory stages] ### 00:03:22 # Adopting AI is not the same as being built on AI - **80%** of engineering time is not writing code - **12x** Gap between AI leaders and laggards — and still widening. You can't just SOMEHOW transform. - **30x -> 10x** Accenture's cash-flow multiple in recent years - **2x** Revenue for firms that **rewire around AI**, not just adopt it (a16z, 515 startups) Sources: METR time-horizons, agent tool-call domain mix, a16z "AI enabled or AI 'enabled'?" (515-startup study; ACN P/FCF, MacroTrends), Denisov-Blanch, Agarwal, Azaletskiy et al., Stanford (ASE 2026); GitClear; DX. ## AI Implementor Loses Its Sheen Accenture's SACN Fwd P/E and TEV/FCF # Engineering the future of AI [Line graph showing Accenture's Forward P/E and TEV/Unlevered FCF from 2017 to 2026] ### 00:03:56 # Adopting AI is not the same as being built on AI - **80%** of engineering time is not writing code - **12x** Gap between AI leaders ### 00:04:27 # Build software factory like you'd build a team ## 1 Agnostic - Works where you work: Slack, CLI, desktop, phone - Bring your own key, model, and ### 00:04:52 # Build software factory like you'd build a team ## 1 Agnostic - Works where you work: Slack, CLI, desktop, phone - Bring your own key, model, and machine - Automatic model routing ## 2 Autonomous - Earns trust, organizational and technical - Runs in the loop, over long horizons - Per-person, per-task permissions ## 3 Always improving - Prep your codebase for agents - Share agent skills across the org - Compounds on your own data Source: Factory # Engineering the future of AI [Screenshot of a dark-themed application showing a list of completed tasks or jobs.] [Screenshot of a security settings panel with options for Droid Shield and restricting member visibility.] [Screenshot of an analytics dashboard displaying a graph, a radar chart, and several metric cards including Code Ownership and Cognitive Complexity.] ### 00:05:24 # Build software factory like you'd build a team ## 1 Agnostic - Works where you work: Slack, CLI, desktop, phone - Bring your own key, model, and machine - Automatic model routing ## 2 Autonomous - Earns trust, organizational and technical - Runs in the loop, over long horizons - Per-person, per-task permissions ## 3 Always improving - Prep your codebase for agents - Share agent skills across the org - Compounds on your own data Source: Factory ### Engineering the future of AI [Screenshot of a terminal interface showing "Factory Droid" tasks] [Screenshot of a security settings panel with Droid Shield and member visibility options] [Screenshot of a dashboard with a radar chart, line graph, and metric cards] ### 00:05:53 # We need to build software factory primitives that work for everyone ## Work across interfaces - Cloud, local, CI, or IDE – one agent - Switch environments, no rewrite - **Portability** beats lock-in ## Works with your subscriptions - Bring your own model keys - Bring your own machine Source: Factory, Anthropic ## Every domain works differently ### In what domains are agents deployed? [Horizontal bar chart showing the percentage of tool calls by domain, with Software engineering at 49.7%, Back-office automation at 9.1%, and other domains following with smaller percentages.] # Engineering the future of AI