# Ship more as a team with Kiro Crew — Doug Clauson — session 2026-11-12T20:10:00.000Z → 2026-11-12T20:30:00.000Z _83 transcript lines · 40 slides_ ## Transcript Well, good afternoon, folks. Wow, that's loud. You guys hear me okay? I take it yes. I'm gonna start with a question here today. How many people feel like they're getting everything that they could possibly get from their agents today, and that the promise of A. I. For your team has been delivered and you're satisfied with where you are? Show of hands. This is about what I usually see. Like, there's very few people, and for us at Amazon, it's actually the very same. Like, when we first started this process, what happened is is we were seeing between 1020% gains. As I show on the chart over here, you know, 1.1 x to 1.2 x. And what we noticed was what was happening is as teams were bolting AI onto a process that they already had. So they're taking their normal process they were doing, they're trying to attach some AI here, attach some AI there, and expecting that they're going to get this massive result out of it. In addition, what we saw was teams were only shrinking the cogeneration phase, and as you all know, you guys are all experts in this, the cogeneration phase has gotten better and better and better as time has gone on. But there's a lot more to software development than just code generation. We did a study in June. I... I've published a blog post on... Or we published a blog post on it as well. And in the study, we looked at teams who changed how they worked in our Amazon stores division, and we we looked at what kind of returns were they getting on their AI investment. Many of these teams were going slower before they were able to go faster, but the median team was seeing a 4.5 x return in in terms of productivity improvement. And specifically, what this was was normal Highest development velocity against that own team's historical baseline. And we had some teams in the study who exceeded 10 x. Right? So we're talking significantly more productive. And what they did was just they changed the way they were working. But I see this still so often, and I... Just by your show of hands, the same thing, right, where people don't feel like that promise of AI has quite been delivered yet. Now my name is Doug Closs, and I'm a product manager on the Curo team. And I've been on this AI dev tool journey really since the beginning. And I still see, like I said, teams who aren't quite getting what they expect. Now as a PM on Kiro, I get asked literally the same two questions all the time. Like, this is a story of my life and meetings. What are you shipping next? And they want to know, like, what are the new features? What new use cases might be enabled by those new features? And, you know, I can walk through that, and that's great. The other question, and oftentimes the more interesting one because everybody builds software slightly differently, is are we doing this right? Like, people feel like they've hit some sort of local max. Moment they're scared that somebody else has figured out how to you use the AI better than they have, and they're just gonna get surpassed. We're actually gonna spend today on the second question here, not the first question. And I'm gonna walk you through how one of our teams, our our Kuro Crew team, is shipping now thousands of PRs per week. The big lessons we learned on the way and sort of the process we went through to get to where we are today. And then at the end of this, I'm gonna leave you with a workshop that if you wanna try to build your own software factory in Kuro Crew, you can go and do that as well. Excuse me. Now first off, just some bit of context on Kiro. So Kiro is an agentic development system that lets you plan, build, and ship software with AI agents. It works wherever you work. So up here, I'm showing you we have an ID client, a CLI, a web, and a mobile client, and then I just talked just briefly. We have Curo Crew as well. We support a variety of models. So we have both the anthropic models and the OpenAI models in in Curo as well as some Openweight models. I'll talk a little bit later how we use that and how it's been beneficial to And just like the rest of AWS, we really push the barrier on cost performance. So when we start saving money and we make the models cheaper to run and the agents cheaper to run, we end up passing those savings on to you. So we're really happy with where Ciro has gotten on that Pareto front of sort of cost performance and cost effectiveness. Excuse me. Now, like I said, we're gonna spend this today talking about Kiro Crew, and Kiro Crew was built to run many Kiro agents at the same time. It's all open source. It actually started internally as an internal Amazon product. Where, some engineers grabbed our CLI, and they're like, hey. We wanna extend this. It was during the the whole OpenClaw craze. If anybody is... Has their Mac minis at home, they've been running those. It happened during that craze where people wanted to figure out how they could get these agents to be more like personal assistants. And so this mesh claw product internally was developed, got around 40,000 new, internal Amazon developers on it in the first two months. And so us at the Cura team, we're like, hey. We should... Since it was built on Cura, we should take that and extend it and release it. And so we've done that now. Kuro Crew is released. It's open source. And if I talk about what what have we seen with the Kuro Crew team and and our team in terms of building and developing this, we we shipped on August 4 externally, And again, it's open source, so you can go look at all the source and look at all the PRs that came through. We go by one philosophy, that if we don't use it and love it, then we don't ship it. Since we launched, we've merged about 16,500 PRs with a very small team. Just last week, there was over 3,000 Single week alone. Take contributions from external, entities, both internal to Amazon and an external Amazon, people who just wanna contribute and take a good idea with open source. And if you look at the chart here, we sort of leveled out at around... You know, if you look at just our shipping speed, we sort of leveled out at around, high hundreds, 600 to 800, sort of the the first few months we... After we shipped. And then you'll see in, like, 09/28. Right? All of a sudden, we break a thousand again, and then we're sort of hovering around. I'll average it out in my head. And say 1.2 k ish. Right? And then you can see we made another step function improvement. And while I'm showing you PRs on this chart, what we really care about among the team is not necessarily the PRs, but we care about how many sessions can an individual developer operate at the same time. And I wanna be clear, operate at the same time. And I wanna be clear that, like, this is not a one size fits all thing. So as I walk through this, just keep that in mind that that what works for us may not work for you. And so you should challenge, you know, what are you doing that you could take or you could learn from this. Last thing on this one, about one in 10 of these is features. So I always get asked, like, man, 3,300 features? That's a lot of features in a week. Well, no. It's not all features. Right? This is what shipping with speed looks like. A lot of... So I get asked, okay. If... So about one in 10 of these is a feature, so about 10%. And so if 10% are a feature, what are the other 90%? Well, they're... For us, it's tests. Test is a big part of what we do. It's, I'll say paper cut fixes for things we've already shipped that we're just sort of learning. How it... It's getting used, and we're massaging it a little bit. It's bug fixes as well, and that's really what shipping at a high speed with velocity looks like. Right? It... It's not all new features, but if you just think about 10% of our features, there's still hundreds of features... New features being shipped over this time line. If I think about, sort of how do we evolve with our development cycle, we started... If if I go back to when our CLI product initially launched, it was one session, one person. So I'm sure many of us experienced this when these new agents came. And I was typing, and it wouldn't work, and then it would stop working, and then I would type again. Right? We'd just go back and forth. And then from that... So that was one session. Right? So we're trying to optimize sessions per person. Then we move to parallel sessions, where instead of just running one session, I'm looking at, you know, maybe I'm running two or three, but I'm running in the same manner. Right? And the problem with both of these is that if you make yourself the feedback loop, you are now the bottleneck. Right? These agents got faster long before you're going to be able to keep up. And so these first two sessions, those Kill on one chart here where it's, you know, we got to two or three sessions per person, but we sort of topped out. Now stage three introduced memory and schedules. And with a memory, a new session now all now all of a sudden knows your conventions. Right? So implicit memories. And then schedules make it so you don't have to kick all of the work off on your own. Things can happen on a schedule. The thing we learned here was that schedules only help if your machine is on. So in the previous phases, most of our engineers were running locally. Right? As I'm sure many of you do still today. But Moving to remote ends up making it so you no longer have to worry about is it going to run because I have an always on machine. Something we're very good at at AWS. So obviously, we are running remote. But what happened then is this allowed us to sort of, I'll say, build up between ten and twenty sessions. Right? And then we sort of plateaued again. We were unable to sort of break that barrier with just memory and schedules because much of what we were doing, was still being the glue between the sessions. Right? And then if we... Because our next phase was a pipe. But not a CICD pipeline like you might be thinking, but instead more like an assembly line. I actually have, my previous life, I worked in aerospace, so a lot of assembly lines. And the hard thing with assembly lines is nonstandard work. Right? So if everything is exactly the same as it flows through the system, an assembly line works great. As soon as you get nonstandard work, all of a sudden it becomes difficult. And I'm sure all of you know this, but software doesn't follow the exact same path every time. And so what we found our own teams doing then was this... Sorry. This got us to maybe, like, 30. 20, 30, 40 sessions ish. But what happened was is our team was then trying to unblock the assembly line or the pipeline. Right? So as things came up, if we thought something was a small issue and then it turned into a big issue, sometimes the assembly line would be stuck. Somebody would have to come in and unblock that thing, maybe restart it, maybe replan it. And so this this is only gonna carry us so far because, again, we are in the midst of the agent. We're blocking the agent more than the agent wants to work. And so what do we do for that? Well We launched a thing called UltraMode. UltraMode is still... Just to be clear, if you go try it and you look for it, it's still in the developer, preview, so it's behind a feature flag. But because it's open source, the code is all there. If you want to go try it, you can you can try it out. As I said up front, if we don't love it, we don't ship it. So we're still sort of finishing... Putting the finishing touches on this one. But I wanted to talk about it anyway because it takes you from being... What we did was... If you look at this, middle part here, where we are the person driving the effort, these Things that we might have been doing. So watching for problems was one of the biggest ones and then trying to fix the assembly line when the problem came up. Maybe you're judging a result or you're deciding what to do next. What UltraMode does is it does that job for you. Right? So you put an agent on top of your agents that then is responsible for doing that, defining the goal based on whatever you're trying to accomplish, starting things, watching for problems, unsticking things. And this got... This is getting us today to 50 plus sessions. And so if you think about where we started with a Session in a single person. Now if a single developer is doing 50 plus sessions, that's how you get that sort of the volume in the PR chart I showed you earlier. And we can already see the next wall that's coming. The orchestrators working with other orchestrators, and then the governance around what you want that to be and the guardrails become even more important. Right? So again, that's how we got the volume. Those were the five phases we went through. Ultra mode has been a game changer for us as was the assembly line. And and, I invite you Try those ideas if you think they might work for you. The next thing I wanted to talk about though was how do we... What did we learn from this? Like, what are the biggest things that prevented us from shipping quickly? The first one is test everything. And now this is no... This is not a new lesson, right? Everybody in software knows this. We should be doing this. But I still see people too often, discount the value this is providing. And this is gonna build in with the next one behind this around reviews. Like, if you think about where do you get confidence to ship something? I think many people still feel like, hey, I get confidence because I looked at all the code and I know exactly what that code does. Again, if you do that, like, maybe that's a good thing, but where we have gotten our confidence, and Amazon's always been very test driven, is from the tests. Because you might not be able to look at something every time, but you can certainly test something every time in the exact same way and test the result, right, and come out with the result. And so for us, again, quality really becomes a testing problem, and we're spending a significant Of our time around 20% just on testing today. The other big thing we learned, and this is a a bigger one and and also a contentious topic, is is reviews. So quick show of hands, how many people have done a review with an agent thinking that there's gonna be, like, one or two comments, and then you come back and you're like, oh my god. Like, this is terrible code. Right? There's comments everywhere. We had the same problem. So we originally wrote a a review agent that would review everything, and our goal was to fix every single comment. Well, as you guys, I heard the laughter. So as you expect, you you submit changes. So you submit a PR, you get the feedback on that PR. You come back to... You submit it again. After you fixed it, you come back, and now it's three new things that weren't even test... Weren't even told about before, and it's four things on the things you already fixed. You submit it again, and and this cycle just continues to repeat. So what we've done with reviews is now we categorize the reduce or triage them into blocking and non blocking issues. So for example, something that breaks the build. If there's something obviously that breaks the build, that's a blocking thing, and we need to go fix that before we merge this PR. Changing a variable name, not a big deal. We'll take care of that later, so we let those things slide through. The other thing that we've done with reviews is we wanna have fast fast feedback for an agent. And as you can see by the process here that I'll walk you through in a second, this is not that fast. And so we have a fast gate that really double checks things really quickly to make sure that, like, things are in the right folders before we move on to our reviews. We have six reviewers, actually, that run. Two code reviewers. Both in different models with different model families. Intended... And the intent there is to really find... To not make sure you don't have a model gap. Right? So that the model wasn't trained on something that we don't catch. This is where we use the, anthropic and open AI models or an open weight model to be those, levers. We run a design review that asks, does this thing have the right shape? Meaning, is what I'm building... Like, if I'm adding another cache in this PR, can the cache I already have take care of this issue or not? We have a first principles review that says Is this change addressing the real problem? We have a UX review as well that if you're making it... Let's say you're creating a new error message, we have an agent decide whether or not that error message... The human will know what to do with that error message based on what you've... We've published. And then we're consistently tightening our security posture. Now each one of these reviewers gets one vote. It's either yes or no. It's red or it's green. Everything's green. It goes on to merge. If not, we go to a prepare PR skill that's in the repo so you can check the skill out if you want. And that prepared PR skill then tries to resolve the issues as we go. The third lesson and... Sorry, on the reviews one more thing. Like I said, we've stopped reviewing code a long time ago. We have moved on, and what we're responsible for is not the code itself. See, this actually goes back. It does. What we're responsible for is the review record. Responsible for is the review record. Did it follow the process as we expected? And did we get the results we wanted? If not, we fixed the review process. Two other things. So, one, the over engineering problem. So we have a specific role to teach our agents to Tracked, not just add, and so we try to remove code from the process as we go to prevent that over engineering. And we use that same philosophy on our road map as well, where saying yes is becoming very cheap or has become very cheap. Right? It's very easy for us now with code to say, like, yes, I'll do that, and I'll do all of these things, whereas we used to have to be very particular about what we were gonna build. The problem with saying yes all the time is that you might lose your product shape, and we're already seeing this in in KiroKribbit that it's moving so fast and so quickly that you don't want your users to get caught in the noise. And so, again, use the same principle on the road map is what we do. And then last thing on lessons is you're not done once you merge a PR. Right? There's still other phases. And so for us, merging is not the last check that we do. Instead, we run a release gate where the agent will download the repo. It installs it, runs it, and then checks the core flows to make sure that the overall system is still in good shape before we would do a release. And then we do security all the time as well. So we have, every hour, every day, and always on, every hour against our security. Guidance every day against an AppSec review and then an always on pen test agent. So again, merging is not the end of the process. You have to crunch everything down. So I'm gonna come back to where I started. Are you getting everything from your agent? And many of us don't feel like we are yet. And what I'm gonna offer here today or what I've tried to offer is the agent was fast enough to work on this a long time ago, and it's only getting better and better and better. And so what what we have been doing is challenging how we work and figuring out Like, where are we preventing the agent from working as quickly as possible? And I've given you numerous examples today. Again, I'll just call it a couple of them here. So it started with our attention span. Right? When I was the feedback loop, I could no longer ship because I was the feedback loop. When I wasn't there, the agent stopped working. The laptop lid closing. All of a sudden, you lose your connection. Moving to remote can solve that. Right? The fixed pipeline with a fixed shape that can't respond when things go slightly off path. Again, how do you fix that? You... We fixed it with an orchestrator. The endless reviews that I think I've gone into probably more depth than you were interested in, but, the endless review cycle and then the loop that only adds, right, the loop that only adds things to it and doesn't try to subtract. And so I'll leave you with... I also wanna be clear that, like, this is going to be a continual, adjustment. Right? This is not... Like, this may be what works for us today. I promise you another month, we're gonna find something else that works, and we're gonna continue to evolve significantly. But like I said in the beginning, the pace that we're finally able to Deliver at. I feel like for us is that finally we're we're meeting that promise of AI, and and I still see this disparity among other teams. And so, like I said, I'm gonna leave you with this. If you're interested, in trying hero crew in building your own software factory, scan this QR code. It's a workshop. We have a pretty generous free trial on hero, so you can go sign up for a free trial, go install hero crew, and you'll be off and running. And if you don't, if you wanna try, like I said, if you wanna try kiro crew, you can do that. The same free trial. Works across all of the products. So the I D E, the CLI, the web client, the mobile client, try them all with that free trial. And with that, I'm excited to see what you all build. I appreciate the time. Thank you. Enjoy the rest of the conference. ## Slides ### 00:00:03 # LIVE TRANSLATION // AVAILABLE **Event**: AI Engineer Code Summit **Presented by**: Google DeepMind Scan the QR code to access real-time translation. [https://events ### 00:00:37 [Abstract background pattern of dark grey dots on black, appearing wavy] ### 00:01:00 # everything they can # agent(s) ### 00:01:34 # Very different results - **1.1 to 1.2x**: Where we started - 10 to 20% faster, AI added to the old way of working - **4.5x**: Frontier teams - Median gain after changing how they work - **10x+**: Some frontier teams - Past 10x 10 to 20% is anecdotal. 4.5x is normalized deployment velocity in Amazon Stores pilots. Source **AWS blog, Frontier teams 2026** --- ### Ship more as a team of one with Kiro Crew Doug Clauson / Principal PMT KIRO [https://kiro.crew/](https://kiro.crew/) [Bar chart comparing three levels of results: 1.1 to 1.2x, 4.5x, and 10x+] ### 00:02:02 # Very different results - **1.1 to 1.2x** - Where we started - 10 to 20% faster, AI added to ### 00:02:34 # Very different results - **1.1 to 1.2x** - Where we started - 10 to 20% faster, AI added ### 00:03:00 # AI Engineer Code Summit Presented by Google DeepMind ## 01 What are you shipping next? ## 02 Are we doing this right? Ship more as a team of one with Kiro Crew Doug Clauson / Principal PMT Kiro [QR code] ### 00:03:34 # AI Engineer Code Summit Presented by Google DeepMind ## The Question for Today - **01** What are you shipping next? - **02** TODAY Are we doing ### 00:04:06 # AI Engineer Code Summit Presented by Google DeepMind ## Kiro, built to be extended - Kiro IDE - Kiro CLI - Kiro Web - Kiro Mobile ### 00:04:35 # Kiro, built to be extended - Kiro IDE - Kiro CLI - Kiro Web - Kiro Mobile - Kiro Crew **~40 ### 00:05:06 ## Kiro, built to be extended - Kiro IDE - Kiro CLI - Kiro Web - Kiro Mobile - Kiro Crew ~40k internal users ### 00:05:34 SINCE LAUNCH ON AUGUST 4 # How we ship today If we don't use it and love it, we don't ship it. - **16 ### 00:06:02 # AI Engineer Code Summit Presented by Google DeepMind SINCE LAUNCH ON AUGUST 4 # How we ship today If we don't use it and love it, we don' ### 00:06:38 # How we ship today SINCE LAUNCH ON AUGUST 4 > If we don't use it and love it, we don't ship it. - **16 ### 00:07:02 # How we ship today SINCE LAUNCH ON AUGUST 4 > If we don't use it and love it, we don't ship it. **16, ### 00:07:34 AI Engineer Code Summit Presented by Google DeepMind SINCE LAUNCH ON AUGUST 4 # How we ship today If we don't use it and love it, we don ### 00:08:01 # Five stages, and the wall each one hit HOW WE GOT HERE 1. One session > refactor auth > waiting for you 2. Parallel sessions > like I said, > this repo uses pnpm... 3. Memory and schedules 4. A pipeline 5. Ultra Mode If you're the only feedback loop, you're the bottleneck. Ship more as a team of one with Kiro Crew Doug Clauson / Principal PMT 👻 KIRO [https://kiro.crew](https://kiro.crew) [QR code linking to kiro.crew] ### 00:08:34 ## HOW WE GOT HERE # Five stages, and the wall each one hit 1. One session 2. Parallel sessions 3. **Memory and schedules** 4. ### 00:09:02 # AI Engineer Code Summit ## Presented by Google DeepMind ### HOW WE GOT HERE ## Five stages, and the wall each one hit 1. One session 2. Parallel sessions ### 00:09:34 ### HOW WE GOT HERE # Five stages, and the wall each one hit 1. One session 2. Parallel sessions 3. Memory and schedules 4. ### 00:10:06 ## HOW WE GOT HERE # Five stages, and the wall each one hit 1. One session 2. Parallel sessions 3. Memory and schedules 4. ### 00:10:33 # HOW WE GOT HERE ## Five stages, and the wall each one hit 1. One session 2. Parallel sessions 3. Memory and schedules 4. ### 00:11:06 # HOW WE GOT HERE ## Five stages, and the wall each one hit 1. One session 2. Parallel sessions 3. Memory and schedules 4. A pipeline ### 00:11:34 ## Five stages, and the wall each one hit 1. One session 2. Parallel sessions 3. Memory and schedules 4. A pipeline 5. Ultra ### 00:12:07 ### PART 2 • THE BIG LESSONS WE LEARNED ## Four lessons from shipping this fast 1. Test everything We can't read every change. We can test ### 00:12:30 ## PART 2 • THE BIG LESSONS WE LEARNED # Four lessons from shipping this fast 1. **Test everything** We can't read every change. We can test all of them. 2. Review has to change 3. Teach it to subtract 4. Keep checking 20% of our time goes into tests and checks Ship more as a team of one with Kiro Crew Doug Clauson / Principal PMT 👻 KIRO [https://kiro.ai/crew](https://kiro.ai/crew) [Donut chart showing 20% highlighted in purple] ### 00:13:02 ### PART 2 · THE BIG LESSONS WE LEARNED # Four lessons from shipping this fast 1. Test everything 2. **Review has to change** 3. ### 00:13:34 ## AI Engineer Code Summit ### PRESENTED BY Google DeepMind ## PART 2 · THE BIG LESSONS WE LEARNED # Four lessons from shipping this fast 1. Test everything ### 00:14:06 ## PART 2 · THE BIG LESSONS WE LEARNED # Four lessons from shipping this fast 1. Test everything 2. Review has to change 3. Teach it ### 00:14:37 ## PART 2 · THE BIG LESSONS WE LEARNED # Four lessons from shipping this fast 1. Test everything 2. **Review has to change** 3. Teach it to subtract 4. Keep checking ### What our reviews look like today - **Code reviewers**: two models - **Design**: right shape? - **First principles**: needed at all? - **UX**: screenshots first - **Security scope**: new refusals Ship more as a team of one with Kiro Crew Doug Clauson / Principal PMT 👻 KIRO [https://kiro.ai/crew](https://kiro.ai/crew) [Flowchart illustrating a PR review process, showing inputs like "Code reviewers", "Design", "First principles", "UX", and "Security scope" leading to "PR Readiness" (one verdict). From "PR Readiness", a "green" path leads to "Merge", and a "not green" path leads to "prepare-pr" (fix · reply · push), which loops back to "PR Readiness". A separate box shows "Review record" (what we answer for).] [QR code linking to kiro.ai/crew] ### 00:15:10 AI Engineer Code Summit PRESENTED BY Google DeepMind PART 2 • THE BIG LESSONS WE LEARNED # Four lessons from shipping this fast 1. Test ### 00:15:38 # Four lessons from shipping this fast PART 2 • THE BIG LESSONS WE LEARNED 1. Test everything 2. Review has to change 3. Teach it ### 00:16:10 ### PART 2 • THE BIG LESSONS WE LEARNED # Four lessons from shipping this fast 1. Test everything 2. Review has to change 3. ** ### 00:16:38 # AI Engineer Code Summit ## PRESENTED BY Google DeepMind ### PART 2 • THE BIG LESSONS WE LEARNED ## Four lessons from shipping this fast 1. ### 00:17:06 # The agent isn't the bottleneck. ## **How you work is.** Are you getting everything you can from your agents? - Our attention - A laptop lid - A fixed pipeline - Endless review - A loop that only adds ### Ship more as a team of one with Kiro Crew Doug Clauson / Principal PMT KIRO [https://kiro.ai/crew](https://kiro.ai/crew) ### 00:17:38 # AI Engineer Code Summit PRESENTED BY Google DeepMind ## Are you getting everything you can from your agents? ## The agent isn't the bottleneck. **How you work is.** ### 00:18:06 # Are you getting everything you can from your agents? ## The agent isn't the bottleneck. ### **How you work is.** - Our attention - A laptop lid - A ### 00:18:38 # AI Engineer Code Summit ## Presented by Google DeepMind TRY IT TODAY # Build your own **software factory**. Scan to open the workshop - [github.com/kirodot ### 00:19:10 # Build your own **software factory.** Scan to open the workshop [github.com/kirodotdev-labs/sample-kiro-crew-app-workshop](https://github ### 00:19:38 # AI Engineer Code Summit ## Presented by Google DeepMind ### LIVE TRANSLATION // AVAILABLE Scan the QR code to access real-time translation. [https://events.