# Building a Safe Software Factory — Andrew Finch — session 2026-11-11T20:10:00.000Z → 2026-11-11T20:30:00.000Z _45 transcript lines · 38 slides_ ## Transcript Okay. All right. Can you all hear me? All right. My name is Andrew. And I think we're starting now. I don't know if there's a timer anywhere. My name is Andrew. And I'm excited to talk to you today about the safe software factory. But first a little bit about me. I am a staff software engineer at Sonar. I work on Guitar, which is Sonar's AI code review product. And previously, I was at Uber working on the driver trip experience and navigation. And then after that, I was at a health tech startup called Forward for a little while. Building sort of an AI enabled primary health care system. Okay, so the software factory This is like the hottest buzzword in our industry right now. Everyone's talking about software factories. Everyone's trying to build a software factory. And the question is well, oh, well, so first of all Most of you probably know what this is already. But if you don't, I'll just give a really quick overview. This is sort of the main software development lifecycle loop, as it existed before the AI era. So we have inputs, basically, product specs, feature requests, tickets, things like that. We take those into the planning step. If they're Especially if they're complicated. And then we build. Then we write the code. And then we put up a PR. And that PR gets reviewed. And it goes through CI. And then, it eventually merges. And then, we deploy that to production. And then, users interact with our software. And they give us feedback or they find bugs. Or we might also get some Monitoring and alerts as well. And then, those go back in as more inputs. And the cycle continues. And so in the era of agentic coding and AI, automation has followed every bottleneck. So the bottleneck used to be the coding part, the building. Now, with coding agents, that's been automated. And now, the bottleneck has shifted over to That review step. And we're starting to introduce AI code review systems for that. And then, we're seeing AI systems and automation happen really at every step of this now. So we're starting to see AI systems for monitoring deployments, controlling rollouts, triaging, triaging tickets and incidents, planning. And so this here Is the sort of software factory that we're creating. And so the question that we all have now is, how do we make this factory safe, right? In other words, how can we introduce enough layers of verification into this so that we can trust that the software coming out of it is it works as expected. It's free of bugs, free of secure. Vulnerabilities. Our CEO, Tarek, said recently code is abundant. Confidence is not. So what I'm going to talk about today is just how we at Sonar have been thinking about how to introduce confidence at all different points within this software factory. And we're approaching this This challenge with a defense in-depth approach that comes from adding more loops, you might not be surprised to hear. And I'm going to talk about four of them today in particular. So there's the loop one, which is the PR checks loop, loop two, which is the post deploy verifications loop, loop three, which is the maintenance loop. And loop four, which is the knowledge loop. And this is kind of a special loop. So first, I'm going to talk about loop one, the PR checks loop. So this is the loop that we probably all have in some form. You put up a PR. It goes through some CI checks. It gets a code review. Nowadays, it's probably getting an AI code review as well. But we're starting to see a lot more automation being introduced here as well. So So we're starting to see agents push Fixes to respond to the code review comments. And this loop kind of goes around and around until the PR turns green. And there's actually a lot more that we can be doing here. Because catching a problem at this point is still cheaper than after it merges into the codebase. So we have a real opportunity here to widen this review past this Sort of normal tests and code review that most of us do. So of course, we have AI code review. We can also do static analysis here. Many of us do that. We could do what's called functional validation. So this is where we can look at what the ticket asks for and see if the PR actually addresses those things. We can look at rules that the team may Set for coding conventions, see if best practices, things like that, see if the PR is adhering to those. We can look at knowledge. So this can be knowledge accumulated from company documentation, from things that have been learned on past code reviews or past outages. We could do cross repo checks. So we could actually look at other repositories and see if anything in this PR Will break or cause something to break in another repository. We could do cross PR analysis. So we can look at all the other open PRs as well and see if there's anything in this PR that conflicts with one of those. We could do security review, check for vulnerabilities, secrets, dependencies. We could do end to end testing. So for example, if this is a Web app we could open up in a browser and click around and see if it works as expected. And finally, we can do risk assessment. So we could look at this and compute the blast radius and categorize this into low, medium, or high risk. And that can actually help us determine if this is a PR that a human even needs to look at at all. And so all of these things can flow into this This sort of decision, this merge gate, this decision of whether this PR is mergeable. And of course, every time a fix is pushed, all of these run again, at least in some degree. And with all of this, it's important to gate that merge decision, not just on the fact that a test ran or that it passed, but on the evidence. So that would be things like the actual code review findings, the results of the quality gates, any Vulnerabilities, the risk level. So all of that can be basically fed into this merge gate and checked against criteria that the team can set for what is a mergeable PR. And when you record that evidence to the PR, it just makes this, the merge gate, much stronger. And so what we're seeing now with teams who do this well is that they can To delegate more and more trust to this type of automation on the PR review. So we're seeing teams begin to turn on not just automatic merge blocking but even auto approve, in some cases, for low risk changes and even auto merge for certain types of PRs that are low risk and approved. And this can just free up a lot of review bandwidth for human reviewers to review the more high risk changes. Okay. So now I'm going to talk about loop two, which is our post deploy verification loop. So this is, I would say, very much in its early stages right now. People are experimenting with all kinds of things here. But the idea is that once your code is deployed, you can build AI systems to check, to read production logs, traces, metrics, and see if your Code is actually behaving as you expect it to in production. And then from there, you can actually, we're starting to see things like agentic rollout, where agents can decide whether to widen a rollout or rollback, turn a feature flag off. And you can also do AI outage monitoring. So you can monitor alerts and use AI systems to help you root cause and fix Outages. Okay, so moving on to loop three. So this is what I call the maintenance loop. This is kind of a parallel loop that runs alongside your main software development lifecycle loop. And this is where we continuously scan the code base on some schedule. And we check for a lot of the same things that we check for on the PR. We could do code review, static analysis, security check. Etcetera. And then we can if we find an issue, we can open up a remediation PR to address that. And this lets us catch things that we couldn't catch in the PR. So, for example, you could imagine maybe a new rule was introduced after code merged. Or a new vulnerability was published after code merged. Or even something like slow drift, where you slowly get duplication or complexity. Building up that you can't really see on a single PR. You can catch these kinds of things here with the continuous code based scans. Okay. And so finally, we move over to the knowledge loop. And this is becoming, we're seeing this become a very important loop for a lot of companies. Every company has a lot of knowledge. Documentation, product specs. Business decisions. And the agents do a better job if they have that knowledge, right. So this loop basically collects all that knowledge from different sources, As well as things that were learned on prior code reviews and from prior outages. It's curated. And then it can be fed into all of the other loops. All of these other agents. So they can do a better job. And for example, this is really powerful with your coding agents. Because if they have some of this knowledge, they can probably do a better job the first time, fewer issues. And then you'll wind up with, basically, fewer review laps at the review stage. Okay. So what does this Mean for our role as human software engineers in this factory? You might be asking, Okay, so if we nail all of this, can we just turn the lights out and run a dark factory? And I would say probably not. Not right now. I like to say the factory runs dim, not dark. It's still very important for us to intervene in places where our expertise is really important. So that would be things like auditing risky code, verifying that evidence, verifying things are actually working as expected, Handling exceptions, like rollbacks and incidents. Reviewing architecture Agents are not very good at this yet, I would argue. And supplying the why or helping build that knowledge. That powers all these agents and all these loops. Supplying the product intent, the trade offs, the business decisions, things like that. And of course, we engineer the factory ourselves, right. We've sort of moved into this era where we're building the software factory as a whole. So with that, I hope that gave you sort of a framework for how to think about Not just how to build your software factory, but how to build a safe software factory. So, thank you. That's all from me. ## Slides ### 00:00:05 # AI Engineer Code Summit LIVE FROM SF//CA [Faint, dark map of a city in the background] ### 00:01:08 # Andrew Finch ## Staff Software Engineer · Sonar ### Career Progression - **PREVIOUSLY** - **Uber**: Mobile apps, driver growth, trip experience and navigation ### 00:01:42 # AI Engineer Code Summit ## The Software Factory Presented by Google DeepMind ### Building a Safe Software Factory Andrew Finch / Staff Software Engineer Sonar [Progress bar showing "FACTORY" as ### 00:02:11 # The Software Development Lifecycle [Diagram showing a circular software development lifecycle with five stages] - Inputs - Plan (if complex) - Build - Review & Merge (CI, code review, merge) - Deploy (to production) ## Building a Safe Software Factory ### Andrew Finch / Staff Software Engineer Sonar [QR code linking to https://www.sonarsource.com/](https://www.sonarsource.com/) ### 00:02:36 # The Software Development Lifecycle - **Inputs** - product specs - feature requests - tickets - slacks - **Plan** ( ### 00:03:08 # Automation has followed every bottleneck. - **Inputs** - product specs - feature requests - tickets - slacks - ### 00:03:36 # Automation has followed every bottleneck. - **Inputs** (from product specs, feature requests, tickets, slacks) are handled by **triage agents**. - **Plan** (if complex) is assisted by **planning agents**. - **Build** is performed with **coding agents**. - **Review & Merge** (CI, code review, merge) involves **AI code review** and **merge queues**. - **Deploy** (to production) uses **progressive delivery**. - A feedback loop from user feedback, monitoring, and alerts is ### 00:04:08 # How do we make the factory **safe**? ## AI Engineer Code Summit Presented by Google DeepMind ### Building a Safe Software Factory Andrew Finch / Staff Software Engineer Son ### 00:04:37 > “Code is abundant, confidence is not.” > Tariq Shaukat · CEO, Sonar ### Building a Safe Software Factory Andrew Finch / Staff Software Engineer Sonar - ### 00:05:09 # Defense in depth comes from more loops. - **Inputs** - *Receives:* product specs, feature requests, tickets, slacks - **Plan** - ** ### 00:05:41 # The PR review loop repeats until green. ## PR Review Process (Loop 1) - **PR checks** (central) - CI checks - push revision - AI code ### 00:06:08 # The PR review loop repeats until green. ## Revisions on one PR 1 ## Every revision runs - CI checks - AI code review - more layers of checks ### Current Section Building a Safe Software Factory ### Speaker Andrew Finch / Staff Software Engineer Sonar [Circular flow diagram showing a PR review loop with steps: push revision, CI checks, PR checks, AI code review, agent fixes.] [Horizontal progress bar indicating "PR CHECKS" as the current stage.] [QR code linking to https://g.co/codesummit] ### 00:06:39 # Widen review past the diff. Catching a problem here is cheaper than after merge. - ONE CHANGE PR - AI code review: defects in the diff - Static analysis: deterministic quality rules ## Building a Safe Software Factory Andrew Finch / Staff Software Engineer Sonar [Diagram showing a process flow from "ONE CHANGE PR" to "AI code review" and "Static analysis", which identify "defects in the diff" and "deterministic quality rules" respectively.] [Horizontal timeline showing stages: Intro, Factory, PR Checks (highlighted), Post-Deploy, Maintenance, Knowledge, Humans, Outro.] [QR code for live translation.] ### 00:07:09 # Widen review past the diff. Catching a problem here is cheaper than after merge. - **ONE CHANGE PR** - AI code review: defects in the diff ### 00:07:41 # Widen review past the diff. ## Catching a problem here is cheaper than after merge. - **ONE CHANGE PR** - AI code review: defects in the diff - Static analysis: deterministic quality rules - Functional validation: does what the ticket asked - Rules: the team's own conventions - Knowledge: what the team learned before - Cross-repo: impact on other repositories - Cross-PR: collisions with open PRs ### Building a Safe Software Factory Andrew Finch / Staff Software Engineer Sonar [https://www.aicode.ai/](https://www.aicode.ai/) [Mind map diagram showing "ONE CHANGE PR" connected to seven bullet points, each with a short description.] [Timeline diagram showing stages: INTRO, FACTORY, PR CHECKS, POST-DEPLOY, MAINTENANCE, KNOWLEDGE, HUMANS, OUTRO, with "PR CHECKS" highlighted.] [QR code linking to aicode.ai] ### 00:08:05 # Widen review past the diff. Catching a problem here is cheaper than after merge. ### ONE CHANGE PR - **AI code review**: defects in the diff - ### 00:08:40 # Widen review past the diff. Catching a problem here is cheaper than after merge. ## One Change: PR - **AI code review**: defects in the diff - **Static analysis**: deterministic quality rules - **Functional validation**: does what the ticket asked - **Rules**: the team's own conventions - ### 00:09:04 # Gate the merge on evidence. ## Layer - AI code review: `head sha + merge base` - Static analysis: `quality gate conditions` - Functional validation: ### 00:09:40 # Delegate more as trust builds. ## Risk analysis - **on** for every PR - Every change gets a risk band. ## Merge blocking - **block** - High risk ### 00:10:12 # Verification continues after deploy. ## LOOP 2: **Post-deploy** - monitor - rollout + flags - verify - root cause ### Building a Safe Software Factory Andrew Finch / Staff Software Engineer Sonar [Diagram showing a circular "Post-deploy" loop with steps: monitor, rollout + flags, verify, root cause.] [Progress bar indicating "POST-DEPLOY" as the current stage in a sequence: INTRO, FACTORY, PR CHECKS, POST-DEPLOY, MAINTENANCE, KNOWLEDGE, HUMANS, OUTRO.] [https ### 00:10:36 # Verification continues after deploy. ## LOOP 2 Post-deploy - monitor - rollout + flags - verify - root cause 1. **Post-deploy verification ### 00:11:08 # Scan the whole codebase on a schedule. ## LOOP 3 Maintenance - scan main - findings - remediation PR - schedule ### The same checks as every PR - AI code review ### 00:11:37 # Scan the whole codebase on a schedule. ## Loop 3: Maintenance - findings - scan main - schedule - remediation PR ### The same checks as every PR ### 00:12:08 # Scan the whole codebase on a schedule. ### The same checks as every PR - AI code review - static analysis - security - rules - knowledge ### Catches what no ### 00:12:36 # Knowledge is the loop that feeds the others. ### WHAT EVERY LOOP TEACHES - **Architecture decisions** Why the system has the shape it has. - **Product intent** ### 00:13:09 # Give the coding agent what review knows. ## Process Overview - **Build** feeds into **LOOP 4 Knowledge**. - **Review & Merge** (CI · code review · merge ### 00:13:40 # So what is our role in the factory? ## Building a Safe Software Factory Andrew Finch / Staff Software Engineer, Sonar - INTRO - FACTORY - PR CHECKS - POST-DEPLOY - MAINTENANCE - KNOWLEDGE - **HUMANS** - OUTRO [Timeline showing presentation stages with "HUMANS" as the current stage] [QR code for live translation] ### 00:14:08 # The factory runs dim, not dark. ## Software Factory Process Diagram * **Inputs** (fed by product specs, feature requests, tickets, slacks) * **Plan ### 00:14:41 # The factory runs dim, not dark. - **Inputs** - *Details*: product specs, feature requests, tickets, slacks - **Plan** ### 00:15:09 # Safe Software Factories Presented by Google DeepMind at AI Engineer Code Summit ## Building a Safe Software Factory Andrew Finch / Staff Software Engineer Sonar [Timeline showing stages: Intro ### 00:15:41 # AI Engineer Europe 2027 ### ExCeL London ### 02.17 - 02.19 2027 Europe's flagship technical ### 00:16:06 # AI Engineer Code Summit ## STAY IN THE LOOP SCAN THE QR CODE OR VIEW THE SCHEDULE AT AI.ENGINEER/SCHEDULE [https://ai.engineer/schedule](https://ai.engineer/schedule) [QR code linking to ai.engineer/schedule] ### 00:16:37 [Abstract background of dark grey dots forming a subtle wavy pattern] ### 00:17:05 # Code Summit ## AI Engineer LIVE FROM **SF//CA** [Faint, wireframe-style map of a city in the background] ### 00:18:06 # Code Summit ## AI Engineer ### 00:18:37 # LIVE TRANSLATION // AVAILABLE ### AI Engineer Code Summit Presented by Google DeepMind Scan the QR code to access real-time translation. [https://events.google.com/io/translation](https://events.google.com/io/translation) ### 00:19:09 [Dark background with a subtle grid of grey dots] ### 00:19:37 ## AI Engineer Code Summit Presented by Google DeepMind ### Available # LIVE TRANSLATION Scan the QR code to access real-time translation. [QR code to access real-time