# All Code Is Slop (Except Mine) โ€” Lauren Tan โ€” session 2026-11-11T18:10:00.000Z โ†’ 2026-11-11T18:30:00.000Z _33 transcript lines ยท 39 slides_ ## Transcript Because they go stale easily. You know, humans just really don't bother reading the comment slot anyway. And again, it gives an agent the... An escape hatch to justify not doing the right thing because it's hard. And the worst part about comments, though, is that they're not enforceable. So what seems well intentioned, you know, like, I'm gonna tell the the next agent not to do something, never ends up actually being anything that you can trust. So Grokbot and Dune and in Dune, we now have lint rules that give, good and very helpful error messages telling the agent to fix the root cause, the real problem, rather than just patch around it. And so our general principle is that every time we correct our agents, it turns into one of these deterministic checks, or we think about how to categorically eliminate that problem into code through better API design, through better architecture. And if you do this enough times, You'll find that you have just less things to look out for in code review. And at some point, you realize there's there's nothing left to review, and you can just let agents merge their... Most of their own pull requests. Alright. Moving on. Verification is also a really big part of how we make our agents more autonomous and also increase our confidence and trust in their work. And our code review process has now really shrunk into looking at verification artifacts instead. So things like videos, screenshots, diagrams, formal verification. And the most basic example of this is runtime verification, which is just a fancy way of saying giving your agent a way to run the code for itself and use debugging tools that, you know, human developers use as well, like the Chrome DevTools protocol. In our runtime verification skills, we have this thing that we call a feature map, and that's basically kind of like a sitemap of all of the features that are In your application from the user's point of view. And so with this, agents don't need to guess when you say, you know, the thingy in the left sidebar. It gives you some shared vocabulary. And we also make use of custom CLIs inside these skills so that, agents can reuse the the CLI and the driver every time, which cuts down on token usage and is also testable. And then recently, we've also started looking at formal verification. So we now use TLA Us to do some model checking of major user flows, such as, notifications across multiple devices and platforms. We use a lean as well for proving certain algorithms. For example, we have our own... We wrote our own custom virtualization library, library, and we use Lean there to check that we don't have, you know, flickering or jittering problems. And we're still experimenting with this approach overall. It seems very promising. We've already found quite a few bugs, with formal verification. Not a silver bullet, and we're treating it as another tool, another layer in our agents' environment. And then this is a very fast video, but, another thing that we just started looking at is property based testing as well for UIs. So we use this library called Bombadil. And this has also caught a few UI bugs for us already. Okay. So zooming out, I saw this tweet from Cindy. And six months ago, She said that she was, she was talking about not being able to trust her agents because of subtle bugs that were getting merged. And really what was missing for her in being able to trust her agents was the verification process. And now six months later, she says that 70% of her daily tokens go into verification, and I'd say that that tracks with what I see as well, that most of my token usage goes to verification. And if you really look closely at our process, none of these things here are New or novel techniques that we've never had before. These are just good software engineering tools, but ones that might have been prohibitively expensive to use before as they were hard to understand or write or maintain. But agents drastically cut this cost down and really we're entering a golden age of software testing and verification because agents let us test everything to a much greater degree than used to be economical. And then formal verification. Can also help you better reason about how your product should work and whether your agents really understand your intent and goals. I made this new skill called statechart, in my plug in p stack, potato stack, which helps you visualize state machines from, from the user's point of view. And it's... I find this much easier to review alongside the videos and screenshots that we provide. And so that's been great. Sorry for the flashbang. Okay. Now let's look at what the promised land looks. Like, what do you get in return for all of this, you know, investment? So on the GrokBot team, our velocity is, taking off and rapidly increasing. And last month, our team shipped a ridiculous 11,000 pull requests to production, which is a wild number. And our pace of releasing the client has also increased, and we cut a new stable version about once a day. And then the core team is also shipping my More. And we also have, a lot more casual or drive by contributors who don't always have a lot of context, but they're also contributing. And in October, we had about two twenty seven people who contributed at least one pull request. So despite this velocity accelerating, you might be thinking to yourself, like, isn't this gonna be chaos? You know, isn't this gonna be a bunch of flop? But actually, the number of, PRs that get reverted has actually gotten lower on a relative And it's steady at about half a percent. But I think the really interesting thing about this shipping pace and this volume of pull requests is that the performance of GroBot has stayed not just stable, but has improved on multiple dimensions compared to when we launched. So start up, typing, opening new chats, switching chats, all of this has gotten faster. And keep in mind, Grokbot is still an Electron app for now. So there is a ceiling, right? But it's already much faster than many other agent UIs out there. And for an Electron app with this high velocity and this many people contributing, I'd say that's pretty good. So before you jump into building out a software factory and, you know, automating everything that you can, whenever you correct your agent, try and do it by improving your code base first. Know, start with a lint rule to prevent new occurrences of anti patterns, and then clean up those Those, old patterns as well. And then make sure that your verification story is really strong. A lot of this work is a prerequisite to automation anyway. And if your inner coding loop, produces slop, then automating that loop just produces more slop and you need to throw more and more and more tokens at the problem, which we all know is very expensive. And once you've made these investments into, you know, code into your codebase, static analysis, runtime informal verification You'll start to find that many things become possible with these foundations and that you can really easily build your own factory on top of these foundations. And when you're ready, GrotBot is a great way to build that, outer loop. We have tons of connectors, routines that can kick off Cursor, Cloud agents, and projects, which let you orchestrate thousands of, hundreds or even thousands of agents. And as a quick example, you know, whenever we cut a new release now, GrokBot will automatically Up a new cursor project for us that will fuzz the entire application, the entire build, and it will find bugs, by running the app and using it like a real user. And then it's going to open fixed PRs for each of those bugs that it finds, and it's going to auto merge it, as, crazy as that sounds. And that's because our codebase is good enough that pull requests tend to just be good by default, and most product code that we land no longer really requires deep human review. So when you have solid foundations and good organic architecture, you can turn slop into gold, and you can ship with high quality, high velocity without compromising on quality. My name is Potato, also known as Lauren. Thanks to you again for listening to my talk. That concludes this morning's, keynote sessions. Thank you for your patience with the various technical challenges. The root cause was, of course, a GitHub outage. Not really, but the fact that no one would think to question that does say something. I can say that because they are not sponsors. But do you know who Is... That's right. Presenting sponsor, Google DeepMind. Another round of applause for them, please. Without them, this would not be doable. Of course, our lounge and platinum sponsors as well have been incredibly supportive of various things. Not my nonsense, but, you know, most of The rest of the good stuff. Absolutely. And let's give it up for our silver sponsors too. Thank you. And finally, all of our supporting partners as well. There are, and, of course, viewers like you, to quote PBS from my childhood. I'm old, back when we used to watch it on the Telegraph. There are now some breakout sessions. My favorite part of those, if you erase part of the letter b, it reads freak out sessions. You can read into that what you'd like. There are another series. Of closing keynotes later today starting at 04:30. And I'll see you all back here then if they don't wind up having a conversation for me out back next to the dumpsters with some broken beer bottles. Thank you all for coming. ## Slides ### 00:00:09 # quick fixes considered harmful ## Sep 9 ```javascript // release later, in case it comes back setTimeout(() => release(id), 0) ``` - fixes a ### 00:00:41 # comments are liabilities also see /no-comments in pstack: [github.com/cursor/plugins](https://github.com/cursor/plugins) @poteto ### All ### 00:01:08 # comments are liabilities [Bar chart showing a value of 65,824 on July 31st decreasing to 1,227 today] also see /no- ### 00:01:40 # whenever you correct your agent lint against it or categorically eliminate the problem ### All Code Is Slop (Except Mine) Lauren Tan / Principal Engineer SPACEX @poteto [Step ### 00:02:08 ## runtime verification also see /create-verification-skill in pstack: [github.com/cursor/plugins](https://github.com/cursor/plugins) ### All Code Is Sl ### 00:02:39 # runtime verification also see `/create-verification-skill` in pstack: [github.com/cursor/plugins](https://github.com/cursor/plugins) [Flow diagram titled "feature map" showing a sequence of nodes: "what features exist" connects to "how users reach it", which then branches to two separate nodes, both labeled "what proves it works".] ### 00:03:09 # formal verification ### TLA+ - Model a state machine. - Check that its rules hold. ### Lean - Write proofs about code. - Check every logical step. ### 00:03:41 ## ui property based testing [github.com/antithesisha/bombadil](https://github.com/antithesisha/bombadil) [Screenshot of a chat application ### 00:04:14 # All Code Is Slop (Except Mine) ## Cindy Sridharan @copyconstruct > Revisiting my tweets from 6m ago is funny, because I don't run into ### 00:04:37 # PRE-PRODUCTION - Cindy Sridharan - @copyconstruct - **UNIT TESTS** - **FUNCTIONAL TESTS** - **COMPONENT TESTS** - **FUZZ TESTS** ### 00:05:08 # AI Engineer Code Summit ## All Code Is Slop (Except Mine) /statechart [github.com/cursor/plugins/tree/main/pstack](https://github ### 00:05:37 # grok bot team merged 11k PRs in oct ## All Code Is Slop (Except Mine) Lauren Tan / Principal Engineer SPACEX [Bar chart showing monthly pull request ### 00:06:09 # everyone is more productive ## All Code Is Slop (Except Mine) Presented by AI Engineer Code Summit, Google DeepMind Lauren Tan / Principal Engineer SPACEX [Stacked bar chart ### 00:06:41 # velocity without compromise ## Performance Metrics ### startup - **cold launch** - may: 2,304 ms - now: 1,3 ### 00:07:08 # velocity without compromise ## startup - **cold launch**: may 2,304 ms, now 1,395 ms (-39%) - **relaunch**: ### 00:07:41 # Should I build a software factory? * aka should I automate writing, shipping, deploying code? ## All Code Is Slop (Except Mine) **Lauren Tan** / Principal Engineer SpaceX ### 00:08:04 # don't automate a bad loop ## low trust most people get stuck here - agent writes **slop** code - **human** verifies it - issues found ### 00:08:36 # grok bot as your outer loop sandcastle AGENT 11 minutes ago Grok Bot ๐Ÿค–, Linux fuzz results We fuzzed the stable Grok Bot ### 00:09:14 # all code is slop (except yours) ## Speaker poteto (aka lauren tan) Grok Bot @ SpaceXSI ### Talk Title All Code Is Slop (Except ### 00:09:34 # AI Engineer Code Summit PRESENTED BY Google DeepMind [https://events.google.com/ai-engineer-code-summit/](https://events.google.com/ai-engineer-code-summit/) ### 00:10:08 # AI Engineer Code Summit ## PRESENTED BY Google DeepMind ## SUPPORTED BY G2i ### AI Engineer Code Summit 2026 Corey Quinn / Chief Cloud Economist **duckbill** ๐Ÿฆ† [QR code for live translation] ### 00:10:37 # AI Engineer Code Summit PRESENTED BY Google DeepMind ## SILVER SPONSORS - BAND - Cloudflare - Datadog - Judgment Labs - Keycard - Nimble - Parasail - Warp ### AI Engineer Code Summit 2026 Corey Quinn / Chief Cloud Economist **duckbill** [Logos for ### 00:11:09 # AI Engineer Code Summit Presented by Google DeepMind ## Supporting Partners - Firecrawl - Vercel - ramp - X - Lovable - M - Reflection ### 00:11:41 # AI Engineer Europe 2027 ## ExCeL London 02.17 - 02.19 2027 Europe's flagship ### 00:12:08 ### THE IDEA # KV-first security. ## The signal your engine already writes. ### WAYS AN AGENT GOES WRONG - Prompt injection - Jailbreaks ### 00:12:40 ### THE IDEA # KV-first security. ## The signal your engine already writes. ### WAYS AN AGENT GOES WRONG - Prompt injection - Jailbreaks - Rogue ### 00:13:05 # AI Engineer Europe 2027 ExCeL London 02.17 - 02.19, 2027 Europe's flagship technical AI conference ### 00:13:33 # AI Engineer Code Summit ## STAY IN THE LOOP SCAN THE QR CODE OR VIEW THE SCHEDULE AT AI.ENGINEER/SCHEDULE [https://ai.engineer/schedule](https://ai.engineer/schedule) [QR code linking to ai.engineer/schedule] ### 00:14:13 [A grid of dark grey dots on a black background, with a subtle radial gradient effect making the dots in the center slightly lighter and more prominent] ### 00:14:37 # AI Engineer Europe ## AI Engineer Europe 2027 ### ExCeL London - 3 Days - London, UK ### Dates: February 17-1 ### 00:15:10 # KV-first security. ## The signal your engine already writes. ### THE IDEA ### WAYS AN AGENT GOES WRONG - Prompt injection - Jailbreaks ### 00:15:34 # AI Engineer Code Summit LIVE FROM SF//CA [Stylized map of a city in the background] ### 00:16:09 # KV-first security. The signal your engine already writes. ## THE IDEA ### Ways an agent goes wrong - Prompt injection - Jailbreaks - Rogue tool calls - Broken tool calls - Sensitive-data exposure - Hidden backdoor - Drift - ...and the ones not named yet ### Judge or classifier - reads the prompt - reads the response ### Already there Your engine writes it anyway. The gems from the pile. ### Covers the blind spots What slips past the guardrails still becomes computation. ### Stays in place Read it where it lives. The prompt text goes nowhere new. [Diagram illustrating a security flow: various agent failure modes feed into a computation process labeled "KV pages", with a baseline learned from traffic and a "guard ### 00:16:37 # KV-first security. ## The signal your engine already writes. ### THE IDEA ### WAYS AN AGENT GOES WRONG - Prompt injection - Jailbreaks - Rogue tool calls - Broken tool calls - Sensitive-data exposure - Hidden backdoor - Drift - ...and the ones not named yet ### ALREADY THERE Your engine writes it anyway. The gems from the pile. ### COVERS THE BLIND SPOTS What slips past the guardrails still becomes computation. ### STAYS IN PLACE Read it where it lives. The prompt text goes nowhere new. [Diagram illustrating a security model where various "ways an agent goes wrong" (like prompt injection, jailbreaks) are processed. A "Judge or Classifier" reads the prompt, which enters "KV Pages" for computation, based on "one baseline, learned from your own traffic". Some items "does not fit" and are outside the computation area, representing "the guardrails' blind spot". The output leads to "Tool calls, actions".] ### 00:17:09 # KV-first security. ## The signal your engine already writes. ### THE IDEA **WAYS AN AGENT GOES WRONG** - Prompt injection - Jailbreaks - ### 00:17:39 # AI Engineer Code Summit ## LIVE FROM SF//CA [Subtle wireframe map of a city in the background] ### 00:18:05 [Dark background with a subtle halftone dot pattern, appearing to bulge slightly in the center] ### 00:19:03 # AI Engineer Code Summit ### Presented by Google DeepMind ## // AVAILABLE ## LIVE TRANSLATION Scan the QR code to access real-time translation. [https://codesummit.withgoogle.com/events/ai-engineer-2024/live-translation](https://codesummit.withgoogle.com/events/ai-engineer-2024/live-translation) [QR code linking to the live translation page] ### 00:19:34 [Dark background with a subtle, repeating pattern of small grey dots]