# fighting slop with slop — Vaibhav Gupta — session 2026-06-30T22:20:00.000Z → 2026-06-30T22:40:00.000Z

_66 transcript lines · 22 slides · source: full recording_

## Transcript

Allow me to download it so I could share it with them, and you could see that the agent created this. Fighting Slop with Slop. My name is Vaibhav, and I'm going to talk about something that is a little, I would say, maybe a little silly at first. I'm going to show you our team's engineering practices really quickly. We do no code reviews. We require every engineer to work on things in parallel, and we have no standardization on how people do AI. And I know immediately what almost all of you are thinking. We're probably a Zoomer YC startup, and I can guarantee you I'm clearly a millennial. So what do we actually do at our company without code reviews? Well, we about three years ago, we decided to build Programming language. That's something that has absolutely no room for slop. It's something that has to work every single time exactly the same way. It's something that you can't just change a year later or a month later because you made a bad design decision. You have to be correct. And for the last three years, we've been in an onslaught of war against slot. And when I first met this enemy, I went to my great mentor, And he taught me something. To to defeat the slop, we must become the slop. So we began and we prepared and then we started winning. So when we think about it, what is slop? Slop is just any code you don't read. And whether any of you admit it or not, this is the least amount of SLOP that your code base will ever have. Cherish it. So we started fighting back against the SLOP, and we started fighting back with SLOP. So how do we go ship a stable programming language with these engineering practices? Well, the first skirmish we ever had was the skirmish of standards. The hard part about hiring great engineers is you sadly can't tell them what to do. Some of them want to use clouds. Some of them want to use codecs. Some of them want to use the latest thing that they just found on Hacker News. So instead of trying to hold standards in our code base, we did something that is an invariant. We built an architecture dot MD file. Instead of using Cloud MD, just pick something that every model can just Understand. This file has to be incredibly small, and it can only have things that will not change for months or for years. In our case, it's the layers of the compiler. You go deeper into the compiler, tell the agent to just talk to at least one other person. That slows it down a little bit. So now we have standards so anyone can use whatever they want, but the real foe we faced was actually the battle of design. Everyone here knows that you have to write perfect design docs, and we have a very simple rule in our team. It can be slop, writing cannot. And of course, if I tell every engineer this, they write beautiful writing and they handwrite everything, they don't use AI. Well, sadly not. So we built a design tool, design doc tool. What this design doc tool does, it's a replacement for both Notion and GitHub effectively for design docs. It allows versioning, commenting, all the stuff you want, and obviously we do this, people use this. Well, sadly not. We built another tool on top of that, and this tool was a Slack integration for that tool. Every time a design doc got updated, this channel got notifications. And what ended up happening is this Became the most popular channel in our company really fast. At 2AM, someone shipped a new design doc. Three people started reading it right away because it's just interesting. The most interesting stuff is design docs that are not going to change. But this wasn't enough. All of this is actually backed by markdown files and simple CLI scripts that make it treat like GitHub without being GitHub itself. So now agents can go do this, But the real problem with all this is I built this, and I hit a little bit of AI psychosis, and I started shipping 10 design docs a day, and soon, the team was fighting My slop. So we have to go in back at the last rule. This last rule was if you're going to ship a design doc, you require people to actually go read it, and with this last standard, we suddenly had design docs that are incredibly high quality. But what about the battle of architecture? You know, how do you have your code base converge? We built another tool. This tool basically visualizes our dependency graph internally with some external dependencies as well and allows us to watch the code base change. It has semantic boundaries, individual packages, but what's more interesting is we can go build CLI tools that guarantee that certain invariance can't be broken. And what this Does is when Claude builds a new package or adds a dependency that's leaky, we now have CICD changing or a simple git commit history that tells us exactly where things break. And by this, we're actually able to make our architecture change. We haven't changed our architecture in the last three or four months. But as much as we might do design docs and as much as we might have stable code, would you genuinely ship code without reading it? Would you trust your team to go Do that, and think about a programming language. A programming language has so many invariants. You have generics. You have closures. You have memory allocation. You have f f five boundaries. Could you trust that system? Python has bugs twenty five years later. Well, here's where we did something slightly different. What we did was we built a system that actually has agents constantly running and creating BAML programs. We take these BAML programs one second and we have agents try and spin something up from scratch. We then look at the entire cloud. Transcripts, see what tools they used, see what happened, and obviously, we as humans can inspect them. But more importantly, we can have agents go inspect them, and agents find what was good, what was bad, and not just what was bad in terms of what was incorrect in the language, but what took three tool calls when it should have only taken one. And then we can go ahead and find issues and we can have humans collaborate with these issues to figure out which ones are real, which ones are hallucinations, which ones are don't have. As much as I hate to use that word, and then we can have agents go ahead and create fixes to these problems and go to address them, and most importantly, instead of trying to just detect these issues, we can go one step further. What if you could find language features, and instead of guessing what was good, guessing what skill was good, you could go and AB test it. You could figure out which ones took less tool calls, which one took, which one made less errors, which one produced the correct outcome, and deterministically know what's going on. The point is you can start building data driven systems without ever writing a single line. Code, and the thing that really I care about the most over here is not that any one of these tools is specifically what you should go build, but the fact of the matter is, in order to build a programming language, it wouldn't have taken eight people. It wouldn't have taken less than two years. It would have taken hundreds and thousands and tens of thousands of man hours, and then you would still have a broken system. And today, we can just spend billions of tokens and make it work. And we can make it stable, and you too can go home and build these internal tools and these sloppy tools to make sure that your code bases can ship without really having to read necessarily every single line of code because your engineers aren't going to. And I think we can start winning this battle against slop, and as we win this battle, slop can be defeated, but sadly, I have a sad thing to say. I think we're still going to lose the war. I think the reason that we're going to lose this war is because some of the foundational stuff that we try and go use itself is Broken. How many of you have used TypeScript? Probably most of you hopefully at this point or at least your agents have, so something around there. Did you know that TypeScript's main design goal is to strike a balance in correctness and productivity? And there's an asterisk here because what they really mean is human productivity. And if you think about it, there are things you would never do in a programming language. At the very core, Or layer if you were designing in a world where humans never wrote a single line of code. Let me show you what that really means. I'm gonna write something and try and guess what this code does. Pretty safe. What about this one? Or even more so, this one. Why do we turn things to strings when we sort them? This is just slop. Baked into the language whether you like it or not. What about this? I love this part of TypeScript, and you know what my agent loves? This part of TypeScript. This is slop baked into the language, and whether you like it or not, the systems will have slop if you build using these tools. Oh, I'm sorry. Wrong talk. But if you think about what JavaScript does, JavaScript exists, and then after JavaScript existed We started building systems to layer it on. We built CoffeeScript, then TypeScript, and now we're trying to build effect. The thing is, the thing underneath is already broken, and more so, the way we write code is also different now. So why are we trying to patch something like this? Why don't we just try and do something a little different? And I think what we might need if we try and go do that is basically going to be a made up language. So let me show you what BAML really can do and when you start thinking from first principles, how you can try and combat slop from a very foundational layer itself. I keep talking about not reading code. Does it even matter? Well, let me show you a new way to think about code, and this isn't to say we all have to go do this right away. But what if every single time I looked at code whoops. What if every single time I looked at code, what I really saw was not the code. Itself. But a quick little thing that could actually visualize all the code for me. As I clicked around, it took me to exact lines of code that was linked to. If I wanted to have a slightly broader view, I could zoom in and click around and have it expand, and I could navigate my code bases more interestingly. I'm gonna let this run really quickly, but while it runs, I'll show you a different pipeline. Without any of you ever reading the code, you know I'm setting up stuff and I have an agent loop because the semantic boundaries in there. I can expand this. I can keep expanding this, and I can say, nope, that's too much slop. Let's let that be slop and walk away. So instead of having to understand all the code, I can opt into what parts of the code I want to read and understand and go to the exact lines when I really care about them. But if we go back to the previous pipeline that I was running, what if while it's running, I can actually get a full execution trace? In a world where we don't read all the code, the only way to understand the code is actually by the execution trace and actually by seeing exactly how much time was spent on what parts of my program at any Given time. If you want to go and actually track your program through, think about how slow your program would be if you had to go trace everything in Python or TypeScript. It's untenable, and the best part here is if you start from first principles, you can make this effectively zero performance cost. Not only can we make it great for humans, but because it's all built for agents anyway, you can go ahead and make it so that every single file has a tracing system that Can navigate through. So Claude can find what were bugs, what were errors, and what were inefficiencies and start optimizing your code without you having to do it yourself. And I think if we go start thinking about it from this way, it's not so much about reading all the code, but it's more so about as a human understanding the system that you're working with and the tools that you can build can give you information about the system that you're working with. But, I think there's another layer to it. We've spent decades building IDE tooling, and that, think about how long it took before someone like me who does not know how to escape Vim to this day. Can finally start using Versus Code. It was a beautiful day when that happened. I became a real programmer. Well, according to some people, I'm still not because I can't write VIM code. But, what does agent first tooling look like? I think we're all familiar with grep, so I'm not going to go and talk about it. But I will talk about rip grep because grep should not be used anywhere. If I want to grep through my code base and understand what it was, I would rip grep, say something like calculate, and it would give me a bunch of code where everything was being used, and maybe it would be somewhat useful. But what if you could instead just start describing code and say, can you describe calculate for me? What if it came with all the docstrings? What if it came with the actual source code? And what if it also told you everywhere it was actually used under the hood? We can make something that used to be multiple tool calls a single Cool call all of a sudden. What if the way you wanted to learn about libraries that you are using, instead of having to do a web search, you just said you could just ask for any external library as well, and it would just give in to you because when I first started learning how to code, one valuable lesson I had was the code is always a source of truth. Don't read anything but the code itself. The docs may lie. The, the actual description or architecture file or readme file will definitely lie, but the Code cannot lie except if you're working on some weird architectures. And then when you go down this road, you go from not reading the code to understand the architecture. You go from not searching the code to understanding exactly what you're getting in every one tool call, but what's the next thing you do? Well, the last thing I do to truly understand code is I run the code. So what if every single thing you ran, every single function you ever wrote, was immediately available and I'll pop this code over here Was immediately available as a simple CLI command. So if I run add, add becomes a CLI command that has a and b parameters attached to it. And I can just run it really quickly and see what happens. What if every single CLI tool I had could be packed into a live into CLI that's completely standalone that I can just run without ever having to actually execute any of the code and is now a total CLI binary that has a function just Suddenly, we can build really quick tooling where agents don't have to go grep through what's happening. Everything is type safe, everything is deterministic, and everything is actually guessable. And the best part is imagine you could build on any system, and your agents don't have to worry about deployments across Windows, Mac, and Linux, and you can just target any layer you want and it builds for any architecture, including Wasm systems. All of a sudden, as an Near, you're supercharged. You're no longer bottlenecked by what you can do in the systems underneath you are preventing. You can just move very fast. You can move at agent speed. But a lot of the stuff that I've been talking about to this date has been about tooling. What if we tried to fix some of the real sins of JavaScript? Some of the stuff that is deep in the language. Not the sort stuff, but I mean more important stuff like errors. Have you seen error handling be beautiful ever other than Rust? What I see agents do over here is you do try catch and then they Keep nesting try catch after try catch after try catch, and eventually they give up and say console dot log, some error happen, and deal with it. What if we could do error handling from very first principles? What happens in that world? Well, I showed you add, multiply, subtract. I didn't show you divide. Divide is dangerous. It's spooky. So let's go look at divide. You can see over here, divide. Throws a division by zero error. But what else happens? The function actually knows that it throws division by zero error without you having to write any any code. If I go up to the calculate function, which at some point calls divide, this function also knows is throws division by zero error. So error types now get inferred without you ever having to do any guesswork. That means if you catch or handle errors, we can do exhaustive guarantees, and the compiler can prove that you have handled the error or not handled the error. It's no more guessing. There's no unknown. It's guaranteed to be proven. So if you wanted to ship an API where it guarantees that it never throws, well, this system is broken because it doesn't meet the constraints. It has two errors that you're not throwing. If you wanted to go catch that, well, I can write the code for that in a second. But you can start catching certain errors. Let me, I'm just going to return a sentinel value for now. And now this parsing, which previously threw a division by zero error, is now guaranteed to no longer throw the division by zero error because if I catch any exceptions in here, I return a zero value every single time. The compiler and the tooling can do a lot of work for us, and we're already used to this in our code bases. Many of us probably don't know how compilers work under the hood and we trust them. Code is a matter of trust. The reason that we don't use lm code blindly is because we don't trust Yet because the systems underneath them don't have enough rigidity. One more thing. But before I tell you all to go write a bunch of BAML code because I've been there and I can tell you what someone would tell me if I said, hey, use this new programming language. It's going to solve all your problems. It's just going to become- come with a whole slew of new problems. So we said, I think we'll lose the world. On SLOP if we tried to ask everyone to rewrite all their code in the world into this new system. So what does a solution like that look like where you don't have to rewrite all your code? Well, we what we started to do was we started to think about that about two years ago, And we said, what if you could use BAML, not just standalone like I showed today, but from within any existing language of your choice, from Python to TypeScript to Rust to Go to Ruby to Java to anything new that comes up even after it. What if every function in bamil is immediately accessible in the language of choice? So in this case, I'm calling the bamil calculate function directly from Python, and it's completely type safe. Not only do we get calculate, we get calculate async in case some of us want to write async code. So BAML, while it has no function coloring, it does give you the benefit of having to do whatever you want across your code. But what if you went a little bit sillier? What if you started passing around lambdas across language boundaries? I have a function here called with time out. This function times out after a certain number of milliseconds and if this work doesn't complete, and it's guaranteed to no matter how long it takes. Well, in that world, you can even pass Python lambdas across the bridge. You can pass generics across the bridge. You can pass closures. It It should just work so engineers don't have to go fuss with it, and more importantly, so when the agent does something, the type system never lies. The type system becomes the Center of truth that prevents invariants from entering your code base. And what I really wanted to talk about today was not any one specific thing, but it's a general concept. You can build incredibly complex systems without traditional systems like code reviews. You can work in things in parallel and you can use AI however you want without requiring any sort of standardization, but the most important part is you have to be incredibly thoughtful about how your engineering team actually uses the systems under the hood. When we started building BAML, I didn't think it would be possible to build some of the software we

## Slides

### 00:01:13

1. No code reviews.
2. You must work on things in parallel.
3. Use AI however you want.

### 00:01:41

# Engineering the future of AI
## BAML
[Illustration of a person in a purple sheep costume sleeping on a bed]

### 00:02:26

# Slop pronounced slæp
Any code you don't read

## Engineering the future of AI
[Small purple logo with the text BAML]

### 00:02:43

# Skirmish of Standards
casualty: 200M tokens lost

### Engineering the future of AI

[Hand-drawn illustration of two stick figures fighting with sticks; one has a ChatGPT logo for a head, the other has a red starburst for a head.]
[BAML logo]

### 00:03:18

# Engineering the future of AI

## Architecture.md

| Situation           | Rule

### 00:03:46

# BEP-050 Metrics and Tracing IDs

## Summary

Any function with a single non-deterministic component anywhere in its call stack becomes non-deterministic. Traditional testing ( **assert output == expected** ) breaks down — you need statistical evaluation. Today, evaluation of BAML functions is done entirely outside the language: separate eval scripts, YAML configs, dashboard rules. If you rename a function, the metric binding breaks silently.

This proposal makes metrics a **language-level concept** in BAML. It introduces:

- **$id** — an opt-in trace identity parameter that keeps return types clean, supports parent-child trace trees, and enables retroactive metric attachment
- **metric blocks** — co-located with the function they measure, type-checked against its signature, with a reactive DAG model where timing is derived from data dependencies
- **after blocks** on testsets (extending BEP-023) for multi-dimensional batch assertions

### 00:04:16

# Engineering the future of AI

## BEP List
- BEP-001-match
- BEP-002-universal-catch
- BEP-0

### 00:04:51

# Engineering the future of AI
2026-01-09

[Directed graph showing dependencies or components of a system, starting with `__project__` at the top and branching down through

### 00:05:20

# Engineering the future of AI

[Dependency graph showing interconnected modules and components of a software project, with many nodes labeled with names like `baml_compiler_ir`, `baml_

### 00:05:49

# Agent Tries BAML
## Runs
Engineering the future of AI
[Screenshot of a web application dashboard showing a list of agent "runs" with details like task, outcome, turns, cost

### 00:06:14

### Screenshot of BAML application: Agent Run Analysis
[Screenshot of a web application dashboard showing an agent's run results, with sections for "WHAT WENT WELL", "WHAT FAILED",

### 00:06:53

# AI Engineer World's Fair
## Presented by Microsoft
### Skill arena
Engineering the future of AI

[Screenshot of a web application showing a "Skill arena" dashboard with lists of skill variants

### 00:07:12

# Engineering the future of AI

## Skirmish of Standards
Architecture.md

## Battle of Design
beps.boundaryml.com

## Battle of Architecture
cargo stow

## Battle of Release
boundaryml.com/atb

[Stick figure illustration of two figures with different heads, facing each other in a dispute]
[Stick figure illustration of a person with a sword facing a flag with a bear logo]
[Stick figure illustration of a person with a bow and arrow aiming at a flag with a bear logo]
[Stick figure illustration of a person with a spear facing a flag with a bear logo]

### 00:07:47

# We're winning the battles...
## Engineering the future of AI
[A simple stick figure drawing with a smiling face and outstretched arms]

### 00:08:10

# TypeScript Design Non-Goals
> 3. Apply a sound or "provably correct" type system. Instead, strike a balance between correctness and productivity.

[Screenshot of a GitHub document titled "TypeScript Design Non-Goals"]

### 00:08:47

# Engineering the future of AI

```bash
vbv@vaibhavs-MacBook-Pro-3 ts-demo % bunx ts-node
> [2, 3

### 00:09:20

# Engineering the future of AI

```typescript
vbv@Vaibhavs-MacBook-Pro-3 ts-demo % bunx ts-node
> [2, 3].sort()
[ 2, 3 ]
> [2, 3, 11].sort()
[ 11, 2, 3 ]
> [2, 3, 11, 23].sort()
[ 11, 2, 23, 3 ]
> let x: string = 1;
<repl>.ts:7:5 - error TS2322: Type 'number' is not assignable to type 'string'.
7 let x: string = 1;
  ~
> let x: string = 1 as unknown as string;
undefined
>
```

### 00:09:42

# the most broken thing is
# the way we code is **different** now

## Engineering the future of AI

[A stick figure with a robot head]

### 00:10:18

if we aren't reading the code,
does the foundation even matter?
[BAML logo]

### 00:10:43

# Engineering the future of AI

```python
function improving_image(goal: string = "a founder building a progra
    let img: image = generate_image(goal);

### 00:11:21

# AI Engineer World's Fair

## BAML Playground - agent.baml

```baml
// Interactive REPL. Run with: baml run agent.main

//

### 00:11:41

# Engineering the future of AI

```typescript
function improving_image(goal: string = "a founder building a progra
    let img: image = generate_image(goal);