# I love you so jevving much — Diogo Almeida — session 2026-11-11T22:05:00.000Z → 2026-11-11T22:23:00.000Z _71 transcript lines · 33 slides_ ## Transcript Cool. Give me a sec to make sure this is working. Heck yeah. I'm Diogo Almeida, and this talk is I love you so Jeving much. Glass Animals reference. We're not just weird. Like I said, I'm Diogo Almeida. I'm known as the Jev guy, sometimes Jevi Pinkman. Also co created ChatGPT r LHF and instruct to GPT. But I assume that for the crowd today, y'all know who I am, at least mostly. I have a bunch of AIE specific stuff I want to get to. But I like going into my favorite topics in AI. First one is what the fuck is actually going on, and where is all the automation? If you've heard me say this before, I'm sorry, but it's such an important thing to beat into the field because people still do not get it, and I need to yell this until everyone really understands what the fuck. Is going on. And if you're an AI, you should have a really good answer about why AI can do so many awesome things and yet still be so useless for a bunch of other ones. My relationship to ChatGPT is somewhat unique. One of the few people who hates on ChatGPT who made it. I don't hate ChatGPT, to be clear. I love using agents. I love using coding agents. I love chatbots. They're just quite limited. And in particular, I think it's a little tragic how much they've bifurcated the field into overpromise under Deliver. Really, really terrible stuff for the world, in my opinion. And I like to talk about, like, the two cults in AI and how we can resolve the cognitive dissonance that everyone, I think, should be having unless they have a very good answer. There's Cult one, which is the cult of EAs. They look at all the benchmarks and they say, not only are LMs surpassing human level on basically every benchmark, They have been for two years. We've been announcing AGI three months away for two years now. Yeah. Even Like, the o three days. Do people remember that? No one even care... Cares about that model anymore. And it it was... That that was, like, peak internal to OpenAI AGI hype. And on the other hand, you look at the non benchmarks, and you look at the reality, and you see the rest of the world, especially if you guys go outside and leave San Francisco. A lot of the world is, like, really unchanged. Outside and leave San Francisco, a lot of the world is, like, really unchanged despite AI being so fucking cool. And I think AI is transformative. It's a rare point of view to actually think AI is cool and transformative and also, like, underwhelming. The world right now. And the tricky part is this is the same field, and we are all seeing the same facts. So what gives? Why is AI jagged, so to speak? Why can it be so good at some things and so not good at other things? And what is the Occam's razor explanation for that? I will be blasting through my slides fast. I'm sorry. Maybe I could slow down for people taking pictures. But, I'm noticing that, this is gonna go fast is all I'm saying. So I think that we as A Field should have an Occam's razor explanation for why are there some things that are just not just good, they are unbelievably good, superhuman, like Millennium Price problems in math. But on the other hand, they're not just bad. They're so useless. They can't even be used to touch accounting. That sounds way easier than Millennium Prize problems in math. Right? I could do the latter. I definitely could not do the former. And I was a mathlete. So what gives here? What is the reason for this? And I normally love this as an interactive thing. But I will instead just tell you my answers. Sorry for the power dynamic. Everything on the left side is intrinsically a human in the loop task. Like, deep research, I could not do better than deep research. The models are so fucking good at that. But if, gun to my head or gun to your head, you had to pick between me and deep research and the answer had to be correct, not well written, you'd probably pick me every time. And that's the distinction between optimizing for automation, which tends to be a lot easier and a lot simpler, yet AI keeps failing at this. Every year, there It's like an infamous failure of AI failing to even automate drive thrus. And that should be shocking to people. And my claim is, actually, for these tasks, pleasing a human is nothing to do with doing the task. What you want to do is basically be an abstraction away. You want the decision to be made. And you want to get back into doing automation. And code is the language of automation. Therefore, you need to integrate with code. This is a lot, but I'll summarize it again. I get the irony that I co created RLHF and I do RLHF is part of the problem. And RLHF, in particular, is why all AI today does require a human in the loop with an asterisk unless the task is very trivially verifiable and can run on its own, but still requires a human in the loop to verify it afterwards. It is because we literally put it in. That is what the HF is in RLHF. And my claim is that there's a better way. My summary of the state is 100% of LMS today are optimized for assistance. You might talk about RLVR, but the RLVR models are Bad to interact with if they've not been r h f ed. The result is l m's become too good to be true at assistance. They are just so fucking good, like, you should never pick me over chat g p t for your chat g p t queries. But for actually doing, like, you know, run-in the background cognitive work, you should not trust LMs at all because they have not been optimized for that, which is why they're so bad at the tasks of automation. And I would also guess that automation is a lot more valuable than assistance because you can do way cooler, deeper I'm going to get into this later. There's going to be so much stuff that we can do. And we are just scratching the surface of that right now. And that's the tragic thing about AI the state of AI for me. Let's talk about the Gevolution. I can't tell if this is appropriate, but it's too late. Oh, god. Don't take pictures of this one. Cool. Oh, I'm getting a thumbs up. Yeah. Excellent. So instead of doing RLHF, we just have to do something else. We need to think from first principles. How do we optimize for machine native Hi. You guys hopefully know the interface of Jev. And that is what we are doing. More important than optimizing for its interface, we are, rethinking how AI should be done because this is what we did again for RHF. For Sutton's bitter lesson, I'm going to loosely paraphrase to make my point easier to say, is that scaling, and compute matters more than algorithms. This is why people become like scaling pill Jaronis. And I would argue that there's an even bitter lesson than that, which is data always Matters more than compute. We are learning that quite a bit in the field now, but it has been obvious since the beginning of time. I'll tell you when... The... This is the one you should be photographing. I would argue that the the bitterest lesson is doing the right task. And doing the right task is a weird thing to think about, especially for ML people, because they like algorithms. They want number goes down or up or whichever is a good direction. They don't want to think about the real world. But this is always how you make gigantic gains in in performance. This is how the big breakthroughs happen. 100% of the time, it Comes from this. And this is true for classical RL games. This is why Sutton had his bitter lesson, which is data is derived by compute when you're doing rollouts. Like, there's this... Why would you need off policy data when playing chess or something like that? And doing the right task is defined by the rules. So actually, when you're doing RL, you don't even need to think about the right task. For pre training, we actually, as a field, had to do a lot of thinking about the right task. I don't know if people remember the BERT days with, masked language modeling. But it took While it was very unclear to figure out that next token prediction was super fucking cool, not because it was better at masked language modeling. You know, the objectives on their own aren't special. It's about what you can do with them afterwards. And it turns out next token prediction is incredible at generating strings, which is what these pre trained models were so good at. So no level of mass language modeling, even if you had today's compute and algorithms, would make it as interesting as GPT three was because it was just doing a different task. And data, you know, the internet is obviously a giant. Bottleneck. Now we're, like, you know, burning books or whatever in order to get more data. But data obviously matters a lot more than computing. People keep rediscovering this and figuring this out. It's extra true for making LLMs useful. Before RLHF even existed, the data for RLHF didn't exist. And you could like it was night and day. I'll just show you the next slide. This is probably the relevant part. This was the work in the RLHF paper. And the high level was that even with the dumbest algorithms in the world, even with a GPT two sized Model, you got something like GPT nine level performance. Like, real GPT nine, not today's fake GPT numbers. The numbers used to mean 100 x scaling and effective compute for pre training. And they've given up on that because it just can't scale anymore. And still, even with that, it would take something like GPT seven to nine to match GPT to the model from, like, twenty eighteen or whatever it was at the task of instruction following. That is kind of wild because you think big LMs are so important for this. And they are, but that Is not the most important thing compared to even the gigantic jump of doing the right task. And spoilers, this is what we did for Jeff. Right? We made this gigantic leap in the field that no one realized could be coming because we were optimizing for a task and very few people think about the right task. This is why Jev has the jeve nissie quoi. That's a really smart pun, I think. A little bit of a pity laugh, but thank you. You get what you optimize for. I'm looking for a better word than reliable. So if you guys have suggestions, let me know. But there's there's this reliability of intelligence that's really missing in LLMs. I often call it you get Einstein 95% of the time, but mister Bean 5% of the time. And if you can't tell which is which, how the fuck are you gonna automate anything? And instead instead of, like, focusing on, like, sometimes occasional smarts that you can then verify, what if you focused on extreme, extreme narrow focused reliability so you can get that smarts every time? And this doesn't show On benchmarks. It is the reason why people love Jev. And this is also, I realize in hindsight, we obviously didn't prepare for a chat GPT moment for developers. But it happened because people in the AI space just overpromise, underdeliver so fucking much that having a model that actually don't do that was a surprise. And it is not the speed, cost, or even interface. This is something that you know, my team tells me I shouldn't be telling people so that we can have a long term advantage, and I just care too much. Like, it's reliable intelligence. Make sick models, please, even We, you know, if it doesn't involve our models. We see that JEV is like a foundation for smart software. I'm gonna skip all over this. But I think software is gonna be so fucking back. I think the narrative of Sassapocalypse is obviously not gonna happen. But I think that even calling it a sassapalooza is like an understatement. And I'll get into that a bit. And you guys ideally are the perfect crowd for this. I'm I'm so excited for this part because that's the part that's new to me. I want to make a distinction between smart software and just in time software. The the crazy part is AI is so transformative. And yet somehow, despite AI being so awesome, software has essentially unchanged in the last decade except being cheaper to write. Isn't that weird? Isn't it weird that the best we can do in software is having a chatbot to the side that sometimes interacts with the software but cannot take expensive decisions for you? That is a weird thing if AI was really that transformative. And I think the problem is we're focusing too much on coding agents and not enough of Putting the intelligence into the inner loop, which I'm gonna get more into a bit. And people are jebbing. There's been tons of successes. We don't need to flex this. There's so many cool things. But my I... A AIE secret is going to be that the field isn't jebbing hard enough. And this is what Swyx said, that I can make a call to action for you guys that's unique. And hopefully, I have enough time for this. I think the world still has LLM Stockholm Syndrome. We are still thinking from a really LLM filled place, which is, I think, tragic. You know? Like, I I love the soft For saying the tools you use change the way you think. And I think that it's changed the way we think way too much. And we're not really rethinking software like I think the field needs. I think we could just be doing so much more. Hopefully, I can try to do my best for my remaining time to convince you guys of that. So let's jiv to the future. An analogy I love bear with me is about LEDs. Before, there was a blue LED. There was red and green LEDs. These were really limited in capability. They were used for, like, simple indicators and, you know, like, clocks and stuff like that. And when the blue LED was made, discovered... I actually don't really know how things outside of computer science work. But when it existed, I guess, it enabled not just 50% more use cases, which is what you'd expect about changing colors from two to three. It enabled so many more. Like, now LEDs could be used for white light. It could be used for, like, the displays that we all use right now, and it created a revolution in lighting. And it's it's it's unfathom How much bigger the capabilities of LEDs were, just adding a new primitive that composes together with an old primitive. So you can see where I'm going here, maybe. And I think it's a similar story for software. Software is so fucking incredible. Ignore AI for the last half decade. But software has done so many insanely cool things. We've done a lot of stuff. You know, like, do I love using Sass? Not really. But does it solve problems? Abso fucking lutely. And it's How much we've done with three fundamental logic gates. You know, like, we are doing this with, like, simple binary operations that can do things like greater than, less than, equal. It's kind of nuts how much software has been built from such simple primitives. And my claim is that what we are doing with AI now when we're making our intelligence machine native, this is conceptually adding a fourth semantic logic aid. What if we can program with common sense, with intention, with human understanding? Like We still need regexes with, like, crazy things, or can we just say, like, we want a Gmail email address or a real looking one? There's so much that we can do that we are going to need to push the frontiers of cheap, smart, fast in order to get it done. But I think there's just going to be, like, so many things here. And I think just like the blue LED, it's going to be way bigger than just increasing the representational ability of software by 33% because it's adding effectively another kind of logic gate. And this sci fi is the stuff that I think will be impossible to Just like who could predict how powerful the internet would have been? Because it's much more grassroots and decentralized. And it requires people to be empowered to do insane stuff. I call this software research. That's not a joke. But I think that there's going to be an explosion of what software can do. And we will want people to be exploring those boundaries of software, not necessarily because it's valuable, but because they can. Because we have, like, new frontiers. To explore. And this is what the early Internet was like. Should all of those websites existed? Probably not. But, like, it was very cool, and we found a lot of awesome stuff from that. And I think we we, you know, we have another opportunity for that. So we should not just keep living in this little fish tank that we are the victims of OpenAI and Anthropic on whatever they ship, and we just have to be like LLM wrappers. So I'm going to pitch software research as a direction here. The theme is going to be real software engineering, not just vibe coding. It's going to be, like, the kind of stuff that everyone has known if you've been a software engineer. Decomposition, breaking tasks down. We need to think about stages, ideally ones that can be verified so we have real encapsulation. And we need to actually do state management instead of assuming everything is global state within a kV cache. And the theme is just going to be real fucking engineering by people who care a lot about the end result. It doesn't have to be valuable. Ideally, it's sick as fuck. And we should be pushing the limits. I'm going to go into a couple of my ## Slides ### 00:00:38 ### AI Engineer Code Summit Presented by Google DeepMind // AVAILABLE # LIVE TRANSLATION Scan the QR code to access real-time translation. [QR code for real-time translation service] ### 00:01:37 # AI Engineer Europe 2027 ## ExCeL London Europe's flagship technical AI conference returns to ExCeL London for three days, **February 17-19, 2027** — bringing the world's leading AI labs, platforms, and tools to Europe's builders. **02.17 - 02.19 2027** 3 DAYS - LONDON, UK [ai.engineer/europe](ai.engineer/europe) [Black and white photo of a grand building in London] [Black and white photo of the London Eye] [Black and white photo of Big Ben and the Houses of Parliament] ### 00:02:39 # AI Engineer Code Summit ## Keynote ### Diogo Almeida / Co-founder Presented by TypeSafe AI Presented by Google DeepMind [G2i logo] ### 00:03:10 # WTF IS GOING ON IN AI ## AND WHERE IS ALL THE AUTOMATION [Meme of John Travolta from Pulp Fiction looking confused] ### 00:03:35 # MY RELATIONSHIP TO CHATGPT - **MADE CHATGPT**: OpenAI - **HATES ON CHATGPT**: Yann LeCun, Gary Marcus - **ME**: ### 00:04:11 # Massive output uptick due to agentic AI. Complete flat adoption. ## Corporate needs you to find the differences between this picture and this picture. They're the same **AI**re ### 00:04:35 Corporate needs you to find the differences between this picture and this picture. They're the same **AI**re. ### Screenshot of chart: Measurements above 10 hours are ### 00:05:07 # In other words: today's AI frontier is “jagged” > "We suggest that the capabilities of AI create a “**jagged technological frontier**” where some tasks are easily done by AI, while others, **though seemingly similar in difficulty level**, are outside the current capability of AI." - Dell'Acqua, McFowland, Mollick et al. 2023 [Diagram illustrating a "jagged technological frontier" where a solid, irregular blue line (the jagged frontier) represents the current capability of AI, and a dashed arc represents tasks of seemingly equal difficulty. Points are marked for a "task inside the frontier" (green), a "task on the frontier" (yellow), and a "task outside the frontier" (red).] ### 00:05:39 # The Sane View of AI ## Too good to be true - Instruction Following - ChatGPT - Copilots - Coding agents / Claude Code - NotebookLM ### 00:06:06 # The Sane View of AI ## Too good to be true - Instruction Following - ChatGPT - Copilots - Coding agents / Claude Code - NotebookLM - DeepResearch - Voice agents - Customer Service w/o decisions - Slop / Spam ## Too bad to be useful - Customer Service - Drive Thrus - Accounting / taxes - QA - Insurance - Data entry - Anything requiring reliability ### Keynote Diogo Almeida / Co-founder TypeSafe AI [https://ai.engineer/codesummit](https://ai.engineer/codesummit) [Red stamp graphic with text "HUMAN IN THE LOOP" and "ASSISTANCE"] [Red stamp graphic with text "MACHINE ONLY IDEALLY" and "AUTOMATION"] ### 00:06:37 # The Sane View of AI ## Too good to be true - Instruction Following - ChatGPT - Copilots - Coding agents / Claude Code - NotebookLM - DeepResearch - Voice agents - Customer Service w/o decisions - Slop / Spam ## Too bad to be useful - Customer Service - Drive Thrus - Accounting / taxes - QA - Insurance - Data entry - Anything requiring reliability [Red stamp graphic with "HUMAN IN THE LOOP ASSISTANCE" text] [Red stamp graphic with "MACHINE ONLY IDEALLY AUTOMATION" text] ### 00:07:08 # Why do all LLMs require a human-in-the-loop ## Keynote Diogo Almeida / Co-founder TypeSafe AI [Meme showing a crying figure labeled ### 00:07:39 # WTF is going on in AI? 1. 100% of LLMs are optimized for assistance (RLHF) 2. LLMs are too good to be true at ### 00:08:08 # VIVA LA **JEV**OLUCION [Stylized poster of a man in a sombrero, pink furry coat, and pink mustache, holding a rifle] ### 00:08:38 # Sutton's Bitter Lesson [Pink triangle diagram divided into two horizontal sections: "algorithms" at the top and "compute" at the bottom.] ### 00:09:07 # The Bitterest Lesson - doing the right task - data - compute - algorithms Keynote by Diogo Almeida / Co-founder, TypeSafe AI [Pyramid diagram illustrating ### 00:09:39 # This it true for classical RL games - algorithms - compute - data - doing the right task ## Keynote Diogo Almeida / Co-founder TypeSafe AI [https://typesafe.ai](https://typesafe.ai) [Pink pyramid diagram showing a hierarchy from algorithms at the top to doing the right task at the bottom, with annotations "derived by compute" pointing to "data" and "defined by rules" pointing to "doing the right task"] [QR code linking to typesafe.ai] ### 00:10:12 # This it true for pre-training - algorithms - compute - data - doing the right task ### Keynote Diogo Almeida / Co-founder TypeSafe AI [Pink ### 00:10:39 # This is extra true for making LLMs useful - **Pyramid Layers (from top to bottom):** - algorithms - compute - data ( ### 00:11:11 # AI Engineer Code Summit ## Keynote Diogo Almeida / Co-founder TypeSafe AI Annotated from figure 31 from the InstructGPT paper. [Line graph showing performance (y-axis, unlabeled) versus Model size (x-axis: 1.3B, 6B, 175B) for four models: PPO-ptx, PPO, SFT, and GPT. The PPO-ptx, PPO, and SFT models show higher performance, annotated as "Right task + GPT-2". The GPT model shows lower performance, annotated as "Wrong task + GPT-3".] ### 00:11:35 Right task + GPT-2 Wrong task + GPT-3 Annotated from figure 31 from the InstructGPT paper [Line graph showing performance versus Model size (1.3B, 6B, 175B). Four lines represent models: PPO-ptx, PPO, SFT, and GPT. Annotations indicate "Right task + GPT-2" for higher performance at smaller model sizes and "Wrong task + GPT-3" for lower performance at larger model sizes.] ### 00:12:06 ## Jev's secret: reliable intelligence - you get what you optimize for - this is why benchmarks don't matter - this is why people love the model over copycats - this is why there was a "ChatGPT for developers" moment - NOT: speed/cost/interface ### Keynote Diogo Almeida / Co-founder, TypeSafe AI [Dark grey t-shirt with the text "jev ne sais quois" and "typesafe"] [QR code] ### 00:12:39 # Jev's secret: reliable intelligence - you get what you optimize for - this is why benchmarks don't matter - this is why people love the model over copycats - ### 00:13:08 # What is Jev for the world? - AI that is - made for automation - programmable - made to run in the background - for devs! ### 00:13:40 ## Traditional Software ## Just-in-time Software ## Smart Software Keynote Diogo Almeida / Co-founder TypeSafe AI [Three-panel diagram comparing software approaches: ### 00:14:08 # AI Engineer Code Summit Presented by Google DeepMind ## PEOPLE ARE JEVVING - 30+% of fortune 500 using jev - 1T ### 00:14:39 # AIE secret 🤫 - the field isn't jevving hard enough - people still have LLM stockholm syndrome - a lot of ideas are just replacing LL ### 00:15:10 ### Before Limited LEDs - Indicators, simple uses ### Blue LED The missing piece ### Full-color displays RGB light makes every color ### Efficient white lighting Blue light + phosphor = bright, efficient white Typesafe AI [Diagram showing red and green LEDs leading to a blue LED, which then branches into two applications: full-color displays and efficient white lighting.] [Image of three overlapping circles (red, green, blue) next to a smartphone and a TV displaying a colorful image.] [Image of blue light hitting a yellow phosphor to create white light, next to a light bulb and a street light illuminating a house.] ### 00:15:38 ### The Blue LED: The Missing Piece #### Before * Limited LEDs * Indicators, simple uses #### Blue LED * The missing piece #### Full-color displays * RGB light makes every color #### Efficient white lighting * Blue light + phosphor = bright, efficient white Typesafe AI [Diagram showing red and green LEDs evolving into a blue LED, which then branches into two applications.] [Image illustrating full-color displays with RGB color circles, a smartphone, and a TV screen.] [Image illustrating efficient white lighting with blue light hitting a phosphor to create white light, shown as a light bulb and a street light illuminating a house.] ### 00:16:12 # AI Engineer Code Summit Presented by Google DeepMind ## Infinitely replicable lego blocks - → software ### Keynote Diogo Almeida / Co-founder TypeSafe AI ### 00:16:36 AI Engineer Code Summit Presented by Google DeepMind # Infinitely replicable lego blocks - -> software ### Keynote Diogo Almeida / Co-founder TypeSafe AI [Diagram ### 00:17:06 # AI Engineer Code Summit Presented by Google DeepMind ## Keynote Diogo Almeida / Co-founder TypeSafe AI SOFTWARE RESEARCH [Meme of Dr. Evil from Austin Powers making a hand gesture, with the text "SOFTWARE RESEARCH"] [QR code linking to https://live.aiengineer.org/ for live translation] ### 00:17:40 # SOFTWARE RESEARCH ### Keynote Diogo Almeida / Co-founder, TypeSafe AI [https://typesafe.ai/](https://typesafe.ai/) [Meme of