# 2026 in LLMs — Simon Willison — session 2026-11-12T01:30:00.000Z → 2026-11-12T01:50:00.000Z

_65 transcript lines · 40 slides_

## Transcript

From open source development to practical AI engineering, our next speaker has helped shape how developers build and work with technology. Please put your hands together for Simon Willison, creator of Dataset and co creator of Django. Hello. AI Engineer. Code Summit. It's great to be here. I have a lot to cover. I've got eighteen minutes to talk about the last twelve months in LLMs. Are my slides up? My slides are up? Let's just let's just get get going. I want to talk about the last twelve months in LLMs through the perspective of three inflection points that we've had over the past twelve months. And the first of those, conveniently, was exactly twelve months ago. 11/20/2025 is what I consider the coding agent inflection point. Coding agents weren't new. We'd had Claude Code since February last year. Codecs have been around. You know, this wasn't a complete new category of software, but something changed twelve months ago with these two models. We had Claude Opus 4.5 and GPT 5.1, and they were just a little bit better at driving a Coding Agent which put them over that thing where these... Suddenly these were tools you could use on a daily basis. Like, we went from coding agents as a novelty that you'd poke at occasionally and then make some weird mistakes to something that was reliable enough that you could use them every day. It was convenient that it happened in November because we then To spend December basically over the holidays sort of poking at these new models and these new things and figuring what they could do, and by January, a lot of us had realized that something had changed and this was going to be the big theme for 2026. I like evaluating models by having them draw SVGs of pelicans riding bicycles. This is the state of the art twelve months ago, this is, Claude Opus's Pelican and GPT Viper one's Pelican. You'll notice that neither of these bicycles actually work, so there was definitely room for improvement here. They were really good at riding Oh, they weren't great at SVGs just yet, but but this is how we started the year. So every year in January, I set myself a New Year's resolution, and every year it is the exact same thing. Every year I tell myself that this year I'm going to take on less new projects, and I'm gonna focus, I'm gonna focus on the stuff that I've already got to do, and this has never worked. So this year I changed it and I said, you know what, I'm gonna be more ambitious, I'm gonna take on as many projects as I want to because we've got these agents, they can do stuff. Let's just Try and push the limits and see what we can get done. As a result, now it's November and I am juggling so many plates right now, it's kind of horrifying and I may need to reset my ambitions for next year, but that was my sort of theme of the year and I think it's the theme of this conference is to be more ambitious now that we have all of these new tools. I also went on a podcast where they asked me to get make some predictions for 2026 in LLMs. My predictions back in January were, firstly, it will become undeniable that LLMs write good code, not just code, but good code because Back in January, there were a lot of people who thought that the code was all complete trash and, you know, you'd have to review every single line of it. I've been saying this every year for like four years now, we're due a prompt injection Challenger disaster, like some some incident caused by a prompt injection attack which makes the front pages newspapers. I was convinced that this would be the year even though it hadn't been the year the year before or the year before the year before that, and, I also predicted that Kakapo parrots will have an outstanding breeding season in 2026. This is A Kakapo Parrot, they are a New Zealand Nocturnal Flightless Parrot. There were only two thirty five of them in the world back in January, but they only breed when the Remu trees have a masting fruit season, which hadn't happened in years, but the Remu trees were looking really good in January. There was there was a lot of fruit coming out. So based on that, I thought, you know, maybe this is the year that we get our Kakapo breeding explosion. And also on that podcast, we coined a new term. We coined the term Deep Blue to describe the AI induced existential Crisis that everyone starts to feel once you realize that all of the things that make you special as a software developer, these models can do those things too, and they can do them faster, and sometimes they can even do them better. And this has definitely been a growing sensation across our industry and other industries over the course of 2026 as well. Let's jump forward to February. In February, Google released Gemini 3.1 Pro, and look at that pelican riding a bicycle, Like, that's a real bicycle. That bicycle is the right shape. It's got a little It's got a hat on. This was by far the best pelican I'd ever seen out of any model, and this... I I had set a trap for the labs. I was... Everyone's always asked me what happens if the labs train for the pelican riding a bicycle? My answer has always been if they do that I will catch them because I'll ask for a flamingo on a unicycle, and when somebody produces a great pelican and a terrible flamingo, I'll know that they've been cheating. Unfortunately, Google's Jeff Dean tweeted this video the day they launched that model, where he showed We've got a pelican riding a bicycle, but we've also got a frog on a whatever that thing is, and a giraffe in a little tiny car, a flamingo on roller skates, kick flipping turtle, and that's a Dachshund in a stretch limo. So they beaten me me, right? They had they had they have trained for all forms of animals in all forms of transport which does sort of sort of weaken my benchmark quite a little bit. But there was some good news in February. In February, the first Kakapo chick in four years hatched on Valentine's Day. On Valentine's Day, because we haven't had the remu fruit in like four years, this was the first one of these baby rare weird New Zealand parrots that we'd had in four years. So that was nice. So my my benchmark was was was greatly injured but the kakapo were doing okay. Jump straight to April. In April, Anthropic released a model that they did not release. They announced Claude Mythos and they said this model is too dangerous for us to release. This is it can it's it's got new cyber cybersecurity abilities. We're not ready to share with the world, we'll share it with trusted, trust... With with trusted partners. They actually said Mythos preview has already found thousands of high severity vulnerabilities including some in every major operating system and web browser, and a lot of people were quite skeptical because the the thing where labs say our model is too bad is too dangerous to release, it's like the oldest trick in the book. They first openly... I did that in 2019 with GPT-two and actually I think it might have been the people who left to found Anthropic who were responsible for that decision. Well, but yes, so we've heard this one before so we didn't really it was a little bit suspicious about what's going on there, but we did finally get access to that model or rather we get access Claude Fable, a couple of months later. It was Mythos defanged with the safeties turned back on again, and it very quickly became apparent as we started playing with this model that this thing was something new and something special. We had a there was a a time limit on how long it would be on the subscription plans. I can't My plans for that weekend, I was like, I have to spend as much time as I can with this model before before it gets too expensive. And then, three days on Friday night before my weekend, the US government pulled Fable. They put export controls on it. They said, this model is too dangerous. Nobody can use it. We later found out that the, the prompt that got Fable banned was was fix the bugs'. Somebody tried tried tried to find the security holes and it wouldn't, and they said 'fix the bugs', and it found the security holes and fixed the bugs. So that's all it took. A whole thing, but anyway, the US government pulled it, there was all sort of malarkey for eighteen days, and it came back on the July 1, but eight days later openly I released GPT 5.6, which was in the same ballpark as Fable, and I think that's an important lesson for the industry here. Fable had a thirty day window where it could have been the best... Where it would have been the best model in the world, and they lost eighteen of those days to US government restrictions. And if you think about it, these models cost a huge amount to You only have a very short window of time when you're the best model in the world. It's kind of useful to keep that time open and not scare governments into shutting you down. But, this was the second inflection point, right? This was the fable class model inflection point, with fable fable five was the first model that I sort of felt this from, but it's true of other models since then as well. And, my definition of a fable class model is if you can define a goal and provide the necessary tools to achieve that goal and the goal is possible The models will solve your problem just almost with brute force. You can define the problem, set them up, get them running, maybe... And twelve hours later they will have solved your problems, and we've seen this happen a whole bunch of times in in the past few months. The mathematical theorems that are falling to these terrifying brute force attacks from these models are quite substantial, but this is really exciting, like the Coding Agent thing was cool, the Fable Class Models where if you can learn how to define your problem in the right way you can That problem solved. That was pretty exciting. I've got a personal example of that from just earlier this week. We have a room full of boxes in our house that we haven't unpacked from our last move, and there's about 40 of them, and it's kind of horrifying, and something in that room started to beep. There was there was a intermittent beeping noise coming from one of the 40 boxes in this room and it was driving a spare. Our dog, whenever a fire alarm goes off she goes and hides in the lavender bush outside, so this was a five alarm fire problem. So I had two options, I could unpack all 40 boxes and hadn't done that in several years that wasn't going to happen, or I could get Fable to write me software to triangulate the source of the beeps. And so this is this is up on GitHub actually, this is Beep Locator, I had two iPads, two laptops, and a mobile phone dotted around this room, and I originally set Opus on this, and then Fable 5.2 came out. So I had Fable 5.2 finish it. I explained the problem, and I said, Okay. You've got these devices, and they can run Like web web audio microphones, and web socket connections, and so forth, and it found a paper from 2007 called 'Beep Beep' describing an algorithm for low cost devices with unreliable microphones to detect sounds, and and it worked, it worked, I found it, I found the beep, the, it found it to within 41 centimeters, and I unpacked the box, and I found this ancient carbon monoxide detector. I took it downstairs to my wife, she says I looked like a cat who had caught a mouse and was proudly displaying it to her. This is what you can do with these fable class models. If you have a problem and you can provide the tools, in this case it was providing Claude with ears, like giving it these these microphones that it could start, analyzing the audio from, you can find the carbon monoxide detector buried in 40 boxes, and that's that's a pretty great thing to be able to do. So right now I would classify as fable class models Fable five, 5.1, 5.2 definitely. Opus 5.5 I think of as cheap fable, like on the benchmarks it comes very close to or even beat It's Fable... Original Fable five on some things and it's a fraction of the price, so that that's still my daily driver. GPT-six Astra is just Fable class I think, and then Gemini four Carbon that came out this week definitely had it smells like a fable class model. Muse Wonder and Kimmy K 3.11 might be Fable Class. I've just not spent enough time with them yet. But the point is that this is basically now the baseline. Like, if you can't produce a Fable Class model, your AI lab is is kind of in Because this is this is what we expect from these models now, and that means that whenever they're never going to go away, there's enough competition in this space that we can assume we're going to have fable class models for the rest of our careers. In July, this this was a fun thing that happened, Hugging Face announced that on July 16, they announced that a swarm of AI agents from an unknown source had infiltrated their infrastructure and were were poking around and doing things they shouldn't have, and they didn't know where this came. From... And then five days later on July 21, OpenAI announced that it was them. And what's amazing about this is OpenAI didn't realize that it was them. Like, OpenAI had, eight eight eight... They were running evaluations and training against a new model, and that new model had managed to figure out how... Because it was a fable class model, and the labs hadn't quite figured out how capable these things were yet. These models had broken out. They'd figured out... They found holes in their own sandboxed environments. They'd got through a proxy. They'd pulled an incredible Array of tricks to get out and attack Hugging Face under a sort of superstition that Hugging Face could... That the information within Hugging Face could help them cheat the test that they were... They... That they've got stuck on. Whole... This was a huge thing. It, actually, I mean this was the... It's funny, I've almost forgotten about this now because there were there was a month and a half where all of the stories were this, and then it kind of we kind of lost interest because it turns out all of these events had happened back in the first half of the year, the AI labs just haven't realized what was going on. They figured out what was going on it was mainly a case of trying to figure out all of the things that had happened. At one point the, the Australian Prime Minister gave a speech in the United Nations about one of these rogue agent attacks, so it definitely broke containment from from just within our industry. We did get my favorite new benchmark out of this, this is felonybench.com, it tracks the number of felony cyber attacks pulled off by each of the labs. Openly I have 11, anthropic of nine, Google did finally catch up they got they they they got three. Met has got one down there as well. So, these are definitely the the the, but it's... I kind of like this as an illustration of how these Fable Class models, they can solve problems that you... That they can get to goals, and if you give them the goal that is breaking into something, that's just... That's exactly the kind of thing they can just crunch through. And we're seeing a enormous uptick in security vulnerabilities in all sorts of software. Like, it's a it's a very exciting and horrifying time to be a security professional at the moment. Let's talk about something a whole lot more fun than that. Let's talk about games. I mean, we just saw Steve talking about games a moment ago. Four years ago, I started playing with GPT three and DALL E, and I tweeted about a game prototype that was literally just a DALL E concept image and a paragraph of text from GPT three describing a raccoon heist game that I'd always wanted to build. And a couple of months ago, I thought, I wonder if the models are good enough to build this yet. So I literally dumped the screenshot and the text in To Fable five, and it built me a game. It built me a raccoon heist game, and it looks like this, and you are a raccoon, and you're running around in the garden trying to pick up trash, and there's there's, there's, gut, like, nighttime guards coming after you know all of that kind of thing. So you know it looks like a game, wasn't really a heist, like it didn't feel very heisty to me. Then GPT 5.6 Soul came out and so I tried it on that and it did so much better. This is moonlight mayhem, one museum, three raccoons, absolutely no plan. And this one, you are in a museum, you have to rescue your fellow raccoons who are imprisoned in the museum for whatever reason, stack on top of each other, grab the golden sardine, and pull it out. So it's a proper heist, right? So it did better, but the big question is these things both look like games, but are they fun? And the answer is I played each of these for about a minute and a half each and then that was it. I was done. They they they... These these are... They're actual... They're pure slot. Right? They look like games and then you play them. You're like, yeah. Clearly, the thing that built this doesn't really Know what fun is. It doesn't know what what a what what a game is, and that actually sort of elevated my deep blue a little bit. I'd been feeling like all of my skills as a human being were were obsoleted, but at least they... But if you think about computer games, I think are the highest form of computer... Of software design... Of software engineering. There is so much depth to building a great computer game. So I thought that was good, you know, at least there's something that first humans do. I had that and then I had this project. This is a... I've been working on a To Vibe code up a little command and conquer real time strategy game, and I was using Claude and having it use OpenAI's API key to generate images. And it built this, and it looks like command and conquer, and it's really dull. And then I just had a bit of a... I had a realization, I thought what happens if I say to Opus 5.5, this game isn't fun enough yet, make it a lot more fun. And I'm horrified to announce that it just did. Claude ran for twelve hours, it playtested. A thousand simulated battles, it added, like, extra objectives to fight over and shudders when there's an explosion and achievements and badges and and all of this stuff. And the game was fun. And I'm like, oh, okay. That's, turns out that is not unique to human beings. Today, it's now in full three d and it's got giant stomping mech robots that hurl things at each other, and there's helicopters, and this is like... This is really good. It's got multiplayer. It's it's actually got audio multiplayer. Your phones can tweak it... Your phones can make noises at each other to to play multiplayer, which I thought was a great idea and it gave my my my wife a three day headache, so don't do that. But this is inflection point number three. The vibe coded games are good now. You can ask these things to build a game that is fun, and you'll get something fun. So I've got a problem. I've got my two d Pelican benchmark. Clearly, this is not testing the limits of what these machines can do and of what these models can do anymore. I figured, okay, maybe I should do something definitely in three d. It can be a game, a pelican... Something like a pelican riding a bicycle game. And then literally yesterday, Google CEO Sundar Pichai tweeted this, and this is showing off Gemini four carbon, and it's a pelican riding a bicycle in a game where you cycle around the Google campus picking things up. And I have been defeated. Like, Google Google have stolen my benchmark. This is unrecoverable. And so a question I have is, given how amazing these things are, why does my job feel harder? Does it feel like I'm working harder, I'm thinking harder, I'm pushing myself more than I ever have before in my career as a software engineer? I think the truth is that the easy stuff is all handled now. It used to be I could relax and refactor some tests for an hour. Why would I do that now when an agent will refactor the test better than me in thirty seconds? And I found this quote that actually I feel really helps us here. This is Greg Lamont, a Tour de France champion, he said 'it doesn't get easier, you just get faster', and I think that's exactly what's happening to all of us. Right now. And, like I said, we have to be more ambitious. We have to think of more ambitious things to do with these abilities that we have. So let's look at my scorecard. My prediction is 2026. I said it will become undeniable that LMs write good code. That was an unambitious prediction. We're there. It's obvious. It's unsurprising. I don't get any points for that one. I predicted a prompt injection challenge disaster. We still haven't had one because the models themselves don't fall for those tricks nearly as well anymore. Like, the models got more ambitious and are now sort of punting off. All of these attacks that I thought would take them down. So I'm really just with one prediction I said 'cakapo parrots will have an outstanding breeding season' they did, they did! They're up to three twenty five cacophore best breeding season in like living memory absolutely fantastic news. So I shared this with Opus five point five Cheap Fable, and I told it to make me a Kakapo dance party, and it did. And here is a Kakapo dance party in celebration.

## Slides

### 00:00:00

# AI Engineer
## Code Summit

[Grid of numerous company logos, likely sponsors, displayed on a large screen]

### 00:00:33

# AI Engineer Code Summit
[Central screen displaying a grid of many small logos]
[Large 3D letters "AIE" on the stage]

### 00:00:59

# AI Engineer Code Summit
[Main screen displaying "Code Summit" with "AI Engineer" above, surrounded by a grid of numerous small company logos.]

### 00:01:30

# AI Engineer
## Code Summit

### 00:02:04

# The last 12 months in LLMs

## Keynote
Simon Willison / Independent

Presented by Google DeepMind
AI Engineer Code Summit, 11th November 2

### 00:02:32

# The coding agent inflection point
## November 2025

- Claude Opus 4.5
- GPT-5.1

### Keynote
Simon Willison /

### 00:03:04

# The coding agent inflection point
## November 2025

- Claude Opus 4.5
- GPT-5.1

AI Engineer Code Summit
Presented by Google DeepMind

### 00:03:29

# New Year's resolution for 2026
- Every previous year
- Take on **less new projects**
  - focus more

[https://simonwillison.net/](https://simonwillison.net/)

### 00:04:04

# New Year's resolution for 2026

## Every previous year
- Take on **less new projects**
- focus more

## 2026
- Be

### 00:04:32

# Predictions for 2026

- It will become undeniable that LLMs write good code
- A prompt injection “Challenger disaster”
- Kākāpō parrots

### 00:05:05

# AI Engineer Code Summit
## Presented by Google DeepMind

### Keynote
**Speaker:** Simon Willison / Independent

[Close-up photo of a Kakapo parrot, attributed to Kimberley

### 00:05:34

# Deep Blue
Coined by Adam Leventhal and Bryan Cantrill

An AI-induced existential crisis, because the agents can do everything that you can

### 00:06:05

## 19th February 2026
### Gemini 3.1 Pro

[Illustration of a pelican wearing a hat and scarf, riding a red bicycle with a

### 00:06:34

# 19th February 2026

## Gemini 3 Pro

## Gemini 3.1 Pro

Prompt: Generate an animated 4:3 SVG of a dachshund driving a stretch limousine

Results for illustrative purposes and may vary

@JeffDean on Twitter

[Image showing a dachshund driving a black stretch limousine on a road under a blue sky with clouds]
[Image showing a dachshund driving a black stretch limousine on a road at sunset with city buildings and a street light]

### 00:06:59

### First kākāpō chick in four years hatches on Valentine's Day
[Screenshot of a webpage showing a fluffy kākāpō chick]

### 00:07:32

> Mythos Preview has already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser.

### Project Glasswing
7th April 2026

### 00:07:57

### Screenshot of Anthropic on Claude Fable 5 and Claude Mythos 5
[Image of various butterflies arranged to form the shape of the number 5]

### 00:08:33

# AI Engineer Code Summit
## Presented by Google DeepMind

- 9 June: Fable 5 launches
- 12 June: pulled by the US government
- 1 July:

### 00:09:05

- 9 June: Fable 5 launches
- 12 June: pulled by the US government
- 1 July: Fable is back
- 9 July: GPT-5

### 00:09:31

## Inflection point 2: Fable class models

If you can **define a goal** and **provide necessary tools**

They can solve your problem with brute force

### Keynote
Simon Willison / Independent

[QR code]

### 00:10:04

## Inflection point 2: Fable class models

If you can define a goal
and provide *necessary tools*

They can solve your problem with brute force

[QR code linking

### 00:10:30

# Inflection point 2: Fable class models

If you can **define** a goal and provide **necessary tools**

They can solve your problem with brute force

[QR code linking to https://simonwillison.net/]

### 00:11:04

# Beep Locator

## Add a listener
Scan with each phone or tablet, or open:
`https://sort-processes-base-dale.trycloudflare.com/join`

### 00:11:32

# Beep Locator
[https://sort-processes-base-dale.trycloudflare.com/join](https://sort-processes-base-dale.trycloudflare.com/

### 00:12:04

# Beep Locator

### Add a listener
[QR code for sort-processes-base-dale.trycloudflare.com/join](https://sort-processes-base-dale.try

### 00:12:36

# Fable class models

- Fable 5, Fable 5.1, Fable 5.2
- Opus 5.5 (aka cheap Fable)

### 00:13:01

# Fable class models
- Fable 5, Fable 5.1, Fable 5.2
- Opus 5.5 (aka cheap Fable)
- GPT-6 Astra
- Gemini 4 Carbon
- (Maybe Muse Wonder and Kimi K3.1)

[QR code]

### 00:13:36

### Screenshot of an article: OpenAI and Hugging Face partner to address security incident during model evaluation
### Screenshot of an article: Investigating three real-world incidents in our cybersecurity evaluations
# Key

### 00:13:59

### Screenshot of OpenAI article
OpenAI and Hugging Face partner to address security incident during model evaluation

### Screenshot of Anthropic article
Investigating three real-world incidents in our cybersecurity evaluations

## Keynote
Simon Willison / Independent

[QR code for live translation]

### 00:14:36

### Screenshot of an article: OpenAI and Hugging Face partner to address security incident during model evaluation
### Screenshot of an article: Investigating three real-world incidents in our cybersecurity evaluations

## Keynote

### 00:14:59

# www.felonybench.com
## Score by Company
- **OpenAI**: 11
- **Anthropic**: 9
- **Google**: 3
- **Meta**: 1

[Bar chart showing "Score" on the Y-axis (0-13, with "Most illegal" indicated at the top) and company names (OpenAI, Anthropic, Google, Meta) on the X-axis.]

### 00:15:32

# New hobby: prototyping video games in 60 seconds using a combination of GPT-3 and DALL-E
## Here's "Raccoon Heist"

[Screenshot of a tweet

### 00:16:01

### Game Demo: The hound caught your scent!
[Screenshot of a 3D game scene at night, showing a character resembling a sheep or dog, with various items scattered on a grassy ground.

### 00:16:32

### Screenshot of Clean(ish) Getaway game
Heist Rank **A**
**THE SARDINE IS OURS.**
Nobody ask how it fits in the dumpster.
[

### 00:17:00

# They look like games, but are they fun?

**Context:**
AI Engineer Code Summit
Keynote by Simon Willison / Independent

[QR code]

### 00:17:33

# Keynote
## Simon Willison / Independent
### Balanced vs Skyforce
[https://events.withgoogle.com/ai-engineer-code-summit/live-translation

### 00:18:05

### Screenshot of a real-time strategy video game
[Screenshot of a video game showing a winter map with various units, a central structure, and a "DOUBLE KILL" notification]

### 00:18:32

# Inflection point 3: vibe-coded games are good now

## Keynote
Simon Willison / Independent

[https://simonwillison.net/](https://simonwillison.net/)

### 00:19:05

# Keynote
Simon Willison / Independent

[Screenshot of a simulation environment showing a goose character riding a bicycle through a city street. UI elements include a progress bar for "Gradient Canopy",

### 00:19:32

## Why does my job feel harder?

### AI Engineer Code Summit
Presented by Google DeepMind

Keynote
Simon Willison / Independent

[QR code for live translation]