# Exploring Claude’s frontier — Lance Martin — session 2026-11-12T20:35:00.000Z → 2026-11-12T20:55:00.000Z _70 transcript lines · 39 slides_ ## Transcript Alright. It's very nice to be here. The clicker works. The computer works. I'm Lance from Anthropic. I know we've had some issues with the... With the prior keynotes, so we're set to go. I'm gonna tell two stories about getting the most out of Claude and exploring Claude's frontier capabilities. So the first is MultiAgent. This is a blog I wrote with the fable five two launch, and I wanna kinda tell The story of what happened here. So Fermat's last theorem is one of the more kind of famous, claims in mathematics. It stood and proven for around three hundred fifty years, and it's actually a very simple statement. Just a to the n plus b to the n equals c to to the n has no solution for any positive whole numbers greater than two. Now, this deceptively simple kind of statement has proven to be quite difficult to actually prove. In the mid nineties, Sir Andrew Wiles, did prove it. He put out a proof, and it took him Seven years of work is extremely painstaking. And since that time, a major effort and interest in mathematics has been formalizing that proof, for basically computationally checking every step. So proofs are kind of long chains of logic, and you actually can computationally check proofs using different systems, which I'll talk about, for example, using Lean. So this is just kind of a visual, kind of explaining the... There's a very interesting video showing the effort that he went through. It's an extremely painstaking process to, you know, establish this proof. And In September, we published that we used Claude to formalize it. This was like kind of first very important result that we kind of shared along these lines. And we used Lean to do it. So Lean allows you to computationally verify every step in a proof. Now doing this, though, is extremely painstaking. We start with a team of down a dozen agents, and this is kind of just showing a schematic of that team. Very early on, that team didn't work very effectively. We had to give it something called proof to me. Now, this is a kind of a platform for orchestrating mathematics proofs, and it helped the agents significantly manage the state of the project. Lean, you can think about as a harness. So basically, it provides the agents with a graph that they can query. So it's basically a lot of scaffolding around this problem statement. You can think of it as like a very domain specific harness. Okay? Now this is all setting up what we actually did. We reported. This was back in September. The formalization. Now, what I want to do for for five two is show, can we have a much larger team of agents, so going from around a dozen to a thousand, do this formalization with no domain specific harness, so without prove to me, and with no human steering with respect to the solution. So the report we talked about in September had both this domain specific harness proved to me and a lot of support from a researcher. This is testing whether or not a large team of Fable two agents can autonomously develop this this formalization. Themselves. So this is just giving you a sense for the harness. It looks kind of complicated, but it's actually pretty simple. Each of the thousand agents has its own sandbox with restricted network access. Most of this is directed on safety and security. So we have a thousand agents, each have their own sandbox. They can connect to two different things, Git and a SQL database. That's it. That's the only... Those are the core principles that we give this team. And we have real time monitoring, looking at every tool call for things like escapes. We also have offline monitoring, looking at runs of all the agents every ten minutes. This is kind of the setup. Thousand agent team, very simple primitives, just get in a database. The prompting is pretty simple. It establishes one lead agent, 999 helper agents. It gives them awareness of Git and the database, and the problem statement, that's really it. We want to see if this team could then do this formalization. What's interesting is this Now, the story of what the team did, the lead, very early on, five minutes into the run, basically created these database tables. So agents now are sufficiently advanced that they can actually build primitives that they need to solve hard problems. We didn't tell them to build these tables. These are the tables that the agent actually just kind of built. A table where the lead can announce things, assign roles to workers. Everyone... You know, agents can post notes, a subtask table, a table that indicates whether or not things are landed. So the lead just creates these tables. To orchestrate this team. And this is a visualization showing how the agents wrote to and read from these tables over the course of this run. Now red on the top is showing rights to the tables, green is showing reads. What's really interesting is you can see occasionally the lead right into the announce table. This kind of announces major global, like kind of, announcements about the problem. It also assigns workers, these helpers, different statements in this theorem as they proceed. Notes are where they communicate. And share, kind of, progress. Land is where they, kind of, actually write the Lean code, as part of the proof. And so the agents all autonomously, without specific prompting, use the database in this way to coordinate their work on this extremely hard, long horizon task. This ran for around thirty hours. And for particularly challenging parts of this proof, they actually developed their own abstractions. This is actually a task tree for this Lagrange Tunnel. This is one of the harder statements in the proof. And this is actually showing the different Tendencies of the tasks within this tree. And this is actually showing over time how, you know, the progress from, like, in progress to done in red. So the point of all this is really just to kind of highlight, agents can solve very long horizon problems without much scaffolding. This is showing kind of the overall, kind of, allocation of the agents across these different tasks over the course of this run. And, this kind of shows how the lead and the agents themselves kind of reallocate. Look at themselves and kind of reassign themselves to different parts of the problem as it progresses. And as these statements, there's 60 statements in this proof, are all solved. So, the agents using simple primitives, Git and the database, can coordinate themselves across this very hard problem to solve it progressively over time. That's really the key point. And you can see time progressing here. This, from one of our runs, this is one of the notes that the agents left, basically saying total freeze of the origin, effective now, magnificent work. 999 of you. So so basically, what is really interesting about this is we were able to show this formalization, which was expected to take many, many years. Back in September, we showed initially we could do it with 12... With, like, a dozen agents with a lot of scaffolding. This proved to be harnessed and a lot of poking and and and support from a researcher. This new result is showing, we provide a lot of structure in terms of safety and monitoring, but the agent team was able to Accomplish this task with very simple primitives, without a domain specific harness in this particular case. And agents are getting increasingly good at orchestrating themselves and knowing how to coordinate using these simple primitives for very challenging long horizon tasks. That's really the key observation, which is very interesting from this work. Now I want to shift gears and talk about memory, which is, entirely different when I talk about in terms of Multi Agent. But it's highlighting again another area we're seeing Claude gain significant capabilities. So previously, we Talked kind of about Claude's ability to orchestrate large teams to solve very long horizon problems. Memory is is a different capability. It's showing that Claude is increasingly capable to organize its context to help solve long horizon problems. So I published an NINJA blog on this fairly recently, and this is actually the lead line of the blog. I think this actually encapsulates the idea of the blog. And, like, one of the central ideas around memory, I think, is very interesting. And this is from the philosopher Kierkegaard. So life can only be understood backwards, but it must be lived forwards. This is a fundamental paradox of agent memory. Because agents write to memory, typically, as they go about a task. They encounter something weird or interesting, they write to memory. But memory needs to be a coherent map of an agent's knowledge that agents can look back on and reflect on. So when they're writing to memory, they're writing kind of myopically. They're looking greedily within a session saying, ah, this seems important for this task at hand. But memory needs to be something that's globally coherent and not redundant. So this is like a very tricky paradox of memory. We as humans have the same kind of challenge. We go about our day, we write to short term memories, believe that the hippocampus kind of is the place that that's orchestrated. At night, it's believed that the hippocampus replays to the neocortex, which is where long term knowledge is stored. So it's believed that actually this dreaming process for humans help consolidate the fragments, or like the most important aspects of our daily experience, into long term knowledge. So there's this decomposition between daily memory and kind of long term knowledge. And we found agents can benefit from the same decomposition. That's really the key point of the post. Now, here's some results kind of telling the story. So this is actually, an open source continued learning benchmark called CLbench. And this is showing that, different models, as you go from like SONNET up to like, OpenStack 5.5, get increasingly good at using memory. Now, this task is a very simple setup. It asks Claudius to solve 40 questions about a database. In between each question, it's a fresh session. And Clock can write to notes, like a memory directory, between each session. So it's kind of like a set of sequential questions. And Clock can write memories to help with future questions. Right? So what happens is, the dark bars are showing when you give Cloud the memory store, and then the hash is like without memory. Okay? Now, what's interesting is, with SONNET 4.6, giving SONNET memory, you can see in the dark blue, has very little benefit. Can't really use memory. And we looked into the results, and we Found SONNET doesn't, doesn't reliably know when to write to memory. It also doesn't really, naturally read from memory. Okay? It's also worth noting that these experiments are done with no specific memory prompting. It just tells the agent the directory is available. Use memory if you want. That's really it. So SONNET 4.6 doesn't natively use memory effectively. SONNET five does does indeed better, Opus, and so on. And so basically, the models get better and better with the same simple instruction at using memory. And really, the key idea here is that what happens is, with stronger models, they know intrinsically when to write to memory, and they kind of know intrinsically, when to read from memory, to retrieve information. Their memories are also better. They're like, they're highly verified statements or predictions about, kind of, the state of the database or what may happen next, versus earlier models tend to write, lower quality memories. And I'll show you a very fun example of this from a different game. This is from Pokemon. So this Is showing you Claude three point seven's memory directory from Pokemon versus Fable five. Here's the key point. The size of each box indicates the, the number of memory files in each of these directories. Okay? So when we have Claude Pay Pokemon, we give Claude a memory directory, just like I did in that prior benchmark. Right? Claude can write whatever it wants that directory while it plays the game. What's really interesting is, SONNET 3.7, we see it writes a huge amount to memory. And we At the memories. It's indiscriminate. It's basically writing down all sorts of experience. It creates this huge, unwieldy, just disgusting memory directory of things that are not really relevant or strategic for gameplay. And it proceeds much worse in the game than Fable five. Fable five's memory directory is much tighter. It's really strategic insights, tactical knowledge, things you can't just get from game state very easily. So what we see is better models are much... Are... Have much better instincts for salience and, like, what to write to memory. And you can see this reflected The size of the memory directories, it's much tighter. It's much more strategic and gameplay indeed with Fable five gets much further than SON three seven. So it gives another intuition around the fact that Claude is much better at writing to memory, knowing kind of what to write in salience, and what's salient, and, and also kind of using and reading from memory. So models are getting more and more, kind of effective at knowing salience, knowing what to write, and also reading. Now both of these represent what I would consider in my Initial framing of like writing to memory during experience. So basically, in both of these tasks, we let Claude write to memory while it plays this game. We let Claude write to memory while it does the CL benchmark. But as I mentioned, there's also the dreaming side of memory. Memories. This is allowing Claude to reflect over memories, to fix things. And I'll show some interesting examples of that. This is kind of a a framing of dreaming. And so, you can think about memory in terms of, there are trends Transcripts, you can think about as kind of the foundation of memory. So imagine, a memory system could very simply be just transcript search. Claude writes her transcript. As it goes about a task, future sessions can read from the transcript. Of course, transcripts are large. It's very unwieldy. We talk a lot about in the blog post the fact that transcripts often contain information that, you know, a very, very small amount of information in transcripts is actually salient in matters. So we also allow Claude to do things like writing memory notes, also I just talked about. As it goes, it can write notes to memory. But dreaming lets you do more. Dreaming lets you reflect over transcripts and these written notes and re represent them. Build member representations like wikis, as an example. Actually, promote certain memories to injected context and load it preemptively into every future session. So memory is this... Dreaming is this mechanism where you can kind of process existing transcripts or existing memory notes and build new things from them, including representations, injected contexts, and more. So some fun examples of this. This is from Pokemon Gameplay. Dreaming is an effective way to correct mistakes in memory. So as I talked about, when Claude is going about a task, it can write incorrect notes to memory. It makes a mistake. This happened, in this particular game, Claude wrote, incorrectly wrote the location, of a particular landmark in the game. Now what happened is, with this incorrectly written memory, Claude got trapped in a loop where basically it would mislocalize itself at this particular point in the game, fall down this trap door, and it would Basically send it back to the prior level. That's like the high level intuition. The details aren't that interesting or important. The key idea is a poisoned memory was injected and it's stuck there. Dreaming can rescue that. That's really the intuition on why dreaming is extremely powerful for reflecting over memories and correcting them. So, this is showing some trajectories of the game, where orange is memory and dreaming, up is good. So, basically, up represents game progress, and down represents, kind of, game regression. And what's interesting is Raw memory, basically, without dreaming, we can see crashes out. An incorrect memory is planted, and clock continues to basically regress or fall in the game when it hits that particular point. When dreaming, it rescues that mistake, clock proceeds to the next level. That's really the key observation. And this is a very general principle, that clock can use dreaming to correct mistakes in memory that are otherwise stuck there. It's a very important way you can kind of, kind of correct from errors. Now, this is another fun example of building representation. In memory. So there's been a lot of interest in wikis as kind of general, kind of ways that cloud can organize knowledge. This is a really fun example, from... We acquired a company, Stainless, back in May. And we had access to the company corpus, all its documents and Slack messages and so forth. And we anonymize things, and we built, kind of a data set from it, which allowed us to test dreaming. So what we did was, we created, kind of an email. Set of 25 questions about the company that can only be answered from this corpus. We allow Claude to build a wiki from this information to help it answer questions. What you see is, we compare a raw search tool, that's just like some kind of semantic search over this kind of documentation. We compare that as a baseline, which does reasonably well. You can see it's like 53%, just for search. We also test letting Quad build a a wiki over this knowledge. You can see as you push, you add More dreaming tokens, the wiki gets better and better and better, allowing Claude to answer questions. So Claude natively can kind of build up over time, a representation of this of this kind of corpus of knowledge through a wiki. And as you add more dreaming compute, the Wiki gets more and more useful. We think this is a really nice mental model to think about how agents can kind of build knowledge over time. They can be given access to kind of raw corpuses of texts, of transcripts, and they can kind of build these representations that allow them to search those tonic texts much more easily. Questions and to connect their knowledge in effective ways. We also find that adding the Wiki and the search tool together actually bumps further beyond just raw search. So it's kind of another intuition that, basically, Claude can also build representations, like wikis, in its kind of memory, in order to help it solve problems. And a final example, this is from a different benchmark, but, this is showing two different wikis that Claude built. So this is prior to dreaming and after dreaming. What we saw is when Claude was building wikis, in this particular case, there were some gaps. And, for example, this initial wiki had a bunch of placeholders for pages that didn't exist. This can happen when Claude's kind of building different memory representations. And without dreaming compute, there may not be opportunities to rescue or fix those errors. What we saw is when we ran dreaming, dreaming can look over Claude's existing representation, like for example, Wiki, and can fill in mistakes. So in this case, it fills in all of the missing gaps. It may create an index. And so Again, dreaming is a way that Cloud can reflect and kind of apply compute across its memory to improve it by correcting mistakes, like I talked about, by filling in gaps in its knowledge, like we see here, or for example, by creating new representations like wikis. Wikis. So with that, I, thank you all for listening. And, I... I'll be at the at the Anthrobiq booth. So please, come see me and... Happy to talk about any of this. ## Slides ### 00:00:38 # AI Engineer Europe 2027 ## ExCeL London ### 02.17 - 02.19 2027 Europe's flagship technical AI conference returns to ExCeL London for three days, February 17-19, 2027 — bringing the world's leading AI labs, platforms, and tools to Europe's builders. ai.engineer/europe [Image of an ornate building in London] [Image of Big Ben and Westminster Bridge] [Image of the London Eye over the Thames] ### 00:01:06 # Exploring Claude's Frontiers Lance Martin Member of Technical Staff [Diagram showing a curved line with several points, from which a network of smaller branching lines extends, resembling a tree or neural network] ### 00:01:38 # Fermat's Last Theorem A one-line claim that stood unproven for more than 350 years a^n + b^n = c^n has ### 00:02:10 # Fermat's Last Theorem A one-line claim that stood unproven for more than 350 years `a^n + b^n = c^n` has no solutions in positive whole numbers when n is greater ### 00:02:35 # Formalizing Fermat's Last Theorem > One way to check a proof's correctness is to ask a computer to do it. Proof assistants like Lean verify the logic of a proof algorithmically ### 00:03:07 # Claude's initial attempts Agents had some early success, then lost track of the project's state and stopped collaborating effectively. ### About **7%** of the non-boilerplate ### 00:03:42 # Exploring Claude's frontier ## Try with a larger Fable 5.2 agent team - 1000 agent team - Minimal harness - No steering [https://www.anthropic.com/](https://www.anthropic.com/) ### 00:04:06 # Exploring Claude's frontier ## ① Sandboxing - Own lightweight VM - No network access - No API keys or credentials - No direct path to other sandboxes ### 00:04:38 # Exploring Claude's frontier ## 1. Sandboxing - Own lightweight VM - No API keys or credentials - No network access - No direct path to other sandboxes ### 00:05:06 ## Exploring Claude's frontier This slide illustrates a workflow involving a "lead" and "helpers" interacting through five distinct stages. The workflow starts with **1 lead** at the top, interacting ### 00:05:38 # Hour 0.8 of the run **1,825** writes counted so far **41,706** reads counted so far ## How to read ### 00:06:10 # Hour 18.6 of the run **18,403** writes counted so far **241,954** reads counted so far ### 00:06:34 # Hour 28.4 of the run - **2,418** of 2,418 tasks written - **2,364** marked ### 00:07:07 # Hour 7.8 of the run ## 44 of the problem's 60 statements proved ### The 999 helpers - 262 ### 00:07:38 # Exploring Claude's frontier > "OTAL FREEZE OF origin/main, EFFECTIVE NOW. > ... Magnificent work, all 999 of you." ANTHROP\C ### 00:08:07 > "TOTAL FREEZE OF origin/main, EFFECTIVE NOW. > ... Magnificent work, all 999 of you." > > ANTHROPIC ### Exploring ### 00:08:39 # Memory ## Exploring Claude's frontier Lance Martin / Member of Technical Staff **ANTHROPIC** [Diagram of a spiral network of interconnected nodes, with a thick black line tracing a path ### 00:09:12 > "Life can only be understood backward, but it must be lived forward." > Søren Kierkegaard ANTHROPIC ### Exploring Claude's frontier Lance Martin / Member ### 00:09:39 > "Life can only be understood backward, but it must be lived forward." > Søren Kierkegaard ANTHROP\C ### Exploring Claude's frontier Lance Martin / ### 00:10:10 # Exploring Claude's frontier Lance Martin / Member of Technical Staff ANTHROPIC [https://www.anthropic.com/news/claude-3-5-sonnet](https://www.anthropic.com/news/claude-3-5-sonnet) [Bar chart comparing task scores for various Opus and Sonnet models on a CL-Bench database task, showing performance with a memory store versus a stateless baseline.] [Line chart showing accumulated reward over questions for various Opus and Sonnet models, comparing performance with a memory store (solid lines) versus a stateless baseline (dashed lines).] ### 00:10:34 # Exploring Claude's frontier ## Lance Martin / Member of Technical Staff ANTHROPIC ### CL-Bench database task - **Opus 5.5**: With memory store ** ### 00:11:06 # Exploring Claude's frontier ## CL-Bench database task ## Accumulated reward over questions Lance Martin / Member of Technical Staff ANTHROPIC Source [https://www.anthropic.com/news/claudes-frontier](https://www.anthropic.com/news/claudes-frontier) [Bar chart comparing task scores (mean reward) for Opus and Sonnet models with and without memory store] [Line chart showing accumulated reward over questions for Opus and Sonnet models with and without memory store] ### 00:11:38 # Exploring Claude's frontier ## CL-Bench database task - **Legend**: With memory store, Stateless baseline - **Opus 5.5**: Stateless 0.6 ### 00:12:08 # Exploring Claude's frontier ## Legend - Named like fresh starts or new "canonical" KBs - Alarm-named (e.g. sos, emergency, evac) - Other ### 00:12:39 # Exploring Claude's frontier Lance Martin / Member of Technical Staff ANTHROPIC - Named like fresh starts or new "canonical" KBs - Alarm-named (e. ### 00:13:10 # Exploring Claude's frontier ## Treemap Legend - Named like fresh starts or new "canonical" KBs - Alarm-named (e.g. sos, emergency, evac) ### 00:13:34 # Exploring Claude's frontier Lance Martin / Member of Technical Staff ANTHROPIC - Named like fresh starts or new "canonical" KBs - Alarm-named (e.g. ### 00:14:07 # Exploring Claude's frontier ## Dreaming - **Session** - **Injected context** - Memory loaded into every session - **Memory representations** ### 00:14:39 # Exploring Claude's frontier The slide presents a diagram illustrating a process called "Dreaming" that processes information from a "Session" to prepare for a "Next session". Within the " ### 00:15:07 # Correcting mistakes ## Raw memory Stored note is wrong: Agent heads for the next city. ## Memory + dreaming Offline pass corrected the note: Agent heads back to the ### 00:15:39 # Correcting mistakes ### Raw memory Stored note is wrong ### Memory + dreaming Offline pass corrected the note ANTHROPIC Source [Two side-by-side pixel art maps resembling a game, showing an agent at a cave entrance. In the "Raw memory" map, an arrow points right with the label "Heads for the next city". In the "Memory + dreaming" map, an arrow points left ### 00:16:07 # Exploring Claude's frontier ## Progress toward Cerulean (estimated tiles traversed) - **Legend**: - Memory + dreaming - Raw memory ### 00:16:39 # Building representations ## After 2 sessions Like a new employee's first days - 103 pages, 766 links - Example wiki - O Page at ### 00:17:07 # Building representations ### After 2 sessions Like a new employee's first days 103 pages, 766 links Example wiki O Page at 2 ### 00:17:39 # Building representations ### After 2 sessions Like a new employee's first days - 103 pages, 766 links - Example wiki - O Page at ### 00:18:07 # Building representations ## After 2 sessions Like a new employee's first days - 103 pages, 766 links - Example wiki - O Page at ### 00:18:39 # Filling in gaps ## Before the second pass: 38 entries, 225 links - **escrow-agreement**: Nine links from six pages, but no page by ### 00:19:11 # Filling in gaps ## Before the second pass: 38 entries, 225 links - **escrow-agreement**: Nine links from six pages, but no page by this ### 00:19:38 # AI Engineer Code Summit Presented by Google DeepMind ## LIVE TRANSLATION // AVAILABLE Scan the QR code to access real-time translation. [https://events.google.com/codesummit/translation](https://events.google.com/codesummit/translation) [QR code for live translation service]