# Closing the Distance: Agentic map-reduce for large-scale coding tasks โ€” Yannis Psorakis โ€” session 2026-09-24T13:30:00.000Z โ†’ 2026-09-24T14:00:00.000Z _37 transcript lines ยท 26 slides_ ## Transcript It's going to be competing for attention very, very variable, in diverse type of context. And number three, what is effectively are stopping criteria? Okay. Again, to make this a strong argument as possible, one will say, Yeah, of course, we're not gonna solve those big problems. Okay, in a kind of like in a single session, there will be some notion of agent orchestration. Okay? There will be some notion of, okay, we're gonna have a kind of like a, an the agent agent outer loop. We're gonna break down the problem into different tasks. Okay? We're gonna define I need dug how those things need to be completed. And then we're going to have some notion of inner loop on the agent. We have to build all that stuff. Okay? Because obviously normal, nobody is gonna one shot prompt. Hey, kind of like, please fix the security vulnerabilities. Absolutely. My argument is today is to present to you a framework. Okay? How we think about the problem that gives the best, both in terms of outcome performance and cost. And we make This, okay, as a first class notion in our product. I touched upon this before, kind of like single agent reasoning. I'm not gonna label too much on this point. Okay? Both in terms of kind of like time spent searching, context, pollution, and so on and so forth. Single agent problem does not solve very, very large code base wide tasks. How we approach this? Okay. So, we propose, okay, a variation. Or an or an augmentation of the, map reduce framework, an idea from, distributed systems. How many people here have worked with, map reduce? Awesome. So super, super quick example. Mapreduce introduced by Google was a way to solve to perform analysis on like a really, really large data sets that they are distributed, that they live on a distributed file system. Okay, so you Oh, so the classic example that we give, hey, we have a huge textual corpus on the distributed file system, different files, and we need to do some lexicographic analysis. I'll keep it simple. Count the number of words on each book. Okay, they will be a series of map functions. These are simple stateless as in pure functions that they kind of like would count each book. They can run-in parallel, and there's a reducer step that Consolidates the results in a very similar vein with adaptation. We want to port this into the agentic coding world. Okay, where instead of like huge data sets, consider huge code bases. Okay, an overview of how this thing would work. Okay. Firstly, there are two steps that we introduced at the beginning. There is a main agent that creates a plan. For investigation number three, for each of those candidate areas, we spin a series of parallel sub agents where they look these areas and only these areas, and they perform in the analysis. This is the map step. And then there is a reduced step. The main agent, again, considers the results, the combined results, the distilled results from all the different sub agents, okay, and performs Some reconciliation, some consolidation analysis. This this can be duplication, prioritization, or some more richer analysis on the combined result. Let's kind of like double click a bit on those Concepts. Okay. And I'm going to use, I'm going to use security. Okay. I was going to like a motivating example. Okay. I have this large code base. And what is the security kind of like, the security quality security? Level of this code base. Okay, I will provide the threat model. I will provide any additional context from the code base. Okay, what the pre work step is going to do is okay. We're gonna write does determine a deterministic search function that will identify areas of the code base of interest in the context of security. You folks can imagine API boundaries, where tokens are being minted and be passed Found, SQL and, code entry points and so on and so forth. The outcome of that is to produce basically those deterministic selectors they're gonna run, and they're gonna produce like a series of batches. Okay, or a series of shards of the code base where we can then run each one of those sub agents. Okay, those are pages. Then they have a very nice clean and bonded context, which they can perform the investigations and reports. Build these are the most critical, and there is a ranking of them. Number three, like this is crucial again in the context of codebase wide tasks, presence of chained vulnerabilities. Remember, individual agents may tell you that their individual narrow part of the code base has some low, critical defining. But what the reducer step can do is can assess combinations of those low vulnerability, low criticality finding. Can they produce something much more serious? And again, with the right harness, okay? Devin can run. Okay. Can actually test. Okay. And try to simulate those attack paths. And to do it kind of like a little bit more visually, we treat the tend to treat a code basis as basically a graph. Okay? All there's all those references between, the different parts of the code base. Selectors look for off areas, look for API entry points, data SQL entry points. And then from those selectors that they have run cheaply and deterministic on the code base, we deploy, okay, individual devins, individual agents that perform security analysis for this particular narrow bounded path of the code base. And then on the reduced step, Devon will produce something like this. Okay, where it will sort of like number one triage number two, the duplicate number. Three run and test and try to recreate those vulnerabilities, including kind of like false positives as well. So, yeah, mention about reproducing because obviously, at the end of the day, you don't want to ship kind of like tickets because you we know whenever we have a release, you know, we run security vulnerability from tools that are like tons and tons of things to churn through. We want to be shipping and like legit PRs against a known vulnerability. So recreating is extremely, extremely important. Now we're pushing the PRS and just want to comment. Okay, a little bit on kind of like the relationship between cost and performance. Okay, of course, kind of like you can run very kind of like expensive agents. You can run them against really large code bases. Okay, what is extremely important, as I mentioned earlier, is to think hard and have the best possible. Ways, okay, to mitigate the cost of searching. Searching is a huge problem in large code basis and context pollution and context overflow. This is solved by them. Devon and the main agent are in the agentic MapReduce, writing those deterministic selectors and creating lots and lots of banded context, which end up being much cheaper and end up having a better recall profile rather than, compared to competing models. Okay. And we've run this one fifty on the data sets 50 vulnerabilities from kind of like a GitHub, and they run across like a multitude of languages, Golang, Python, Java, and so on and so forth. Another thing that we should think about, I talked a lot about security, agentic map reduce in the context of vulnerability detection and remediation. Think about it about maybe it can be a generalized concept across all different Code base wide workloads. Okay. Dead code removal. I mentioned a lot extracting common common logic into some centralized library from a very large code base, dead code detection and so on and so forth. Okay? And this is something that again, we from our research, we have kind of like moved it to security and we elevating it into a first class feature, in our product. We are, as I mentioned, kind of like earlier. We are, applied ai research lab. We were doing work on areas like agentic map reduce building our own coding models. So two point o recent to model harnesses like dev infusion, agentic infrastructure. Please check out our research blog. And if you have any questions, this is my email. Looking forward to your comments and see you around in. The conference. Thank you very much. Thank you so much, Yanis. Let's give it up for Yanis once more, please. Alright. Okay. So we all deserve a break right now. I know that, it's been a long day for everybody, but we have a great line of of speakers still. Okay? So we're gonna be back here at 04:30, please, and, enjoy your break. Alright. See you in a ## Slides ### 00:00:38 # Challenges with codebase-wide tasks ### 01 Finding the work Repeated searching and backtracking consume the budget available for investigation. ### 02 Maintaining context Discoveries from different parts of the repository compete for attention. ### 03 Knowing when to stop The agent needs an explicit account of what it has examined and what remains. ### 00:00:59 ## Challenges with codebase-wide tasks ### Finding the work Repeated searching and backtracking consume the budget available for investigation. ### Maintaining context Discoveries from different parts of the repository compete for attention. ### Knowing when to stop The agent needs an explicit account of what it has examined and what remains. ### 00:01:36 - searching and backtracking - the budget available for - n. ### 00:02:05 # Single-agent reasoning doesn't scale to the entire codebase ## 01 FINDING THE WORK Over half the agent's effort goes to hunting code, not analyzing it - Tool-use turns: **56.2%** - Main-agent tokens: **46.5%** ## 02 CONTEXT BOTTLENECK Agent reliability falls as context grows ## 03 COVERAGE BOUNDARY Search agents terminate while constraints remain unresolved - DR-Tulu: **56.2%** - WebExplorer: **46.5%** - TongyiDR: **52.1%** Coverage, accuracy, and cost can't come from one agent loop Need a specialized architecture: - 01 Finding the work - 02 Context bottleneck - 03 Coverage boundary - Deterministic selection - Bounded parallel shards - Finite queue, exhausted [Horizontal bar chart showing percentages for tool-use turns and main-agent tokens, split between "Reading and searching" and "Editing, testing, and other actions"] [Line graph showing agent reliability decreasing as context (in tokens) increases] [Horizontal bar chart showing percentages of "Underverified answers" and "Fully verified answers" for DR-Tulu, WebExplorer, and TongyiDR] ### 00:02:40 # We propose an augmented MapReduce framework Tackle a large job by splitting it into smaller pieces that can run in parallel; the map step processes each piece independently, and the reduce step combines those outputs into an overall result. - **01 Plan** - ### 00:03:04 ## We propose an augmented MapReduce framework Tackle a large job by splitting it into smaller pieces that can run in parallel; the map step processes each piece independently, and the reduce step combines those outputs into an overall result. 1. **Plan** AGENT Study the repository and author relevance selectors. 2. **Shard** DETERMINISTIC Run the selectors and group matches into bounded batches. 3. **Map** SUB-AGENTS Investigate each batch and produce structured findings. 4. **Reduce** AGENT Reconcile findings and synthesize the overall result. [Diagram showing a four-step linear process: Plan, Shard, Map, Reduce, each with a brief description] ### 00:03:36 ## We propose an augmented MapReduce framework Tackle a large job by splitting it into smaller pieces that can run in parallel; the map step processes each piece independently, and the reduce step combines those outputs into an overall result. 1. **Plan AGENT** Study the repository and author relevance selectors. 2. **Shard DETERMINISTIC** Run the selectors and group matches into bounded batches. 3. **Map SUB-AGENTS** Investigate each batch and produce structured findings. 4. **Reduce AGENT** Reconcile findings and synthesize the overall result. [Four-step process diagram showing Plan, Shard, Map, and Reduce stages] ### 00:04:06 ## THE ARCHITECTURE ### We propose an augmented MapReduce framework Tackle a large job by splitting it into smaller pieces that can run in parallel; the map step processes each piece independently, and the reduce step combines those outputs into an overall result. 1. **Plan** AGENT - Study the repository and author relevance selectors. 2. **Shard** DETERMINISTIC - Run the selectors and group matches into bounded batches. 3. **Map** SUB-AGENTS - Investigate each batch and produce structured findings. 4. **Reduce** AGENT - Reconcile findings and synthesize the overall result. [Flow diagram illustrating a four-step augmented MapReduce framework: Plan, Shard, Map, and Reduce.] ### 00:04:33 # We propose an augmented MapReduce framework Tackle a large job by splitting it into smaller pieces that can run in parallel; the map step processes each piece independently, and the reduce step combines those outputs into an overall result. 1. **Plan** (AGENT): Study the repository and author relevance selectors. 2. **Shard** (DETERMINISTIC): Run the selectors and group matches into bounded batches. 3. **Map** (SUB-AGENTS): Investigate each batch and produce structured findings. 4. **Reduce** (AGENT): Reconcile findings and synthesize the overall result. [Flow diagram illustrating a four-step process: Plan, Shard, Map, and Reduce] ### 00:05:04 ### THE ARCHITECTURE # We propose an augmented MapReduce framework Tackle a large job by splitting it into smaller pieces that can run in parallel; the map step processes each piece independently, and the reduce step combines those outputs into an overall result. 1. **Plan** AGENT Study the repository and author relevance selectors. 2. **Shard** DETERMINISTIC Run the selectors and group matches into bounded batches. 3. **Map** SUB-AGENTS Investigate each batch and produce structured findings. 4. **Reduce** AGENT Reconcile findings and synthesize the overall result. [Diagram showing a four-step process: Plan, Shard, Map, Reduce] ### 00:05:35 - Author selection rules for routes, authentication wrappers, the data layer, and deserialization sinks. - Each ... records ... the rule that selected it, and ... su...nce. ### 00:06:01 # Pre-work: plan and shard - Plan (Agent) - Shard (Deterministic) - Map (Agents) - Reduce (Audit) e.g. Devin Security Swarm builds an editable threat model from the repository's architecture and conventions. **PLAN** `AGENT` Author selection rules for routes, authentication wrappers, the data layer, and deserialization sinks. **SHARD** `DETERMINISTIC CODE` Run the rules across the repository and group matching candidates into bounded batches. Each match records its location, the rule that selected it, and supporting evidence. Completeness depends on selector recall; the rules can be inspected, tested, and refined. [Diagram showing a four-stage process flow: Plan, Shard, Map, Reduce] ### 00:06:37 ## Reduce: **holistic reasoning** across the findings A reducer combines structured worker outputs into a coherent, prioritized result. ### Reconcile Deduplicate overlapping findings and reconcile local conclusions. ### Prioritize Assess findings together, attribute ownership, and triage their severity. ### Connect Identify relationships across batches that form a larger attack path. > **EXAMPLE FROM SECURITY SWARM** > An unauthenticated ID leak can combine with an ID-gated execution vulnerability to create an unauthenticated attack path. [Horizontal process flow diagram showing four steps: Plan, Shard, Map, and Reduce, with Reduce highlighted as the current step.] ### 00:07:04 ## Reduce: *holistic reasoning across the findings* A reducer combines structured worker outputs into a coherent, prioritized result. ### Reconcile Deduplicate overlapping findings and reconcile local conclusions. ### Prioritize Assess findings together, attribute ownership, and triage their severity. ### Connect Identify relationships across batches that form a larger attack path. > EXAMPLE FROM SECURITY SWARM > An unauthenticated ID leak can combine with an ID-gated execution vulnerability to create an unauthenticated attack path. [Process flow diagram with four steps: Plan, Shard, Map, and Reduce, with Reduce highlighted as the current step] ### 00:07:39 # Reduce: holistic reasoning across the findings A reducer combines structured worker outputs into a coherent, prioritized result. ### Reconcile Deduplicate overlapping findings and reconcile local conclusions. ### Prioritize Assess findings together, attribute ownership, and triage their severity. ### Connect Identify relationships across batches that form a larger attack path. > EXAMPLE FROM SECURITY SHARD > An unauthenticated ID leak can combine with an ID-gated execution vulnerability to create an unauthenticated attack ### 00:08:04 # Devins are launched in parallel from the points of interest. [Network diagram showing interconnected nodes labeled 'AUTH', 'API', 'DATA', 'SERVICES', 'UI', with some nodes highlighted as "points of interest" (red circles) and others with a 'Q' icon.] ### 00:08:32 ### Devin attempts to reproduce each **finding** to weed out false positives. - SQL injection in user lookup - Missing auth check on admin API - SSRF in webhook fetcher - Path traversal in file uploader [Screenshot of a list of security findings or issues, each with a status indicator and a brief description] ### 00:09:07 # Results on verifiable CVEs SOTA results on both recall and cost; [cognition.com/blog/introducing-deving-security-swarm](https://cognition.com ### 00:09:33 # Results on verifiable CVEs SOTA results on both recall and cost; [cognition.com/blog/introducing-devin-security-swarm](cognition.com/blog/introducing-devin-security-swarm) | Harness | Recall | $/Run | | :------------- | :----- | :-------- | | Devin Security | 72% | $90.23 | | Claude Security| 68% | $131.87 | | Codex Security | 48% | $118.20 | | Cursor Security| 26% | $4.60 | ## 72% recall on 50 real-world vulnerabilities ## 30% lower cost per finding Than nearest comparable alternative [Bar chart comparing recall percentages and cost per run for different security harnesses] ### 00:10:13 # Results on verifiable CVEs SOTA results on both recall and cost; [cognition.com/blog/introducing-devin-security-swarm](https://cognition.com/blog/introducing-deving-security-swarm) | Harness | Recall | $/Run | |---|---|---| | Devin Security | 72% | $90 ### 00:10:36 ## Performance ## Database qu ### 00:11:06 # Check out more of our research! https://cognition.com/research We are an applied AI research lab; our work spans areas from coding models (SWE 2.0), model harnesses (Devin Fusion), enterprise agent infrastructure and new AI-coding form factors yannis@cognition.ai # Thank you! ### 00:11:37 # Check out more of our research! https://cognition.com/research We are an applied AI research lab; our work spans areas from coding models (SWE 2.0), model harnesses (Devin Fusion), enterprise agent infrastructure and new AI-coding form factors yannis@cognition.ai ### 00:12:00 # AI Engineer ## PARIS ### 00:12:36 # AI Engineer Paris ## Organized by - Mistral ## Platinum - NVIDIA ## Gold - Cockroach Labs - FACTORY - SENTRY - Sonar ## Silver - CLOUDFLARE - Daytona - langfuse by ClickHouse - Modal - neo4j - Qdrant ## Bronze - CodeRabbit - cominty - descope - Latitude - MAKORA - FIMO - stripe - ZenMLabs ## Supporting partners - Backblaze - Notion - SIERRA ### 00:25:06 # AI Engineer PARIS ## Organized by Mistral ### Platinum - NVIDIA ### Gold - Cockroach Labs - FACTORY - SENTRY - Sonar ### Silver - CLOUDFLARE - Daytona - langfuse by ClickHouse - Modal - neo4j - Qdrant ### Bronze - CodeRabbit - cominty - descope - Latitude - MAKORA - FIMO - stripe - ZenMLabs ### Supporting partners - Backblaze - Notion - SIERRA [Logos of AI Engineer Paris and Mistral] [Logos of NVIDIA, Cockroach Labs, FACTORY, SENTRY, Sonar, CLOUDFLARE, Daytona, langfuse, Modal, neo4j, Qdrant, CodeRabbit, cominty, descope, Latitude, MAKORA, FIMO, stripe, ZenMLabs, Backblaze, Notion, SIERRA]