# MC Outro — Corey Quinn — session 2026-11-11T18:30:00.000Z → 2026-11-11T18:40:00.000Z _29 transcript lines · 19 slides_ ## Transcript I'm Ari Azafus, founder of Intrinsic AI. This talk is for people who build agents and run their own inference stack. Or plan to. And if that's you, you associate typically the kv cache, the burden butter of, modern inference engines usually as a throughput optimization. The keys and values computed per token kept so the model doesn't compute them again. Today, I'll like to show KvCash in a different light. The way agents... Agent... AI agent application could go wrong, keep growing. Right? Problem injection, jailbreaks, broken tool calls, sensitive tool calls, sensitive exposures, hidden backdoors, and drifts, and so forth. Right? So the way right now we typically guard against against these numerous ways that an AI agent application could go wrong is by guarding the edge by perhaps passing a judge model that checks the prompts, the inputs, and the response outputs against your policy. Alright? Or you build a classifier for each case. But here's a different light. The model turns every input into a computation before it acts, before it decodes, before it generates the response, before that. Response becomes another tool call. The kv cache is your engines record of that computation that modern inference engine for free are using it and reading it. And therefore, on this talk, I would like to suggest as as a compliment to the way you're already using your own guardrails to use this data and integrated with your kv cache infrastructure. I refer to this as a kv first security signals and basically three reasons that matters. Right? So the data is already there. You pick the gems from the pile your inferences stack would already was throwing away. It covers the guardrails blind spots because whatever slips past those guardrails, it still becomes computation. And it stays in plate, so it's perfect for data privacy application for cases where you do not have the luxury to read the inputs and outputs. And the recipe is pretty straightforward. Right? It has three steps. You learn a baseline for your AI workload, for your For AI inference traffic, how your KB page typically look like, and then find the KB pages that are rarer against your baseline, typically your anomalies. And then you can even classify and explain those. So first, let's start with where in this stack this integration could happen. The interesting thing about the kv cache is that in a way, existence of those had made the inference in a way stateful. Right? So for example, things Like kv cache management layers such as LMCache stores them from GPU down to a remote storage. Or LMD and NVIDIA dynamo route each new request towards the worker that already holds perhaps that cached kv page. So there are two integration points you you you can integrate those anomaly detection or those classifiers into. Either you can implement it directly inside of your inference engine via the LMSG line and so forth. The beauty of this approach Is that it's real time. And and the the the alert that it would perhaps generate could ride along your future tool calls or your future policies so that you can harden basically your policies or guardrails by just generating the computational anomalies that you observe within your inference engines. And, you know, perhaps you can integrate it with as a as a way of threat discovery instance. So you can integrate with URL and cache. So Reads the stored KB pages and find things that don't belong with the rest. Right? So now I wanna show you some numbers and and just a very simple hello world use case if if this is actually working and this is a valid approach. And for that, let me use with a use case that has no text to filter at all, a voice agent. So here's a case study of a voice agent. In this case, it was a gemo four e four b. Our baseline was 72 ordinary clips. Spoke in four voices. We're looking at 16 token pages. Obviously, this granular... Granularity could change. But here, we just, use this this, shorter pages so we can have more data in the baseline. And then on the test, we have different voices and one clean voice and the other one has hidden instruction within it. And as you can... As as it is shown here, not only you can detect these cases just using simple anomaly detection, but also you would know which part Of, which page, which part of your request had this anomalous event. We even went further, and we trained a model that uses the KB pages as features, and we didn't try to explain it or tag it with things that are useful. In this case, although it wasn't even trained on audio, it's able to generate correct information about personal information, home address, location data, travel information, destination. So Now back to text agents. In this case, it was a coin $314,000,000,000, served live, unveiled, aluminum, and cash. The one on the left shows you an idea of how attacks and normal traffic would typically separate from each other. And then the one on the right, which is perhaps more meaningful, it was using the AT bench and Hugging Face. It has its own baseline of benign agent traces and eight kind of agent failure from injected tools, description, to misinformation, and so forth. We look at the top 10 alerts that anomaly anomaly detection would generate and about seven to eight times of... Out of 10 of these alerts are actually anomalous, meaningful, and useful anomalous activities. We even even further and we looked at whether there's any kind of anomalous event that stand out more than the rest. To make things hard in production, first, models change. Fine tunes, and new releases, but also agents that each run on a different model, and routers that pick a specialist model per request. That means each model writes its pages its own coordinates. So either you learn every baseline, classifier, and explainer, which is a computationally prohibitive, or you have to find a way to map them all into one shared representation space. Second, real time. Means moving big objects. For example, a $1,256 token chunk of coin $314,000,000,000 is about 40 megabytes of KB. Compression and streaming like hash gen could help, but moving that for security analysis and acting in real time is still slow. And that's what we solve at Infiniti AI. We are looking for teams building their own inferences stacks who treat security as a foundational design decision. I hope this was helpful for you. Thank you so much for Let me start with one number from last month. On average, more than one agent has started on our platform every single minute, day and night. And honestly, the agents were the easy part. Hi. I'm Joey. I built Spectra, the cloud agent platform we built inside Harvey. After 10,000 PRs, the biggest lesson I can share is this. The thing that broke was an agent. It was Everything around them. So what does one submitted add up to? 10,000 PRs that an agent wrote. And it isn't just TFX team playing with a toy. Last month, about four sixty different people at Harvey used it. And over the summer, it stopped being only about code. People started asking a data question in Slack and getting answers back. Here it is. Recently, someone on our marketing team dropped a prototype and a talk track into Slack and asked for a storybook for a product video. Once the harness exists, people bring the work that you never planned for. So a promise upfront, this is not a talk about models, evils, or leaderboard. This is talk about hardness, odd things around the model. Checking on the repo, Holding the key, opening the PR, getting the merge. When we first wrote about Spectre back in April, we put it like this. At some point, a harness becomes the product. Now whenever I tell people this, I get the same question. Why on earth would you build this when CLARK or Cursor already exists? My answer, we don't compete with them. We run them. And here's how I see it, as my own view. ## Slides ### 00:00:09 # AI Engineer Europe 2027 ## ExCeL London Europe's flagship technical AI conference returns to ExCeL London for three days, February 17-19, 2027 — bringing the world's leading AI labs, platforms, and tools to Europe's builders. ### Event Details - **Dates:** February 17-19, 2027 - **Location:** London, UK - **Website:** ai.engineer/europe [Black and white photo of Big Ben and the Houses of Parliament] [Black and white photo of the London Eye] [Black and white photo of a classical building on a London street] ### 00:00:36 AI Engineer Code Summit PRESENTED BY Google DeepMind ### // AVAILABLE ## LIVE TRANSLATION Scan the QR code to access real-time translation. [QR code to access real-time translation] ### 00:01:36 ### THE IDEA # KV-first security. ## The signal your engine already writes. ### WAYS AN AGENT GOES WRONG - Prompt injection - Jailbreaks - Rogue tool calls - Broken tool calls - Sensitive-data exposure - Hidden backdoor - Drift - ...and the ones not named yet ### ALREADY THERE Your engine writes it anyway. The gems from the pile. ### COVERS THE BLIND SPOTS What slips past the guardrails still becomes computation. ### STAYS IN PLACE Read it where it lives. The prompt text goes nowhere new. [Diagram illustrating a security flow: "Ways an agent goes wrong" feed into a "Judge or Classifier" that reads the prompt, which then enters "IN THE COMPUTATION - KV PAGES" (a wavy band of squares). A red square within the KV pages and another outside indicate something "does not fit". The flow continues to another "Judge or Classifier" reading the response, leading to "Tool calls, actions". Text below the KV pages reads "one baseline, learned from your own traffic" and "THE GUARDRAILS' BLIND SPOT".] ### 00:02:09 THE IDEA # KV-first security. ## The signal your engine already writes. ### WAYS AN AGENT GOES WRONG - Prompt injection - Jailbreaks - Rogue tool calls - Broken tool calls - Sensitive-data exposure - Hidden backdoor - Drift - ...and the ones not named yet ### JUDGE OR CLASSIFIER reads the prompt ### JUDGE OR CLASSIFIER reads the response ### ALREADY THERE Your engine writes it anyway. The gems from the pile. ### COVERS THE BLIND SPOTS What slips past the guardrails still becomes computation. ### STAYS IN PLACE Read it where it lives. The prompt text goes nowhere new. [Diagram illustrating a process flow: various "ways an agent goes wrong" feed into a "KV PAGES" computation area, which is described as "one baseline, learned from your own traffic" and "THE GUARDRAILS' BLIND SPOT". A "JUDGE OR CLASSIFIER" reads the prompt and response, with an indicator showing something that "does not fit" within the KV pages. The process leads to "Tool calls, actions".] ### 00:02:41 ## THE IDEA # KV-first security. ## The signal your engine already writes. ### Ways an agent goes wrong - Prompt injection - Jailbreaks - Rogue tool calls - ### 00:03:09 ### THE IDEA # KV-first security. ## The signal your engine already writes. ### WAYS AN AGENT GOES WRONG - Prompt injection - Jailbreaks - ### 00:03:41 # KV-first security. ## The signal your engine already writes. ### WAYS AN AGENT GOES WRONG - Prompt injection - Jailbreaks - Rogue tool ### 00:04:05 WHERE TO READ IT # Inference is stateful now. Read the state. llm-d and NVIDIA Dynamo route each turn back to the KV it built, and LMCache ### 00:04:37 WHERE TO READ IT # Inference is stateful now. Read the state. llm-d and NVIDIA Dynamo route each turn back to the KV it built, and LMCache ### 00:05:05 WHERE TO READ IT # Inference is stateful now. Read the state. llm-d and NVIDIA Dynamo route each turn back to the KV it built, and LMCache keeps that ### 00:05:37 # A VOICE AGENT AN INSTRUCTION HIDDEN IN A VOICEMAIL ## No text to filter. The KV pages still flag it. Gemma-4-E4B with email, calendar, contacts and files - voicemails in voices it never heard ### Clean voicemail Same voice, speed ### 00:06:09 ### A VOICE AGENT · AN INSTRUCTION HIDDEN IN A VOICEMAIL ## No text to filter. The KV pages still flag it. Gemma-4-E4B with email ### 00:06:36 ### WHAT IT SHOWS # Normal stays together. What does not fit, separates. ## ON AGENT TRAFFIC, SERVED LIVE - attacks, above the line - normal ### 00:07:09 ## WHAT IT SHOWS ### Normal stays together. What does not fit, separates. #### ON AGENT TRAFFIC, SERVED LIVE **what does not belong separates** - attacks, above the line - normal traffic, below score against your normal traffic -> more unusual #### ACROSS EIGHT KINDS OF AGENT FAILURE - AUROC - Corrupted tool feedback: 0.78 - Direct prompt injection: 0.68 - Indirect prompt injection ### 00:07:37 ### WHAT IS STILL HARD ## The hard parts. ### Models change, and multiply. Fine-tunes, new releases, agents on different models, routers that pick one per request ### 00:08:10 # WHAT IS STILL HARD ## The hard parts. ### Models change, and multiply. Fine-tunes, new releases, agents on different models, routers that pick one per request. Each ### 00:08:39 # Harvey ## Day and night. ### 00:09:08 ### SPECTRE TODAY ## More than one task a minute - **10k+** agent-written PRs merged across our 7 app repos - **~45k** tasks in October, up from ~10k in May - **~460** people used it in October And it isn't just a platform team playing with a toy. [Three rectangular boxes displaying key statistics] ### 00:09:39 # Not just code ### Tasks per month, by kind of work, 2026 - **Code**: Writes, reviews and fixes PRs - **Data and analytics**: Answers data questions in Slack - **Triage and monitoring**: Alerts, incidents, rollouts - **Creative**: Storyboards, animations and video graphics for marketing [Stacked bar chart showing total tasks per month increasing from ~10k in May to ~45k in October, segmented by Code, Data and analytics, Triage and monitoring, and Creative tasks.]