# Chasing capabilities while waiting for the frontier — Alexander Shevchenko — session 2026-10-14T13:00:00.000Z → 2026-10-14T13:18:00.000Z _30 transcript lines · 50 slides · source: full recording_ ## Transcript **[00:00:00 · 0]** As it begins to work with this technology. How does the work change? Right? What becomes possible that wasn't practical before? And why, while superintelligence feels so powerful, why is the Roy still missing? **[00:00:16 · 0]** And that's part of what makes this gathering so interesting to me. You have people developing the technology in the same room as people who understand in considerable detail the work it needs to do. And there is a lot of those two groups can learn from each other. That's fascinating to me. And our first speakers work at the intersection of those two worlds, at REMP. **[00:00:42 · 0]** So please help me welcome Aleksandar Shevchenko with Chasing Capabilities While Waiting for the Frontier. Thank you. **[00:01:05 · 1]** Hey, everyone. I'm Alex from RAMP. I'll be speaking about chasing capabilities while waiting for the frontier to advance. So a quick intro about myself I joined Ramp around three years ago on the broader applied AI team, working on a bunch of different surfaces and products, applying LLMs and ML to make a great product even better. To caveat my presentation on Ramp Labs, Ramp is an extremely bitter, lesson built company. **[00:01:35 · 1]** Over the last year, the number of use cases that use LOMs has increased 123%, while at the same time, a lot of our top features are growing in token spend. But on a unit economic basis, they're actually becoming cheaper, because there are frontier models that are being released all the time now. And these models are cheaper, faster, and better. And so we're able to provide the same level of accuracy at a much lower cost. And so this is a really bitter lesson pill pattern. **[00:02:08 · 1]** And so given all this, a question that I sometimes get why not just wait? Why have an applied research function inside of a company like RAMP, an application layer company? Right? And to answer that, I would like to draw a parallel to 2023. Back in 2023, you had access to kind of like GPT-four, and creating an agent was a lot of work. **[00:02:36 · 1]** You would have to create very, very strict guardrails. Models weren't RL ed to backtrack. They didn't have thinking tokens. And so if it went in the wrong direction, it would just kind of charge ahead and not realize that it made mistakes. You had to sweat over all of the prompts to the point where some synonyms could change the behavior of your Agent, because they weren't RL'd properly on generalizing to the user's intent. **[00:03:05 · 1]** You had to reduce and scope down the context as much as possible, because giving too much context to the agent would just confuse it. And even back then, we decided, well, we need to ship these things. We can't wait for a GPT 4.5 or GPT five to come out. We need to serve our financial professionals now. And we shipped, things like Grand Tour Guide and many other agents, even though it was not the easy thing to do. **[00:03:32 · 1]** And fast forward now to 2026, building an agent is a lot easier. The models have gotten so much better that you can give them MCPs and access to any context. And they do a very, very good job of understanding what they need to do given a task. But I still believe that we can dig a layer down and create even better experiences for our users. It's just it requires a bit more effort now. **[00:03:59 · 1]** So to present this, some of our work from Ramp Labs, three specific cases of how we've applied research into Ramp itself, starting with creating a specialist model. So in November of last year, we launched Ramp Sheets, which was one of the really early spreadsheet agents. This was before the two frontier labs had launched their versions of spreadsheet agents. And we had a very good launch in November. A bunch of users from great companies had signed up and started using it. **[00:04:32 · 1]** And we realized that, well, this is a very general purpose spreadsheet handling agent. And a lot of the work that people do on ramp, both inside of the company but also all of our customers, accounting and finance professionals, are spending most of their days inside of spreadsheets. So we can take the harness and the brains behind Ramp Sheets and add it into a bunch of other surfaces within Ramp. And as we were building this out, we realized, well, we can probably apply some research from RAMP labs to make this even better. And we sat down, and we started looking, where do models fall short on this? **[00:05:15 · 1]** And back then, this was Opus 4.6. And the place that had the most friction was when you would drop in a lot of spreadsheets into ramp sheets, and you would ask it to build a DCF or model some financial scenario. It would spend a lot of time creating a plan at first and then researching all of the sheets that it had access to to try to get a good understanding of and the lay of the land. And Opus 4.6 is a great model, but it was pretty slow. And the thinking tokens are also pretty expensive. **[00:05:47 · 1]** And so we took a step back and decided to train a retrieval model, or kind of like a financial question answering model that would, while still keeping these frontier models as orchestrators, would then be handed off to this much faster model that would do financial question answering for that planning agent. And so we partnered with the company Prime Intellect to post train a QAN 3.5, 35,000,000,000 model. And we actually got above Opus 4.6, which was the frontier back then, exact task accuracy, while also being able to serve it at haiku level latencies. So a much faster, but just a smart model that could then be used in tandem with the really smart frontier orchestrator. And so this was a very successful example of post training that we had actually sent to prod as part of our Ramp Labs effort. **[00:06:51 · 1]** The second big thing that we are actually exploring up to this day is kv cache tricks. Particularly, cartridges are a really interesting family of techniques, where you are able to compress a lot of context, up to 20 to 30 times more context, into the KV cache offline and serve it for your agent. And so if you compare it to the Ramp Sheets example, for example, there are a lot of tasks that you know are going to happen at some point, right? Like month end close, a month finishes, and on the first of the next month, a financial professional will go and try to reconcile and close their books. You know the ledger, the aging report, and the documents that they will need to be retrieving. **[00:07:36 · 1]** And so you could prepare them as context, but the agent will still need to agentically retrieve and create a good picture and understanding of them. With cartridges, we're able to compress all of that work and build one overnight before the financial professional starts their work on the first of the month. And once they open up the agent, it knows exactly what it needs to do, because it has all of the context already built in. Cartridges are also a much lower lift in post training, because the way that they do self play is a lot more generalizable. The caveat with stuff like this is that compared to post training, this is something that the labs might ship at some point in their APIs, in their foray into continuous learning. **[00:08:24 · 1]** And even if they were to do it in the API, we would actually be happy, because we are developing a lot of this tribal knowledge inside of the company and just learning how best to apply these techniques. So even if it does ship as an API, we can just hit the ground running on day one knowing how to use them. And finally, the third example of applied research that we are sending to prod is our work on agentic map reduce and agentic swarms. So more concretely, we published an article on the Ramp Labs blog on how we used 10,000 parallel agents to pen test all of Ramp. So at Ramp, we have a Cloud agent called Ramp Inspect, which helps out with setting up all of the infrastructure and plumbing for something as ambitious as this. **[00:09:16 · 1]** And we were able to spin up 10,000 parallel agents and then kind of map this work out to try to find any vulnerabilities, and then also agentically reduce on the tickets that they would create. So deduplicate them with LMs and then try to reproduce them adversarially with LMs to try to make sure that the vulnerabilities are real and important. At the end of all of this, we have actually found seven high severity vulnerabilities that we all patched. Interestingly, one of the vulnerabilities that we described in that article is a Lua script injection that you could send over via email. And no single component of that stack was unsafe, and the kind of swarm had to patch together multiple components and understanding of multiple components end to end in order to be able to figure this out. **[00:10:08 · 1]** And the other nice thing with this is that this is all running automatically. You don't need to spend anyone's time. It's just you start it, and the only thing that failed passes that get filtered out is the cost of the tokens. The other thing that is interesting outside of just building out all of the infrastructure to support these swarms is that we do research into how to best use models as part of these swarms. One of the interesting insights from that same article that we published is that you can actually use open weight models that are up to five times cheaper compared to the frontier reasoning. **[00:10:45 · 1]** And because you're setting it up as a swarm, the detection rate is not five times lower. So you can spend more on these cheaper tokens and have almost as good of a detection rate. If you follow through all of these examples, you notice a pattern of a lot of them is attempts at saving on time and saving on money so reducing latency, which is time, and then reducing on token cost, which is money. That's actually Ramp's slogan, saving our customers time and money. And so I'd like to point to two products that we actually offer from Ramp that have been influenced by Ramp Labs' research. **[00:11:29 · 1]** The first one is token spend management that connects to a bunch of your API keys and is able to give you a really good picture of how LMs are being used inside of your company. Not only token spend on an API key basis also goes deeper, and we create a taxonomy that It also goes deeper, and we create a taxonomy that contextually analyzes and semantically analyzes the LOM usage so you can actually understand how LOMs are being used outside of just attributing it to a team or a person inside of your company. And then the second one is RAMP router, which also leans into improving the latency of your agent, where we have a bunch of routing strategies that we work with RAMP labs to develop, things like NVIDIA switchyard, things like smart flex off loading of work to both improve latency and cost per tokens that you spend. And there are a bunch of different strategies already available for routing, and even more coming out later. So if you are interested in any of these two products, you can take a photo, of the URLs or scan the QR codes there. **[00:12:47 · 1]** I'll give you a second. Awesome. And then I would also invite you to check out our Ramp Labs blog, where we publish a bunch of really interesting articles that I have not had the chance to cover as part of this presentation. Notably, one of the really interesting ones is portal, where we figured out a way of transferring learnings from one post train to another. So you could do something like a LoRa post train on one model, so like the Quen 3.5 that I was describing, but then transfer it to newer versions of Quen or even across different families. **[00:13:30 · 1]** So you're post training this latent that then can be projected onto different models. And that really amortizes the cost that you have to spend on post training. More blog posts on agentic MapReduce, so not only the pen testing stuff that we did, but also things like analyzing and creating taxonomies of LOM usage, which is stuff that has gone into the token spend product. Things like self improving monitoring, where we use LOMs to automatically create monitors with Datadog on a lot of our product surfaces, and then as alerts come in, improve those monitors automatically without a person in the loop to try to basically create super, super robust and resilient software. And finally, semantic tagging, which is this interesting foray into creating this agentic MapReduce over a bunch of user sessions, where we can annotate and create taxonomies that are open ended. **[00:14:27 · 1]** So not knowing ahead of time what are the clusters of OM usage, and trying to create a taxonomy that can then be reconciled against the spend inside of the companies. So if you are interested in this type of research, x.com/ramplabs, where you can go and check out more of our research. And here's my Twitter if you have any other questions or want to talk about some use case as part of your applied research. And that's it. Thank you. **[00:15:11 · 2]** As AI agents generate more code than ever, how do we make sure it's ready to ship? To explore the importance of verification in AgenTic software development, please welcome Tarek Shaukat, CEO of Sonar. **[00:15:36 · 3]** Morning, everybody. I'm... My name is Tarek Shokat. I am the CEO of Sonar, as you just heard. And we're here to talk today about how we have been, helping companies build software factories that actually work for enterprises. **[00:15:51 · 3]** And I thought it'd be an interesting way to start to start looking at actual factories, because we know how to build actual factories. And for those of you who have a little bit of the gray hair and all of this sort of thing, you may, remember the nineteen eighties. And in the nineteen eighties, you could find cars that were really good cars. But if you look at the picture here and if you remember those times, the cars, like, didn't quite fit together perfectly. Right. **[00:16:17 · 3]** You'd have different pieces that would leave big gaps. It would be really good cars that just didn't kind of work the way they were supposed to. And then, Toyota, and particularly Lexus, came out with a commercial. And it showed something very simple, which was a ball bearing running across the seams of this car. And it was kind of revolutionary at the time, because it showed that you could actually build a car that fit together perfectly. **[00:16:43 · 3]** And instead of building for volume and building and then fixing the old mindset, you could actually have a mindset focused on ship it right, on no rework, on customer happiness. And there was not... This was not a single change. What happened was that people really started thinking about the factories differently. They started engineering, building factories that were engineered to deliver quality. **[00:17:12 · 3]** And there's a number of different pieces that I think still resonate today. The first is you prioritize quality instead of volume. The second is that you fix the system, not just individual symptoms that you see. The third is that you design for something that can be built as opposed to designing the wheels and designing the washer bottles and whatever it is, and then hoping it fits together. You solve problems as they occur, as opposed to kicking the can down the road and fixing it at the end. **[00:17:43 · 3]** And you empower everybody. Anyone could stop the line, not just the boss. Now why am I talking about this? It's actually because modern software development actually embraced all of these principles. If you think about how we build software today, what is considered ## Slides ### 00:00:40–00:01:00 [Abstract graphic of white vertical lines on a dark background, resembling a stylized bar chart or architectural elements.] ### 00:01:20–00:01:40 # Hi, I'm Alex I lead **Ramp Labs**, Ramp's experimental research team. We take frontier research, replicate it fast, and ship it to finance teams. 3 years at Ramp · Microsoft before that x.com/ramplabs ## Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp ### 00:01:40–00:02:00 # We're a bitter-lesson-pilled company ## Active use cases +123% ## Approval intelligence -78% ## Underwriting -90% ## Vendor mailbox -81% --- Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Line chart showing an increase in active use cases over time] [Three smaller line charts showing decreases in approval intelligence, underwriting, and vendor mailbox over time] ### 00:02:00–00:02:20 # We're a bitter-lesson-pilled company ## Active use cases Approval intelligence **-78%** Underwriting **-90%** Vendor mailbox **-81%** ## Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp [Line chart showing active use cases increasing by +123% from Oct 2025 to Oct 2026] [Three smaller line charts showing a decrease in Approval intelligence (-78%), Underwriting (-90%), and Vendor mailbox (-81%) over a similar period] ### 00:02:20–00:02:40 # So why not just wait? ## Chasing capabilities while waiting for the frontier ### 00:02:40–00:03:00 # We didn't wait in 2023 either ## 2023 Ramp Tour Guide, built on guardrails, prompts and context ## 2026 Agents take the wheel, and the work moves down the stack ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research **ramp** [Screenshot of a web application's "Company settings" page with a "Ramp Tour Guide" tooltip] [Diagram illustrating a multi-step process flow with dots along a winding path, ending in a "Result" box] ### 00:03:00–00:03:20 # We didn't wait in 2023 either ## 2023 Ramp Tour Guide, built on guardrails, prompts and context ## 2 ### 00:03:20–00:03:40 # We didn't wait in 2023 either ## 2023 Ramp Tour Guide, built on guardrails, prompts and context ## 2026 Agents take the wheel, and the work moves down the stack ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Screenshot of a web application's "Company settings" page, showing a navigation menu, settings tabs, and company profile details, with a tooltip pointing to "Company profile"] [Diagram showing a winding path with multiple steps, starting with "Task" and ending with a yellow square labeled "Result"] ### 00:03:40–00:04:00 # We didn't wait in 2023 either ## 2023 ### Company settings Ramp Tour Guide, built on guardrails, prompts and context ## 2026 Agents take the wheel, and the work moves down the stack # Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Screenshot of a web application's "Company settings" page, showing a sidebar navigation, main content area with "Company profile" highlighted, and a tooltip instructing to click the "Company profile" tab.] [Diagram showing a winding path with dots representing steps, labeled "Task" at the start and "Result" at the end.] ### 00:04:00–00:04:20 # Three ways we put applied research to work - **Into a specialist** Post-training - **Precompute ahead of time** KV cartridges - **Across thousands of agents** Agentic map-reduce --- Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Diagram of nested squares with a bright yellow square at the center] [Diagram of a calendar-like grid of circles, highlighting "BUILT" and "SESSION" with a path between them] [Diagram of a grid of dots with lines fanning out from a central point to several yellow dots] ### 00:04:20–00:04:40 # A fast specialist for a smart orchestrator ## Orchestrator - A frontier model plans, reasons and edits the sheet ## FastAsk - A post-trained Qwen3.5-35B-A3B that answers questions about the sheet ### 00:04:40–00:05:00 # A fast specialist for a smart orchestrator ### Orchestrator A frontier model plans, reasons and edits the sheet ### FastAsk A post-trained Qwen3.5- ### 00:05:00–00:05:20 # A fast specialist for a smart orchestrator ## Orchestrator - A frontier model plans, reasons and edits the sheet ## FastAsk - A post-trained Qwen3.5 ### 00:05:20–00:05:40 # A fast specialist for a smart orchestrator ### Screenshot of Anthropic vs OpenAI Revenue & Valuation Analysis.xlsx ## Orchestrator - A frontier model plans, reasons and edits the sheet ## FastAsk - A post-trained Qwen ### 00:05:40–00:06:00 # A fast specialist for a smart orchestrator ## Orchestrator A frontier model plans, reasons and edits the sheet ## FastAsk - A post-trained Qwen3.5- ### 00:06:00–00:06:20 # A fast specialist for a smart orchestrator ### Screenshot of a spreadsheet: Anthropic vs OpenAI Revenue & Valuation Analysis ## Orchestrator A frontier model plans, reasons and edits the sheet ## FastAsk A post-trained Qwen3.5-35B-A3B that answers questions about the sheet in a fraction of the time # Chasing capabilities while waiting for the frontier [Screenshot of a spreadsheet application displaying financial projections and analysis] ### 00:06:20–00:06:40 # Opus-level accuracy at Haiku-level speed ## Exact-task accuracy - **FastAsk** 66.25% - **+14 pts** at Haiku-level speed - Claude Opus 4.6 61.88% - Claude Sonnet 4.6 59.38% - Qwen3.5-35B-A3B, base 56.25% - Claude Haiku 4.5 51.88% ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp [Line graph showing exact-task accuracy on the Y-axis (50% to 65%) versus time per task relative to Claude Haiku 4.5 on the X-axis (1.0x to 1.6x). Data points include Claude Haiku 4.5, Qwen3.5-35B-A3B, Claude Sonnet 4.6, Claude Opus 4.6, and FastAsk. FastAsk is shown at 66.25% accuracy at 1.0x time, with a vertical line indicating a +14 pts improvement over Claude Haiku 4.5 at the same speed. A dashed line represents the "Frontier before FastAsk".] ### 00:06:40–00:07:00 # Opus-level accuracy at Haiku-level speed ## Exact-task accuracy - **FastAsk**: 66.25% (+14 pts at Haiku-level ### 00:07:00–00:07:20 # KV cartridges hand the agent the whole month ## Today - Reading the ledger, aging report, open invoices - Reconcile ## With a cartridge - Reconcile Cartridge built overnight, loaded instantly Lower lift than post-training: no verifiers, no rubrics. Labs may ship this as an API one day. We're learning how it works now. ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp ### 00:07:20–00:07:40 # KV cartridges hand the agent the whole month ## Today - Reading the ledger, aging report, open invoices - Reconcile ## With a cartridge - Reconcile - Cartridge built overnight, loaded instantly Lower lift than post-training: no verifiers, no rubrics. Labs may ship this as an API one day. We're learning how it works now. ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ### 00:07:40–00:08:00 # KV cartridges hand the agent the whole month ## Today - Reading the ledger, aging report, open invoices - Reconcile ## With a cartridge - Reconcile - Cartridge built overnight, loaded instantly - Lower lift than post-training: no verifiers, no rubrics. - Labs may ship this as an API one day. We're learning how it works now. ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp ### 00:08:00–00:08:20 # KV cartridges hand the agent the whole month ## Today - Reading the ledger, aging report, open invoices - Reconcile ## With a cartridge - Reconcile - Cartridge built overnight, loaded instantly ### Lower lift than post-training: - no verifiers, no rubrics. Labs may ship this as an API one day. We're learning how it works now. ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp ### 00:08:20–00:08:40 # KV cartridges hand the agent the whole month ## Today Reading the ledger, aging report, open invoices Reconcile ## With a cartridge Reconcile Cartridge built overnight ### 00:08:40–00:09:00 # KV cartridges hand the agent the whole month ## Today - Reading the ledger, aging report, open invoices - Reconcile ## With a cartridge - Reconcile - Cartridge built overnight, loaded instantly - Lower lift than post-training: no verifiers, no rubrics. - Labs may ship this as an API one day. We're learning how it works now. # Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp ### 00:09:00–00:09:20 # Pen-testing Ramp with ~10,000 agents - ~10,000 sessions - No finding, dropped - ~6,000 tickets ### 00:09:20–00:09:40 # Pen-testing Ramp with ~10,000 agents - ~10,000 sessions - No finding, dropped - ~6,000 tickets - Duplicates merged - ~3,000 tickets - Reproduced adversarially - 7 High, all patched ## One email ran code in our renderer ```lua html = [====[ Hi team, ]====] os.execute(...) ``` Simplified The email closed a Lua string early. No single component looked unsafe; the agent traced the flow and proved the exploit in its sandbox before filing. Failed passes cost only tokens. --- ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Funnel diagram showing a reduction from 10,000 sessions to 7 patched issues] ### 00:09:40–00:10:00 # Pen-testing Ramp with ~10,000 agents - ~10,000 sessions - No finding, dropped - ~6,000 tickets - Duplicates merged - ~3,000 tickets - Reproduced adversarially - 7 High, all patched ## One email ran code in our renderer ```lua html = [====[ Hi team, ]====] os.execute(...) ``` Simplified The email closed a Lua string early. No single ### 00:10:00–00:10:20 # Pen-testing Ramp with ~10,000 agents - ~10,000 sessions - No finding, dropped - ~6,000 tickets - Duplicates merged - ~3,000 tickets - Reproduced adversarially - 7 High, all patched ## One email ran code in our renderer ```lua html = [====[ Hi team, ]====] os.execute(...) ``` Simplified The email closed a Lua string early. No single component looked unsafe; the agent traced the flow and proved the exploit in its sandbox before filing. Failed passes cost only tokens. [Funnel diagram showing a reduction from many sessions to a few tickets] ### 00:10:20–00:10:40 # Pen-testing Ramp with ~10,000 agents - ~10,000 sessions No finding, dropped - ~6,000 tickets Duplicates merged - ~3,000 tickets Reproduced adversarially - 7 High, all patched One email ran code in our renderer ```lua html = [====[ Hi team, ]====] os.execute(...) ``` Simplified The email closed a Lua string early. No single component looked unsafe; the agent traced the flow and proved the exploit in its sandbox before filing. Failed passes cost only tokens. [Funnel diagram showing a reduction from 10,000 sessions to 7 high-priority patched items] ### 00:10:40–00:11:00 # Cheap models still find real bugs Detection rate on the 7 known High findings, 25 attempts each - GPT-5.5 (high reasoning): **52.0% ### 00:11:00–00:11:20 # Most teams overpay for intelligence ## The price of inefficient AI Same work. Unoptimized teams pay several times more per token. **4.15x** higher cost per 1M tokens for unoptimized teams versus efficient ones. Cost per 1M tokens Token Spend Management See where every token dollar goes ## Cost impact over time **40% lower** Ramp Router router.com ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Bar chart comparing costs per 1M tokens for unoptimized ($2.99) versus efficient ($1.11, $0.72) teams] [Stacked bar chart showing cost impact over time, with a 40% lower cost achieved by 60% Flex compared to 40% Default] ### 00:11:20–00:11:40 # Most teams overpay for intelligence ## The price of inefficient AI Same work. Unoptimized teams pay several times more per token. **4.15x** higher cost per 1M tokens for unoptimized teams versus efficient ones. Token Spend Management See where every token dollar goes ## Cost impact over time **40% lower** Ramp Router router.com ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp [Bar chart comparing cost per 1M tokens for different optimization levels: $2.99 (unoptimized), $1.11, and $0.72 (optimized)] [Stacked bar chart showing cost decreasing over time, with a breakdown into "40% Default" and "60% Flex" components] ### 00:11:40–00:12:00 # Most teams overpay for intelligence ### The price of inefficient AI Same work. Unoptimized teams pay several times more per token. **4.15x** higher cost per 1M tokens for unoptimized teams versus efficient ones. ### Cost impact over time **40% lower** Token Spend Management See where every token dollar goes Ramp Router router.com ## Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp [Bar chart comparing cost per 1M tokens for unoptimized ($2.99) versus efficient ($1.11 and $0.72) teams] [Stacked bar chart showing decreasing cost impact over time, split into "40% Default" and "60% Flex" components] ### 00:12:00–00:12:20 # Most teams overpay for intelligence ## The price of inefficient AI Same work. Unoptimized teams pay several times more per token. **4.15x** higher cost per 1M tokens for unoptimized teams versus efficient ones. Token Spend Management: See where every token dollar goes ## Cost impact over time 40% lower Ramp Router: router.com Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Bar chart comparing cost per 1M tokens for unoptimized ($2.99), optimized ($1.11), and highly optimized ($0.72) scenarios.] [Stacked bar chart showing decreasing cost over time, with segments representing "40% Default" and "60% Flex" contributions.] ### 00:12:20–00:12:40 # Most teams overpay for intelligence The price of inefficient AI Same work. Unoptimized teams pay several times more per token. **4.15x** higher cost per 1M tokens for unoptimized teams versus efficient ones. Token Spend Management See where every token dollar goes Cost impact over time 40% lower Ramp Router router.com ## Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp [Bar chart comparing cost per 1M tokens for unoptimized ($2.99) versus efficient ($1.11 and $0.72) AI] [Stacked bar chart showing cost impact over time, with a 40% lower cost achieved by combining "40% Default" and "60% Flex" components] ### 00:12:40–00:13:00 # Try them today - Token Spend Management: [https://ramp.com/ai-token-spend-management](https://ramp.com/ai-token-spend-management) - Ramp Router: [https://router.com](https://router.com) [Two QR codes, one for Token Spend Management and one for Ramp Router] ### 00:13:00–00:13:20 # More from Ramp Labs ## Portal Fine-tune once, carry it to the next model ## Agentic map-reduce Fan agents out over huge inputs, reduce to one answer ## Self-improving monitoring Agents find monitoring gaps and patch them ## Semantic tagging A map of how your company actually uses LLMs [x.com/ramplabs](x.com/ramplabs) Research write-ups and code Ramp Labs is hiring Find me after the talk ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research ramp [Diagram showing two green squares, one moving to the other with a dashed arrow] [Diagram showing a central yellow dot connected to multiple white dots in a fan-out pattern] [Diagram showing a wavy line with a highlighted section and a dashed box] [Diagram showing clusters of yellow and white dots within three dashed circles] [QR code for x.com/ramplabs] ### 00:13:20–00:13:40 # More from Ramp Labs ### Portal Fine-tune once, carry it to the next model ### Agentic map-reduce Fan agents out over huge inputs, reduce to one answer ### 00:13:40–00:14:00 # More from Ramp Labs ### Portal Fine-tune once, carry it to the next model ### Agentic map-reduce Fan agents out over huge inputs, reduce to one answer ### Self-improving monitoring Agents find monitoring gaps and patch them ### Semantic tagging A map of how your company actually uses LLMs [x.com/ramplabs](https://x.com/ramplabs) Research write-ups and code Ramp Labs is hiring Find me after the talk ## Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Diagram showing a square moving from one position to another] [Diagram showing multiple inputs converging to a single output] [Diagram of a waveform with a highlighted section and a dotted box] [Diagram showing three clusters of dots] [QR code linking to x.com/ramplabs] ### 00:14:00–00:14:20 # More from Ramp Labs ## Portal Fine-tune once, carry it to the next model ## Agentic map-reduce Fan agents out over huge inputs, reduce to one answer ## Self-improving monitoring Agents find monitoring gaps and patch them ## Semantic tagging A map of how your company actually uses LLMs [https://x.com/ramplabs](https://x.com/ramplabs) Research write-ups and code Ramp Labs is hiring Find me after the talk --- Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Diagram showing two squares, one with a yellow center, connected by a dashed arrow] [Diagram showing a central yellow dot connected to multiple white dots in an arc] [Diagram showing a wavy line with a highlighted yellow segment within a dashed rectangle] [Diagram showing three clusters of white and yellow dots within dashed circles] [QR code linking to x.com/ramplabs] ### 00:14:20–00:14:40 # More from Ramp Labs ## Portal Fine-tune once, carry it to the next model ## Agentic map-reduce Fan agents out over huge inputs, reduce to one answer ### 00:14:40–00:15:00 # More from Ramp Labs ### Portal Fine-tune once, carry it to the next model ### Agentic map-reduce Fan agents out over huge inputs, reduce to one answer ### Self-improving monitoring Agents find monitoring gaps and patch them ### Semantic tagging A map of how your company actually uses LLMs [x.com/ramplabs](https://x.com/ramplabs) Research write-ups and code Ramp Labs is hiring Find me after the talk Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [Diagram showing two squares, one highlighted, with a dashed arrow indicating a process flow] [Diagram showing a central node fanning out to multiple nodes, then reducing to a single output node] [Diagram showing a waveform with a highlighted section and a dashed box around it] [Diagram showing three clusters of dots, some highlighted, within dashed circles] [QR code linking to x.com/ramplabs] ### 00:15:00–00:15:20 # Thank you! ashevchenko@ramp.com x.com/shevchenkoaalex ### Chasing capabilities while waiting for the frontier Alexander Shevchenko / Head of Ramp Labs & Applied Research [QR code for contact information] [Ramp logo] ### 00:15:20–00:15:40 # AI Engineer ## NEW YORK Presented by **Arize** [Stylized graphic resembling a cityscape or art deco architecture with geometric shapes and vertical lines] ### 00:16:00–00:16:20 # How you build and run the factory matters ## 1980s - Built for volume - 'Build, then Fix' mindset ## 1990s - Built for customer happiness - No rework, ship it right The Machine that Changed the World, vAI Tariq Shaukat / CEO Sonar [Image of the rear of a white 1980s car] [Image of a silver Lexus logo on a dark background] ### 00:16:20–00:16:40 # How you build and run the factory matters ## 1980s - Built for volume - 'Build, then Fix' mindset ## 1990s - Built for customer happiness - No rework, ship it right ### The Machine that Changed the World, vAI Tariq Shaukat / CEO Sonar [Image of the rear of a white 1980s car] [Close-up image of a silver Lexus emblem on a dark car] ### 00:16:40–00:17:00 # How you build and run the factory matters ## 1980s - Built for volume - 'Build, then Fix' mindset ## 1990s - Built for customer happiness - No rework, ship it right ### The Machine that Changed the World, vAI Tariq Shaukat / CEO Sonar [Image of the rear of a white 1980s-style car] [Image of a silver Lexus emblem on a dark car] ### 00:17:00–00:17:20 # What changed? ## The factory was engineered to deliver quality - **Quality first** not volume at all costs - **Fix systems** not symptoms - **Design to manufacture** vs. producing in silos - **Solve problems as they occur** vs. fixing at the end - **Empower everyone** vs. just a few The Machine that Changed the World, vAI Tariq Shaukat / CEO Sonar ### 00:17:20–00:17:40 # What changed? ## The factory was engineered to deliver quality - **Quality first** not volume at all costs - **Fix systems** not symptoms - **Design to manufacture** vs. producing in silos - **Solve problems as they occur** vs. fixing at the end - **Empower everyone** vs. just a few --- The Machine that Changed the World, vAI Tariq Shaukat / CEO Sonar ### 00:17:40–00:18:00 # What changed? ## The factory was engineered to deliver quality - **Quality first** not volume at all costs - **Fix systems** not symptoms - **Design to manufacture** vs. producing in silos - **Solve problems as they occur** vs. fixing at the end - **Empower everyone** vs. just a few ### The Machine that Changed the World, vAI [Five vertical boxes, each listing a principle and a contrasting statement]