# Eyes Up: Edge AI Innovation — Dominic Pajak — session 2026-09-24T13:00:00.000Z → 2026-09-24T13:30:00.000Z _95 transcript lines · 57 slides_ ## Transcript So, yeah, my name is Dominic Pike. I'm at Arm, and today we're gonna talk about what happens when you run inference of the edge. So there's there's three things going on right now, which I think are gonna unlock some incredible opportunities. One is just the speed that small, open weight, capable models are becoming available. You combine that with the fact there is tons of accessible hardware you can then put these things on, and there is this potential for devices that are a little bit more intelligent, a little bit more Aware of the intention of the user and deliver experiences that have not been seen before in all kinds of different environments. And so, yeah, I'm I'm from Arm. So a little bit about Arm. Arm is the compute platform at the heart of devices all around you, everywhere from highly efficient data center infrastructure. There's physical AI with automotive and robotics. So hard real time safety critical applications all the way down to pretty much every smartphone you've ever owned, client computing. And industrial IoT and emerging devices, and that's the the stuff that me and my team have been working on, enabling new classes of devices to be created and come to market. And so I am a computer history geek. I love the fact, by the way, that this station f is on, Harvey Alan Turing. I don't know if I pronounce Harvey correctly, but, it's it's fascinating to look at the history of computing and where it's taking us today. So if we look back. The past eighty years of how people have interfaced with machines starting in the very first days, you know, maybe starting eighty or more years ago with punch cards and tape and then keyboards and then mice and pointers and touch screens. Each generation has made computing easier for people to access and interact with. But irrespective, every time it is still the case that the user is having to give their attention to that device and have the mental Should have converting the thing they want to get done into steps that the machine understands in a form that it can be inputted into that device. And the, you know, the amazing thing about the AI era is it inverts this relationship. Suddenly, the machines understand us. Not I'm not saying in a conscious way, but, you know, they can understand natural language and they convert our intent into actions and tool calls to get things done. And so, in some way We're talking about edge inference. In some ways, this isn't new. So there's been object detection and CNNs running on edge devices for for many years. So, you know, envision applications and factory inspection or parking bay monitoring and all this kind of stuff. More recently, embeddings, LLMs, and then agentic applications, which I guess is LLMs that can reason strongly enough to call tools and, you know, deal with context long enough to act autonomously on behalf of the users. These are just Coming possible today. I think it is good to step back a little bit and just look how fast this has happened. And so I, you know, I graph this out and really this is kind of, you know, look at GPT 3.5, and, you know, that was the first time I really experienced, what an LLM was, and it was, you know, a mind blowing experience. But if you look how quickly this capability is then being compressed into models with fewer and fewer parameters, up into the present Day, you know, this can now fit in a billion parameters roundabout and then run on a smartphone or run on a Raspberry Pi in your hand. This is phenomenal. And I by the way, I know that benchmarks are not perfect, and then the LU probably less so. This indicative of a trend, which is gonna continue. You know, if I'm here next year, this is gonna have gone down even further. And it's it's not just general knowledge question answering, it's reasoning too. And so here we're looking at a frontier of, you know, what can you do with 4,000,000,000 active parameters? The kind of compute on, today's edge devices. And again, you know, this is gone, from especially with mixture of experts coming into the picture, gone from something which is now, exceeding what, Claude Sonnet, 3.5 was doing when it was first released with its Frontier GPQA Diamond score, I think this was like a year and a half ago. This stuff is just moving phenomenally fast. And, of course, it is not just about capability. It's about Latency when we're talking about human user interaction. And there are also models now which can run comfortably on device, which do, speech to text in a latency, which are useful for creating voice UI without any need to send this inference to the Cloud. And I think it's very interesting to consider what drives, you know, these latency requirements. And it's really, you know, it's what people expect. What they're comfortable with is the way we should be designing these things. I Find this graph really fascinating. So this is an anthropological study of human conversation turn taking and the latency between speakers. And it's interesting that it's culturally dependent and language dependent. So Japanese is very quick and efficient in responding. Danish is very laid back and very slow. I don't know if that's because the verb is at the end of the sentence, by the way. And then English is kind of in the middle. And so that gives you a kind of gauge of of where these things need to land. And so, yeah, so I hope you are as As as I am about all of this inference being possible on edge devices, you may say, well, you know, we could do this on the data center already. Why, you know, why put it at the edge? And it's it's a good question. I think there's some, you know, very, very good reasons for this. Number one is efficiency. And so, you know, I think many people are already looking at their token usage and thinking, how how much of this could I offload onto a local device or a device on prem? There's the fact that you can customize this. So maybe you Want to use personal data or proprietary data to your organization and you would like to keep this, in house. And then there's kind of this opportunity for a tent sorry, intention aware devices where maybe they're observing your speech or have cameras, and are constantly looking and listening. And a lot of people may want to choose whether that stuff is shared off the device onto a data center or not. So I think these are some really compelling reasons why we would want to put Inference at the edge. And so I did say, you know, there's something else going on here. And I think it's this combination of you've got this accessible hardware. You've got these open weight models. Putting this together, I think, is is a fascinating opportunity. And so I don't know how many people are like hardware makers or into hardware here. The hardware people here can have a show of hands. Good. Okay. I'm glad to see that. And so, you know, it's been notable that for many years Now people have been building pretty amazing, hardware innovations using these composable modules from, like, Raspberry Pi or Arduino, maybe in the first instance, and then, creating totally new designs based on this stuff. And I think that combination of hardware innovation plus these open models is gonna open up some really interesting potential. I think the observant among you may notice that the micro duck is in the corner there. And, you know, my my favorite example of this combination, because I don't think it's really As widely, you know, it's not happening yet as widely as it could do. But a really nice early example of this is the hugging face robotics. Oh, sorry. Hugging face pollen robotics, Ricci mini. And So I know Ricci mini is from France. I was very happy to be on the beta, test the program for Ricci mini. So last Christmas with my son, I built Ricci mini, and I was really happy to see. So inside of this thing, it has a Stewart platform so it can express emotions with its head, and it has got a bunch of Those that, move the head platform. So this has ARM Cortex seven zero in there. They have actuators on the antenna. They have a Raspberry Pi compute module four for the Wi Fi version. And so with that, you can either run some applications on the device itself or you can use that to then call inference services off device. And so that's super cool. But it's not just hardware. It's also an SDK, which is open source, and it's a community of developers. Creating really cool applications for this platform. And here is one application example I would like to highlight. So this is by our very own Marco Domingo at Arm. And he created an app which basically, it sits on your desk. And if you use your phone, like start doom scrolling or getting distracted, it tells you not to. And this this won a prize. So this won him a VIP ticket to NVIDIA GTC in San Jose this year and a DGX Spark. And it was my Ricci Mini. I have to say, I'm not bitter about that. But well done, Marco. It's a cool app. And actually, if you go on the App Store, it is there today on the it's a hugging face space where they keep Ricci Mini Apps. So in the design of these types of on device applications, you know, there's a engineering challenge because you have constraints. Right? So you have to think about the available computer and memory. And oftentimes, you will have to divide up this task to make best use of the available hardware. So for a vision app. Location is quite common to maybe have, like, a motion gate. Maybe this triggers object detection. And if you classify things you're interested in, you grab a region of interest. Maybe you do image embeddings, and maybe only then do you go and then fire off a BLM. And so these tasks may be carried out by different types of processor on device and in some cases may then go off to other devices entirely or even to the data center to be run. And here's another example. So this is the app. Made during the beta test of Riichi mini. And this is running entirely on a Raspberry Pi five. What I love about this is you, you know, you say something and Riichi will immediately turn his head. And there's something magical about embodied AI in the way that, you know, a smart speaker may just bong or the LED lights up. But having a character turn its head to face you and then respond is a really cool experience. And, I think in doing this, you know, I was using a Raspberry Pi, which is definitely capable of running a bunch of different LLMs, but just because of the Latency constraint. I chose to use embeddings and then do a semantic match and call tools against this. So you could still ask it the weather or the time, and it could generate using text to speech on device also and reply. And here's Marco's demo. And by the way, I forgot to say, so in the main expo outside after this talk, Marco will be showing this, demo live, and you can ask him questions about it. Fundamentally, what we have here is he has his DGX Spark, so he's got a more capable device. From his prize wedding. And he's running, another interactive agent like demo. This time, he's using minstrel minstrel three on device, and he's doing speech to text. And then he's creating responses, synthesizing a response using text to speech and playing it back. In addition to that, he's doing a bunch of tool calling. And so he has this ability to you know, you can ask it what it is saying. It will Tell you, you know, actually, maybe I'll play the video. That could be a better way of doing this. Let me just do that really quickly. Okay. Yes? Can you greet the room? Hi, everyone. I'm Rishi. Nice to meet you all. I've got a badge here for AI engineer, and let's see if Rishi can do anything with it. Hey, Rishi. Yes? Can you meet the attendee? Show me your badge. Nice. Got it. Give me a second. Hi, Dominic. Good to see you at the AI Engineer Conference. I see you're from Arm. They're known for designing risk based CPU architectures powering everything from smartphones to supercomputers. Now finally, can you dance? How was that? Too much? Wants me to dance again? See you later, everyone. Ritchie's off to recharge. Have a great day. Alright. Hey, Rij. So I love the character that, and personality that Riji has through, minstrel, by the way. Too Much is a really cool, demo. If you go outside, like I said, you can actually go and see if it can read your badge and tell your fortune, etcetera. It is really cool to actually interact with it in person. So those are a couple of examples, but I did talk at the beginning about intent. And so you probably heard about the intention economy and you know, the attention Is, you know, you get people looking at apps and there's adverts and and stuff like that. The intention economy is understanding, what people want and providing solutions to it without them having to spell out everything. I think that's the way I would describe it. And here's a little example of an interaction that could be possible through this intent. So, you know, if you have, for example, reaches a bed light bed night lamp there, that's also connected to your home assistant system, for example. If you ask it, you know, you tell it you're done for the day, it may not be able to interpret that right. If it understands your preferences in terms of, you know, AC temperature or lighting, it can go and act on it. If it's got ambient context, and if Ricci is maybe looking or listening and remembering, has memory, about the context it's it's receiving. It can maybe see, oh, your partner's asleep. And so maybe Riichi doesn't say anything. It just nods and it kind of dims the lights, and it goes to sleep itself. And I think, you know, this ability to deliver a better UX with fewer interactions with a device is really the key to this thing. This is just one small example, and I think there are thousands of others. And inherent in this is the fact that, you know, there there is an agent running here. So we talked about vision and audio and sensor streams and maybe integration with with Home Assistant. These are effectively events or maybe state that can be queried from an agentic system. And of course, you know, with with the agent, there's a harness which is running on a CPU, and that CPU is doing a bunch of work, managing that context and and the memory. And then, I think the, key thing here is how much of the inference can be done off or off of the device. And I think this is like an orchestration problem, which a lot of the industry are focusing on today. Certainly, you know, task aware routing and being able to judge whether a task is possible to run on a device is is really key. And we'll look at that in a second. And finally, you know, we've been talking a lot about consumer based applications and how people interact with devices. There are a ton of embedded systems out there that have computers that actually, know, maybe they're in industry or smart greenhouses or, you know, all kinds of commercial building systems where, they're mainly monitoring sensors or maybe doing actuation, but with limited, interaction from a person. And so I really like to think about this idea of of those thousands of embedded systems that are out there. Is there a potential for, you know, an agentic embedded system? So I am not saying that the agent carries out all of these tasks, more that the agent takes the role of a supervisor and when things happen out of band, in other words, you know, if there's a fault that the designer of the system hadn't anticipated, can these agents improvise to go and fix them? And the kind of thought experiment, I think is really interesting is Bits per second communication to do this. And so, you know, it's interesting to consider, could agents be used to preserve these systems and monitor them, and keep them in good health? And I think this is one of the really interesting avenues for exploration. On the tip on the topic of hybrid AI, so, you know, there are a ton of interesting research papers in this area and there's some open source projects, you know, Light LLM or Root LLM. That you can apply to this challenge of, you know, can inference run on this constrained device in a model that fits there? Do I need to send it externally to a more capable piece of hardware? And so I was running some experiments the other day that, you know, there's a couple of different strategies. I think that the embedding kernel is the the quickest because you don't actually have to run the prompt. You really just, you know, you have some kind of calibration set. You look at prompts that successfully ran locally and ones that didn't, and you see which one is more similar to your If a prompting question, there's other solutions out there like calibrated uncertainty or even decoder state probing, which are possible once you've actually run inference. You've already drafted a response to the prompt, and it may be then you need to kind of defer this off device to be run, if the confidence is low or if it classifies as something which should would have failed. This is super interesting stuff. I have to say, you know, these evals are really just looking at single, single prompt. Eval sets. There are, you know, there's pinch bench, Clory. Is it Clory? Now there's a bunch of eval's looking at multi step tasks. I tend to think, you know, it's really your own application, which is the best thing to consider in these cases. So I've talked a lot about what we can do with a combination of really efficient models and accessible hardware. And just like the inspiration of creating devices, they're a little bit more, I think, you know, a more more Considered unless obtrusive. I think this is something that would benefit everyone. But, of course, you know, I should also talk about the hardware beneath a lot of this stuff. And I mentioned at the beginning, I work for Arm. And Arm has processor solutions that span from these really low power, wearable or, you know, sensor node type devices with our Cortex M and Ethos, up to Linux based platforms with Cortex a, which you might find in, in Jetson or in Raspberry Pi. Right the way up to data center infrastructure, and our focus is providing a common software framework and, we have recently announced an AI portal which has optimized models and also performance optimization tools. And one thing, you know, I would definitely encourage you, you know, I'll show this QR code at the end. Please register, become an early access customer to this thing. We're interested to get your feedback and improve this for the applications you'd like to develop. And, Just a couple of examples. So, you know, we've upstreamed kernels for Clidy AI into LAMA CPP into Onyx runtime. So when Gemma four came out, it ran optimally on SME two. But it's not only the new stuff. There's also consideration for, really widely used kind of established models and hardware. So I really like this example, which we have just released or due to release very shortly, which is Ultralytics YOLO 26, kind of opt Specifically for the Raspberry Pi five, ISA. And so they got 1.87 times faster inference. I think the key thing to point out here is, you know, typically, when you look at the FPS numbers for some of these models, they're occupying if they're running on CPU, all four cores. And really, in an application that is not ideal, you would like some headroom to do other stuff. And this kind of optimization is the difference between, you know, having two cores free to run your application, run other Kinds of algorithm on that output. So this is really interesting stuff, I think. Okay. So if you would like to get more resources, there's that QR code again for developer.arm.com/ai. Please sign up. Yeah. There's early access to our optimization tools, which I would encourage you to get into. Also, I'm really happy to connect with you either on LinkedIn or I'll be out by Ricci later on. And so, yeah, just to say again. So Marco Domingo, I think he'll be there for PM. So not directly after this, but at the break. He'll be on the the expo stage with Ricci Minnie if you'd like to go and chat with him and interact with Ricci. Alright. Thank you very much. Cheers. So I feel connected to to Cognition. How you feeling, everybody? Yeah. Good? Yeah? Okay. You know what I'm gonna do. Right? How are you feeling, guys? Okay. I think the next one is gonna be great. Yeah? Well, thank you so much, thank you so much, Dominic, for the presentation. Now we're gonna move on to Our next speaker who's going to talk to us about agentic map reduce for large scale coding agents. I'm super curious about this one. I wanna see what what this means. But please join me in welcoming to the stage VP of Engineering, EMEA at Cognition, Yanis Sorakis. Okay. Let's see. Oops. Am I connecting to the right projector? Classic. Please give me a few seconds to sort it out. Uh-huh. That's good. That's good. It's promising. And I need to see how I extend my screen. Slush. Excellent. Thank you very much for having here. It's such a great pleasure to be speaking to AI engineering in Paris to this great audience. Very excited about what I have to show to you today. An approach on tackling very Arge a codebase wide problems. And, yeah, let's get going. A few words of intro for us. We're cognition. We're in and apply AI research lab. Our founding DNA is in, AI research, competitive programming, competitive maths. We're based in San Francisco, but, we are now all around the world, including London. This is our European base of operations where I'm based. Across the most, like, sophisticated organizations in the world. They range from, systemically critical banks, Goldman City, Santander all the way to the US Army. Big tech, smaller, like a fast moving, tech native startups. And, we love tackling like a really, really like difficult, coding tasks. And I want to reflect on our experience. Rinse's on sort of like the classes of problems that one, can see. Let us consider let's start by think two categories. Okay. Let's call them kind of like localized, agentic workloads, which are basically kind of like, take the your median session with an agent. I want to implement a feature or given a particular area of the code base. I want to kind of like a write some tests or augment the existing test coverage. Given a particular, like, slice of my DB proxy, I want to optimize queries and so on and so forth. This concerns work. It's kind of like a like a bounded problem. It is like a set of relevant files in the code base. Okay. That basically modularize the whole logic. And equally importantly, if I were to tell an agent what, what to do, I will either point them. Explicit to the files. Or if I don't do, they can trivially go and find the relevant area neighborhood of the code base to carry out the task. Now let us consider what I would kind of like simply call codebase wide task. And by the way, codebase wide doesn't only mean like a single repo. You can have multiple repose. Let's take, for example, a case that we're going to revisit again again today. Security scanning. What is the security profile of my application? We know that there are, like, the individual security kind of like a quality levels of different modules of the code base. But what is equally important is the different kind of like the ways that interplay and how those things can evolve in the real world, how they can connect in order to reveal vulnerabilities similar in a similar vein, extracting common logic. We work with clients that have, like, really, really large code bases. They want to commonize ways they treat arithmetic, financial formulas, or even kind of like infrastructure ways of interacting with the database. They're all sort of like, first of all, it is a search problem, but it's also a problem of sort of like understanding and finding all the different nuances. Okay, by which you need to a common library needs to be extra. Acted need to conform to and so on and so forth. There's a complex interplay between the components. Take the security example, the security quality of my individual, let's say microservices. Okay. Could be kind of like a, you know, okay. I have like, if I were to look at them in isolation, low, priority vulnerabilities, they can be the backlog, but when chained together, they can give rise to a critical exploit. And then, therefore, like for those tasks, the result is trustworthy when you consider the totality of the code base or the totality of the deployed application for sets of applications. Alright. And, of course, one can say, kind of like, yeah, that's why we have n to n tests. Okay. That's why kind of like in our SDLC, we have a different quality gates and so on and so forth. Absolutely. Our goal is always called to for to ship. If left, right, I I do to catch those things and be aware of those things as, as soon as possible. Now if if I were to double click a little bit on sort of like, what the challenges, an agent could face in those code base white tasks. I'm telling, can you please find? I want to find instances of a particular formula of a particular logic that appears in different variations across my code base because I want to extract to a common library. Okay, there is a challenge of finding the work in the first place. Okay, it is a search problem. Okay, agents will spend a lot of time searching and backtracking like burning tokens. Okay, number two, polluting the context. There is a great great amount is going to build up. ## Slides ### 00:00:12 # AI Engineer PARIS - NVIDIA - ClickHouse ### 00:00:36 # Dominic Pajak ## Senior Director Edge AI, arm ### AI Engineer PARIS - Organized by Mistral - STATION F - September 23-24 ### 00:01:04 # When AI fits in your hand ## Efficient models Capable, open-weight edge models ## Accessible hardware An accelerating ecosystem of modules and platforms. ## Intent-aware interaction Devices that understand what you need. [Four icons representing AI models: a castle-like structure, the word "Gemma", the word "Qwen", and a smiling face with hands] [Three images of small circuit boards or hardware modules] [Diagram showing a central person icon connected to four surrounding icons: a microphone, an eye, a checklist, and a house] ### 00:01:35 # Arm is the foundational platform for AI ## CLOUD AI ## PHYSICAL AI ## EDGE AI Arm AI compute platform [Illustration showing server racks and data storage for Cloud AI] [Illustration showing a humanoid robot, a drone, robotic arms, and two transparent cars for Physical AI] [Illustration showing a smartwatch, smartphone, smart glasses, VR headset, security camera, smart thermostat, laptop, and a small smart home device for Edge AI] ### 00:02:03 # Arm is the foundational platform for AI ## CLOUD AI ## PHYSICAL AI ## EDGE AI Arm AI compute platform [Diagram showing server racks under "CLOUD AI", a humanoid robot, robotic arm, cars, and a delivery robot under "PHYSICAL AI", and various smart devices like a smartwatch, smartphone, smart glasses, camera, drone, laptop, and smart speaker under "EDGE AI"] ### 00:02:34 1940s Punch Making ### 00:03:06 # Making machines easier to un... ## 1940s - Punch ## 1960s - Type ## 1980s - ### 00:03:35 # On-device inference: from perception to agency Established ----------------------------------------------------------------> Emerging - **Perception** - CNNs - **Understanding** - Embedding models - **Generation** - LLMs - **Agency** - Reasoning LLMs - + tools + feedback loops More capabilities, working together on device. [Horizontal arrow diagram showing a progression from "Established" to "Emerging" through four stages: Perception, Understanding, Generation, and Agency] ### 00:04:06 [Graph showing data with a vertical axis label "1B" (1 Billion) and a horizontal axis label "2022"] ### 00:04:33 ### Disclaimer Benchmark and test conditions vary. [Graph with axes labeled "1B" and "2023"] ### 00:05:06 # Devices can now reason, not just respond GPQA Diamond - 548 active parameters - Dense models - Mixture of experts [Line graph showing the evolution of AI model performance (GPQA Diamond score) over time (2024-2026)] - **2024:** - Meta Llama 3.2, 3B - Microsoft Phi-3 Mini, 3.8B - **June 2024:** Anthropic Claude 3.5 Sonnet - 67.2% - **2025:** - DeepSeek-R1, Distill-Qwen 1.5B - Alibaba Qwen3, 1.7B - Alibaba Qwen3, 4B - NVIDIA Nemotron 3 Nano - Alibaba Qwen3.5, 4B - **2026:** - OpenAI gpt-oss-20B - Alibaba Qwen3.6, 35B-A3B - 86% Phi-3 and Llama GPQA, subsequent models GPQA Diamond. Published results, benchmark and test conditions vary. ### 00:05:34 # On-device voice capability - Moonshine Small Streaming on Raspberry Pi 5 CPU: **527 ms average** from detected endpoint to final transcript. - Comparable average WER to ### 00:06:04 # On-device voice capability - Moonshine Small Streaming on Raspberry Pi 5 CPU: **527 ms average** from detected endpoint to final transcript. - Comparable average WER to Whisper Large (2022) across seven English benchmarks. - ASR component latency; endpoint detection, reasoning and speech output are additional. ## Human-to-human conversation turn taking delay [Grid of line graphs showing turn transition percentages over response offset for languages including Danish, Italian, Dutch, Tzeital, +Akhoe Haillom, English, Yell-Dnye, Lao, Korean, Japanese] [Horizontal bar chart showing response delay ranges in milliseconds for languages including Japanese (7 ms-511 ms), Tzeital (67 ms-519 ms), Yell-Dnye (71 ms-442 ms), Dutch (109 ms-585 ms), Korean (183 ms-689 ms), English (236 ms-756 ms), Italian (310 ms-852 ms), Lao (420 ms-908 ms), +Akhoe Haillom (423 ms-1085 ms), Danish (469 ms-1076 ms)] ### 00:06:31 # Connected apps + - Business tools, data - workflows ### 00:07:10 THE OPPORTUNITY # Why put intelligence at the edge? ## Data center - **Frontier AI capability** - Beyond device memory - and compute budgets - **Scaled compute** - Share capacity for heavier tasks - **Connected apps + services** - Access tools, data - and workflows ## Edge AI On device or nearby - **Efficiency** - Match compute to the task - **Customization** - Adapt models to your domain - **Local context** - Run inference near the data source - **Privacy** - Control what you share [Diagram showing a data center icon connected by an arrow labeled "Selected work + context" to a central "Edge AI" chip icon, which has four radiating lines pointing to "Efficiency", "Customization", "Local context", and "Privacy".] ### 00:07:33 # AI device innovation is accelerating ## Accessible hardware - Logos for Raspberry Pi, Edge Impulse, NVIDIA Jetson, and Seeed Studio. - Compute platforms - Sensors - Radios - Cameras - Batteries - Hardware modules - A decade of crowdfunded wearables, 3D printers, drones, headsets, and robots. - arm ## Efficient open-weight AI models - Logos for Llama, Hugging Face, Gemma, and Qwen. - Language - Vision - Speech - Reasoning - Tools - Open weights - The world is only just beginning to bring these into devices... [Four small circuit boards/modules] [Four small robot-like figures] ### 00:08:04 # AI device innovation is accelerating ## Accessible hardware - Raspberry Pi - Arduino - NVIDIA Jetson - Seeed Studio - Compute platforms · Sensors · Radios - Cameras · Batteries · Hardware modules A decade of crowdfunded wearables, 3D printers, drones, headsets, and robots. ## Efficient open-weight AI models - Hugging Face - Gemma - Qwen - Language · Vision · Speech - Reasoning · Tools · Open weights The world is only just beginning to bring these into devices... [Four small circuit boards/modules, including a Raspberry Pi camera module] [Four small, colorful robot figures] ### 00:08:37 # Hugging Face x Pollen Robotics Reachy Mini ## Antenna actuators - **SERVO**: 2 x Dynamixel XL330-M077-T - **MCU**: Arm Cortex-M0+ - **CLOCK**: 64 MHz ## Raspberry Pi CM4 - **CPU**: 4 x Arm Cortex-A72 - **MEMORY**: 4 GB RAM - 16 GB eMMC - **RADIO**: 2.4 / 5 GHz Wi-Fi ## Head movement - **SERVOS**: 6 x Dynamixel XL330-M288-T - **MCU**: Arm Cortex-M0+ - **CLOCK**: 64 MHz ## Base rotation - **SERVO**: 1 x Dynamixel XC330-M288-PQ - **GEARING**: Custom plastic gears [Illustration of a robot with lines pointing from components to their specifications listed in surrounding text boxes] ### 00:09:04 # Hugging Face x Pollen Robotics Reachy Mini ### Antenna actuators - **SERVOS**: 2 x Dynamixel XL330-M077-T - **MCU**: Arm Cortex-M0+ - **CLOCK**: 64 MHz ### Raspberry Pi CM4 - **CPU**: 4 x Arm Cortex-A72 - **MEMORY**: 4 GB RAM - 16 GB eMMC - **RADIO**: 2.4 / 5 GHz Wi-Fi ### Head movement - **SERVOS**: 6 x Dynamixel XL330-M288-T - **MCU**: Arm Cortex-M0+ - **CLOCK**: 64 MHz ### Base rotation - **SERVO**: 1 x Dynamixel XC330-M288-PG - **GEARING**: Custom plastic gears [Diagram of a transparent robot (Reachy Mini) with lines connecting its components to specification boxes] ### 00:09:37 ## BUILDING THE SYSTEM # Reachy Mini app development ## Reachy Phone Home A focus companion with local vision, voice and motion NVIDIA GTC Golden Ticket winner: Marco Domingo - Arm ### 00:10:07 # arm [Diagram showing a grid background with the word "arm" rotated vertically, connected to several abstract geometric icons (star, hexagon, multi-colored diamond) by lines, with ### 00:10:38 # Right-sized inference Meet task quality within compute, energy and memory budgets. ## Fast, frequent tasks -> Richer inference when needed ### Vision - **Signals + Triggers** ### 00:11:06 # Voice: understand and react ## Raspberry Pi 5 - Reachy Mini Reactions - **User speaking** (Head turn <100 ms from VAD start) - **Silence** ### 00:11:42 # Voice: understand and react ## Raspberry Pi 5 - **User speaking** - Reacts from ~100 ms from VAD start - **Silence** - 200 ms minimum configured - **VAD end** - Timing starts: 0 ms ### Process - **Internal** - **Transcribe**: `Moonshine STT` - **Embed + match**: `MiniM - semantic cache` - **Audio** (Heard) - **Play reply**: `Pre-generated TTS` - **Motion** (Servo control) ### Timeline - **Processing duration**: 200-600 ms - **Reply preparation**: 50-150 ms - **Audio playback** ### 00:12:05 # Voice: understand and respond ## DGX Spark **530 ms** to first audio - User speaking (Before measured interval) - Silence (Detection delay not supplied) - VAD end (Timing starts: 0 ms) - **Endpoint wait**: Confirm the turn is finished - **300 ms** - **Transcribe**: Ziphomer - **8 ms** - **Generate**: Mistral 7.3B - **170 ms** - **Synthesize speech**: Piper - **73 ms** - **Audio playback** - **530 ms** ### Tools - **Camera snapshot + VLM**: Return visual context - **Dance library**: Pollen Robotics - recorded moves - **Web search**: Return search results [Diagram illustrating a timeline of voice processing steps with their respective latencies] [Image of a person interacting with a white robot] ### 00:13:34 [Screenshot of a video player showing a video at 0:01 of 0:53 with a progress bar.] ### 00:14:10 "REACHY, I'M DONE FOR TODAY" • AT HOME # Understanding intent through local context Better UX with fewer device interactions ## Voice command - Voice request ## Personalization - Voice request - Your bedtime routine - Time + room sensors ## Ambient Context - Voice request - Your bedtime routine - Time ### 00:14:37 # Understanding intent through local context ## Better UX with fewer device interactions "REACHY, I'M DONE FOR TODAY" • AT HOME ### Voice command → Personalization → Ambient Context ### 00:15:03 ```python set_ac(target=...) dim_light("yo...") ``` > "Your bedtime ready." ### 00:15:36 VISION Device state [Diagram showing two terms, "VISION" and "Device state", separated by a vertical line] ### 00:16:07 # Agentic embedded / IoT systems NASA Voyager 1 system simulation - Telemetry fault ## Read system health - 85 tokens `read_health()` - Commands still ### 00:16:34 # Agentic embedded / IoT systems ## NASA Voyager 1 system simulation - **Telemetry fault** ### Read system health - 85 tokens - `read_health()` - Commands still work, but both data streams are failing - Operating mode - Radio carrier - Power supply - Command: **available** - Attitude: **Holding** - Engineering: **Invalid** - Science: **Invalid** - Try repair - Model output: - Run - Present - Stable This pattern applies to a robot, an industrial device or a home appliance! [Illustration of the Voyager 1 spacecraft] [A simulated interface showing system health statuses with green "available" and "holding" indicators, and red "invalid" indicators for engineering and science] [A timeline progress bar] ### 00:17:07 # Command available - Play repair - ...e pattern a... - ...inspired simulatio... ### 00:17:34 ### Status - Command: **available** - Attitude: **Holding** - Engineering: **Invalid** ### Action - Play repair ### Context The pattern applies to a robot, an industrial... ...simulation - Recorded Qwen3.6-35B-A3B run - DG... [Screenshot of a user interface displaying status indicators, an action button, a timeline, and text related to a robot/industrial simulation.] ### 00:18:05 # Hybrid AI inference ### Routing strategies - **Embedding kernel** Embed request + weight labelled outcomes by similarity. - **Calibrated uncertainty** Full local answer + token margins + calibrated error risk. - **Decoder-state probe** Full local answer + final hidden state + predict correctness. Local model: Gemma 4.8B QAT Q4_0 - thinking on Spark GPU 400 requests: WebChat-IM (100) MATH-500 (50) MMU-Pro (80) BigCodeBench (120) Dominic Pope, AI Engineer Paris, September 2020 [Line graph showing success rate relative to cloud (%) versus tasks kept local (%), with lines for Calibrated uncertainty, Embedding kernel, and Decoder-state probe] ### 00:18:38 # Hybrid AI inference ## Routing strategies - **Embedding kernel** Embed request → weight labelled outcomes by similarity. - **Calibrated uncertainty** Full local answer → token margins ### 00:19:02 [Partial line graph with y-axis labels 0 and 90, and text labels 'Loc' and '400'] ### 00:19:37 # Different AI needs. One Arm platform ## Arm AI Portal For developers and their agents - Optimized models - Performance + accuracy data - Optimization tools (Early access) - Code + workflows - MCP server ## AI software ecosystem - **Frameworks + runtimes:** PyTorch, ExecuTorch, llama.cpp, LiteRT, ONNX Runtime - **Arm libraries + compilers:** CMSIS-NN, KleidiAI, KleidiCV, Arm Compute Library, Ethos-U Vela ## ALWAYS ON: Sense + control - Cortex-M CPU, Ethos-U NPU - AMBA interconnect - SRAM, I/O, Security Low power ## EDGE: Perceive + reason - Cortex-A CPU, GPU / NPU - System interconnect - Memory, ISP, Media Rich local context ## DATA CENTER: Frontier + scale - Neoverse CPU, AI accelerators - High-speed interconnect - Memory, Storage, Network Frontier AI capability [https://developer.arm.com/ai/early-access-program](https://developer.arm.com/ai/early-access-program) [Three block diagrams illustrating hardware and software components for different AI use cases: "Always On" (low power), "Edge" (rich local context), and "Data Center" (frontier AI capability)] ### 00:20:07 - Cortex-M CPU - SRAM [Block diagram showing system components, including a Cortex-M CPU and SRAM.] ### 00:20:35 # Arm SME2 Scalable Matrix Extensions - CPU instructions - Matrix acceleration ### 00:21:06 # Arm Runtime for Ultralytics YOLO26 on Raspberry Pi 5 ## Optimization results - **1.87x** faster inference - YOLO26 - 4 ### 00:21:38 # More resources [developer.arm.com/ai](developer.arm.com/ai) LET'S BUILD LET'S CONNECT Dominic Pajak, Arm [ ### 00:22:03 # LET'S BUILD [QR code] ### 00:22:36 - NVIDIA - AI Engineer Paris - Cockroach Labs - Mistral - FACTORY - SENTRY - Sonar - CLOUDFLARE - Daytona - langfuse - Modal - neo4j - Qdrant - CodeRabbit - cominty - descope - Latitude - FIMO - MAKORA - stripe - ZenML labs - Backblaze - Notion - SIERRA [Grid of company logos, including NVIDIA, AI Engineer Paris, Cockroach Labs, Mistral, Cloudflare, Daytona, Langfuse, Modal, Neo4j, Qdrant, Sentry, Sonar, CodeRabbit, Cominty, Descope, Latitude, Fimo, Makora, Stripe, ZenML Labs, Backblaze, Notion, and Sierra] ### 00:23:07 - NVIDIA - AI Engineer Paris (repeated frequently) - Cockroach Labs - Mistral - FACTORY - SENTRY - Sonar - neo4j - CL ### 00:23:39 - Backblaze - Sonar - Mistral - AI Engineer PARIS - Notion - NVIDIA - comiety [Collection of company and event logos] ### 00:24:07 - NVIDIA - Cockroach Labs - Mistral - FACTORY - SENTRY - Sonar - CLOUDFLARE - Daytona - langfuse ( ### 00:24:35 ### AI ENGINEER PARIS 2026 # Closing the Distance ## Agentic MapReduce for large-scale coding tasks Yannis Psorakis, Head of Field Engineering EMEA CO ### 00:25:04 # We are an applied AI research lab Founded by a team of leading AI researchers and competitive programmers, with 20 IOI gold medals - **Scott Wu** Co-Founder & CEO - **Walden Yan** Co-Founder & CPO - **Steven Hao** Co-Founder & CTO - **Russell Kaplan** Co-Founder & President **GLOBAL PRESENCE** - San Francisco - Austin - New York - D.C. - Tokyo - Singapore - São Paulo - London **SIZE OF TEAM**: 450+ **FUNDING**: $4B+ **VALUATION**: $48B [Four headshots of the founding team members] ### 00:25:38 # We build Devin, the first cloud-based AI coding agent Devin now spans multiple product surfaces, and agentic SDLC personas ## LOCAL - **Devin Desktop**: IDE + ### 00:26:06 # Devin is deployed across the most sophisticated SDLCs - Citi - Mercedes Benz - Wayfair - ServiceNow - Rivian - AT&T - Goldman Sachs - Dell - Lowe's - Intact - Elevance Health - Philips - U.S. Army - Santander - BNY - U.S. Navy - Fox - NU - Anduril - Itaú - Cisco - Modal - Athenahealth > Devin is an absolutely amazing tool > — Kevin Scott, CTO Microsoft [Grid of 23 company logos, including Citi, Mercedes Benz, Wayfair, ServiceNow, Rivian, AT&T, Goldman Sachs, Dell, Lowe's, Intact, Elevance Health, Philips, U.S. Army, Santander, BNY, U.S. Navy, Fox, NU, Anduril, Itaú, Cisco, Modal, and Athenahealth] [Small profile picture of Kevin Scott] ### 00:26:31 ### Consider two types of agentic coding tasks - "Localized" ### 00:27:08 # Consider two types of agentic coding tasks ## "Localized" - eg implement a feature, write tests against some module, optimize a query, create boilerplate, etc - concerns work localized in a bounded area of the codebase ### 00:28:04 ### Consider two types of agentic coding tasks #### "Localized" - eg implement a feature, write tests against some module, optimize a query, create boilerplate, etc - concerns work localized in a bounded area of the codebase - relevant code lives in a handful of files; agent is pointed at them, or performs a straightforward search #### Codebase-wide - eg: security scanning, extract common logic, breaking-change detection, etc ### 00:29:00 # Consider two types of agentic coding tasks ## "Localized" - eg implement a feature, write tests against some module, optimize a query, create boilerplate, etc - concerns work localized in a bounded area of the codebase - relevant code lives in a handful of files; agent is pointed at them, or performs a straightforward search ## Codebase-wide - eg. security scanning, extract common logic, breaking-change detection, etc - complex interplay between components ### 00:29:36 # Consider two types of agentic coding tasks ## "Localized" - eg implement a feature, write tests against some module, optimize a query, create boilerplate, etc - concerns work localized in a bounded area of the codebase - relevant code lives in a handful of files; agent is pointed at them, or performs a straightforward search ## Codebase-wide - eg. security scanning, extract common logic, breaking-change detection, etc - complex interplay between components - the result is only trustworthy only if *the entire codebase is considered* ### 00:29:59 # Challenges with codebase-wide tasks ## 001 / THE PROBLEM ### 01 Finding the work Repeated searching and backtracking consume the budget available for investigation. ### 02 Maintaining context Discoveries from different parts of the repository compete for attention. ### 03 Knowing when to stop The agent needs an explicit account of what it has examined and what remains.