# Autocomplete for Actions: System One from the UI to the DB — Scott Roberts — session 2026-11-11T23:24:00.000Z → 2026-11-11T23:37:00.000Z _48 transcript lines · 26 slides_ ## Transcript Hey, Hey, everyone. I promise I'm the last, person on the hype train for Jev. I'm Scott. I'm from, I'm from Workday, and I'm gonna go through how we're experimenting with Jev, to drive, new innovations in Are in finance tasks. Exciting stuff. So, at Workday, we have over 11,000 different organizations that use our tools. And a lot of the problems that we deal with break down into kind of two different categories. So one is very quick actions they have to take. I want to see my pay. I want to ask for a day off. The other ones are I have to dive into a ton of data, weed through it, try to figure out what's going on. And like many of your companies as well, we're Trying to, you know, put agentic workflows around this. And so if you look at just a a very simple, agentic loop, you know, we have a user request. We try to decide what to do. We're running tools. We're judging, did we do a good job? And then over and over again until we finally get a result. Now one of the problems here is that on each one of those, we're doing LLM calls. And there's two major problems with that. So one is there's too much intelligence. So if we look at the System one versus system two thinking. We're doing a lot of slow, deliberate, very expensive thought processes at each one of those steps. Another problem is it's just the wrong tool for the job. So, you know, LLMs a lot of times are just a hammer, looking for a nail. And, you know, we wanna figure out, like, what do we actually need, because clearly, LLMs are not doing it. So we want something that is fast. Smart, easy to use, it's generalized, and it's very, very cheap. So when we look at, like, from a code perspective, you know, what are we really looking at from a, from a JEV like model? It's really just a conditional. If this happens, this should happen. Which one of these cases are the ones that I want to explore? And, you know, you've seen it among many of these presentations already, but Jev has the primitives to allow us to do this. There's the probability. Is it, you know, this or that, and what's the probability? What are the different categories that this, particular thing could fall into? And then you have the, the scoring mechanism, which is basically probability and sorted. So, you know, this has been brought up as well. You know, my first thought when I saw, you know, the JEV models was, well, we already have classifiers. I've been doing this for a long time. How is it actually different? And as many of the speakers have talked about already, you know, with classifiers, I need to figure out, you know, what's my golden dataset, I have to train it, new data comes out, I have to iterate over and over. With Jev, just out of the box, it just worked. We plugged in our question, we classified it. We didn't really have to do anything. It generalized to our question. And so we started using it for a lot of the use cases that you've seen from some of the presenters. Already. So we use it for routing. That's kind of the canonical example. You know, figuring out did this step work, you know, as a judge. We looked at, you know, resumes. You know, is this person qualified for ML work? And I wanna do kind of an overly simplified example as well. So no one taught Jev like what a fish is. And so this is actually a question we ask, like, is a fish a pet? Well, fifty fifty, might be a pet. Sometimes might not be. And so, you know, if I ran into this, then I have to figure out, what do I do next? Maybe I can give it some more context. Oh, is efficient a bowl a pet? You know, and after twenty milliseconds, I can ask this again. Alright. That's great. Oh, is a fish on my plate a pet? Clearly not. So this is something that wasn't trained out of the box in seconds. It just worked. All right. So now I want to some of the demos that were do... That, You know, we've done at Workday. So just a caveat on some of these. It's not real data, so we're not using any of your resumes or payroll data. These are also not in production now and may or may not be in the future. So that's something our legal asked me to say. So let's get into, you know, some of the, the demos. Alright. So starting with autocomplete. So autocomplete starts to predict what you're trying to ask. It's just predicting the quest. It's not helping you get to the answer. So as you start to type more, it's gonna guess like, oh, I want to figure out if this person wants to take a day off work. Great. You've guessed the question, but you really haven't, you know, helped me get to the answer. So what we want to do is move from just predicting the word or the next block of code and actually help you finish the task. And so this is best shown with a demo that we have where we go from predicting what you want to say. To predicting what you wanna do. So hopefully the video plays. There we go. Alright. So I need to take time off of off of, next week. And as I start to ask this, Jeff is actually figuring out under the hood, okay, what what more context do I have? How can I adjust this UI based on the information that I'm that I'm getting? This is not something that we could have done with an LLM. We're getting, you know, we're we're doing this, you know, in a loop. While someone's asking. A payslip. You know, you shouldn't have to go in and ask an agent, hey, can you go find out how much I was paid? What you can do is just in the search, say, what was my paid last period? Pops up an interface, allows you to unhide it, you know, see what your pay was in the last period. Very simple. Gets you to the action. Book a meeting. So I know that I wanna book a meeting. And, I'm gonna wanna do it for next Friday. Let me do it at 2PM. Oh, I wanna do it with my boss. Make sure that I get my, you know, my boss on this, Logan McNeil. There we go. Meeting's booked. And then a final one that we have just for this example, but there's many more, is finding a person. So in this one, I may have some semantic knowledge, like find the person who owns this part of the organ Or just like, who's Ashanti? I wanna contact her, pop it up, click on email, good to go. Alright. So what's happening under the hood? So because dev is fast and cheap, just like a type ahead, we could start to render a UI on this... On that same cadence. We can look at, as the words are coming, do a prediction of intent, and start to render an interface. And, you know, we could do this at, you know, about $20 for a million judgments. And the way that we're doing it is kinda twofold, or two parts, I should say. So first, Jeff is picking the card of saying, what's the intent? All right, there's a card that's gonna help you out, and then the team that's responsible in that domain then can decide what are the questions to progressively fill out that card. And So there's sort of this two step process. The first step is getting people in the front door, getting the right Card up, and then the second one is making sure that you fill in that card to help them accomplish the task that they're after. Now we have an eval on here before we merge any card. We wanna make sure that it doesn't collide with any other one. And at least in our initial experiments, you know, we've we've done seven cards or a just chat because it should, if there's no card to actually solve it, it should allow you to just continue chatting. We were able to get, you know, 96% accuracy on that. Alright. So now on to the recruiter. So let's talk about the recruiter problem. Alright. So this is not like a fast, task. It's something where recruiter has a pile of hundreds of resumes and has to figure out who has the best resume to accomplish, or fill the rack that they're trying to fill. Now, I first wanna look at, like, how we would have done this before. We would have done, you know, an embedding, put the vectors in a vector DB, then had a retrieval layer, then have a Anchor and then figure out, based on the re ranker, which ones that we wanna put in front of, the recruiter. One interesting call out is the Quinn re ranker, looks surprisingly similar to, the, the dev process as well. And so that's kind of an interesting corollary if you've used that one before. But with JEV, we basically take out all the embedding steps. I don't have to train anything up. I don't have to do embedding. I don't have to put things in a vector. Database. What I can do is just do a function on my database. So put all the resumes into a database, use natural language to query it, iterate over every single one of them, and then just figure out which ones match my criteria. So, I'll go into an actual demo of this exact same one. So here we have a select statement. And so this is against a DuckDB. We have a function that is gonna call jev to figure out what's the probability that someone matches this criteria. We're gonna get it a score, and then we're gonna order by that score. Now I'm gonna do about 20. And as I said, these are all fakes and none of your resumes are in here. I promise. And then here we see, these are very, ML focused, synthetic data as well. So it's not a surprise that we would have, very high hit rates against someone who's worked in fraud, trust, and safety problems. But let's look at it, let's change this question. Let's be a little bit more adversarial on this one. So a lot of these resumes have the word pipeline in them. So whether it's a data pipeline or an AI pipeline, pipe's gonna come out. So let's just ask, like, is this person qualified to do plumbing? If you were gonna just do a lexical search, it might pick up some of these. So we put it into dev and asked, you know, of all these ML resumes, let's trim the limit down to Five here. Go ahead and run it. And we can see, you know, that very, very low probability on these. So these are ones that we would almost definitely throw out. And you can see the speed. So we went over 117 records. We had two point three seconds of wall time and spent almost nothing on this query to just iterate over everything. So Jeff really kind of changed the game for us because every every place where we're thinking is either this Is too expensive or it's too slow. We thought of new places to inject JEV into our pipeline. And so just to summarize, you know, the two areas that I went through are, one, where can we decide very quickly and get to the intent and get the person on their way? And then the second one is to cut down on some of the, you know, really deep work. How do we filter those out for, you know, our customers? And so I hope this... That this gave you a little bit of insight into the Things that you can take back to, you know, your own companies and how... And, you know, imagine the things that you can do with Jevon decision models as well. And that's my time. Thank you very much. ## Slides ### 00:00:24 # AI Engineer Code Summit ## PRESENTED BY Google DeepMind ### //AVAILABLE # LIVE TRANSLATION Scan the QR code to access real-time translation. [QR code for real-time translation] ### 00:00:55 # AI Engineer Europe 2027 ## ExCeL London - **Dates:** February 17-19, 2027 - **Description:** Europe's flagship technical AI conference returns to ExCeL London for three days, February 17-19, 2027 — bringing the world's leading AI labs, platforms, and tools to Europe's builders. - **Website:** [ai.engineer/europe](ai.engineer/europe) [Black and white photos of London landmarks including the London Eye, Big Ben, and a historic street scene.] ### 00:01:21 # AI Engineer Code Summit - Pinecone - Sentry - Render - OpenAI - Anthropic - Google DeepMind - AWS - Atlassian - ### 00:01:53 # Workday ## Two kinds of work, across more than 11,500 organizations ### QUICK, MANY TIMES A DAY - See my pay. - Take next Wednesday ### 00:02:22 # The basic agent loop - User request - **DECIDE**: Which tool next? (LLM call today) - Tool runs - **JUDGE ### 00:02:54 # Problem 1: too much intelligence **SYSTEM 1** - Fast, automatic, cheap. **SYSTEM 2** - Slow, deliberate, expensive. > Every gold ### 00:03:25 # In code, it's a conditional ## IF/THEN ```python # every harness is full of these if ???: continue_plan() elif ??? ### 00:03:58 # In code, it's a conditional ## IF/THEN ```python # every harness is full of these if ???: continue_plan() elif ???: retry_with_refined_query() else: escalate_to_llm_review() ``` ## SWITCH/CASE ```python match ???: case "time_off": open_card() case "pay_slip": open_card() case "find_person": open_card() case _: just_chat() ``` ### JEV HAS A PRIMITIVE FOR EACH **NOUL** fills the if if `noul(step, q) > 0.8`: ### 00:04:19 # An artificial general classifier ## TRADITIONAL CLASSIFIER ### One question per model - Collect labeled examples - Run a training job - Retrain when the question changes ## JEV ### Any question, written as text - No labels, no training run - Ask a new question at runtime - A typed answer with a probability ### 00:04:53 # An artificial general classifier ## Traditional Classifier ### One question per model - Collect labeled examples - Run a training job - Retrain when the question changes ## JEV ### ### 00:05:21 # Nobody trained it on fish "Is a fish in a bowl a pet?" ## 1 ### QUESTION Is a fish a pet? No ownership or situation given **0 ### 00:05:51 # Autocomplete for Actions ### 00:06:26 ## Autocomplete for words - Search query: `i want to take next week o` - Autocomplete suggestions: - off - off from work - ### 00:06:54 # AI Engineer Code Summit ## Presented by Google DeepMind ### Request time off ### Keynote Scott Roberts (Workday) [Screenshot of a calendar UI for requesting time off, ### 00:07:26 # Workday AI Assistant Interface ## Good afternoon, Betty - **Search/Input:** `+ what` - **Suggested Actions:** - Help me send an email ### 00:07:54 # AI Engineer Code Summit ## Presented by Google DeepMind ### New meeting ## Keynote ### Scott Roberts Workday [Screenshot of a meeting scheduling interface, showing a new ### 00:08:26 # A judgment on every word I need to take time off next week Jev answers again at every word, or after a short pause. - **55-105 ms** ### 00:08:57 # Two questions, then the right screen ## Jev Picks Card 1. **User types** "I need to take time off next week on Wednesday for jury duty" ### 00:09:22 # Built so teams don't break each other ## Two Steps The front door asks one shared question. Each team owns the questions on its own card. ## Eval for ### 00:09:51 # Now the recruiter A pile of resumes and a shortlist to build ### 00:10:21 # Retrieval three ways: Qwen, Cohere and Jev ## QWEN - Embedding - Vector database - Similarity search - **RERANKING** - ### 00:10:54 # Natural language inside the query ```sql scratch.sql :read in jev_shell.py • 117 RESUMES, MADE-UP NAMES SELECT substr(name, 1, 22) AS candidate, round(jev_noul(resume_body, 'Has this person worked on fraud, or trust and safety problems?' ), 2) AS score FROM pool ORDER BY score DESC LIMIT 20; ``` - a question in plain English - No classifier to train. No index to build. ## AI Engineer Code Summit PRESENTED BY **Google DeepMind** ### Keynote Scott Roberts Workday [https://www.workday.com/](https://www.workday.com/) LIVE TRANSLATION ### 00:11:25 # AI Engineer Code Summit Presented by Google DeepMind ```bash scott.roberts@AMQD3F02G21 resume-demo % ../.venv/bin/python ### 00:11:53 AI Engineer Code Summit PRESENTED BY Google DeepMind ## Demo: change the question ```sql -- Live demo scratchpad. Edit the question below, save, then type .read ### 00:12:24 # AI Engineer Code Summit Presented by Google DeepMind ## Live Demo: Jev + DuckDB Resume Screening Shell ```sql -- Live demo scratchpad. Edit the question below, ### 00:12:52 **QUICK WORK** Decide intent quickly and solve the problem. **DEEP WORK** Cut large amounts of data down easily, with plain-language questions anyone can write. Scott Roberts · Workday