# Your Automation Is Overthinking It — Chibi Vikram, Kevin Bernstein — session 2026-11-11T22:48:00.000Z → 2026-11-11T23:01:00.000Z _31 transcript lines · 25 slides_ ## Transcript Gets its own instance and gets all the resources. So assuming we have thousand customers out there, we had to maintain like virtual classifiers across all these thousand customers, which becomes a pretty difficult problem statement in itself which takes up a lot of time from glean engineers. So looking at the results over here, the overall recall of the first phase where we used to have a virtual classifier, we replaced that with JEV, went from 31.5% to 90.2%. At the same We are seeing a significant reduction in cost. Now this cost is only the resource cost mentioned over here, but we are also saving cost on the engineering resources over here. We don't have to invest, like, two or three engineers who are full time maintaining those virtual classifiers or those T4 GPUs across customers. The improvement or the savings that we have are far more than 50%. This slide concludes all the results I have talked about till now. At Glean has been able to influence the product experience of the users, has been able to impact how we as developers have been working at Glean. And most importantly, it also has a very good positive impact on the security security that we provide to our customers. A simple rule that we have been using is can the out in conclusion, a simple rule that we have been using at glean is can the output be listed in advance? If no, then it's a free form generative text problem statement. Go back to something that can. Generate free form text. If yes, then use jev. Are labels stable and volume high? If this is the case where the labels are stable, which is rarely the case anymore, you can also fine tune a model. But if the labels change, which in my opinion is quite frequent these days, then, again, use something like JEV. Obviously, if you need dynamic text, we can, like, feel free to try out more hybrid approaches like JEV along with free form text, LLMs. This ends my presentation. So thank you everyone for listening. I have added a blog, like a link to the blog post where we have posted the exact same results in great detail. Feel free to catch me on the outside if you have more questions. Thank you. Hello, everyone. My name is Sidi, and Joined stage by Kevin. We are here to talk about how we believe that, your automations are overthinking it. So, to tell us a little bit about ourselves, we are from a company called UiPath. We have been doing automation for the last twenty years. We have been doing, RPA before, and, we service more than 10,000 organizations around the around the world. And we build, an agentic process orchestration layer, which basically brings Agents, robots, and people together to interact with each other and, build extremely scalable, automations that run for days and weeks. So we were inspired by this particular slide that you're seeing right now, and you saw today today before us as well. When, Diego presented it three months ago, the same slide, it resonated really well with us because, this is how we see automation as well. So that Traditional software was... Is basically RPA. It's just robotic process automation. It fails, just execute some task and then keeps going. The control flow is deterministic. Just keep going. And then when the LMS came out, we all raced to go and build the just in time software where we have this one agent. We give it a bunch of tools. We give it a task and ask it to finish. And then we were seeing good results, bad results, and then we added evaluations to slowly build the results up. And Then finally, we landed on agentic process orchestration, which kind of weaves in this cognitive decision through a logical, decision layer. So you have deterministic calls and clear cognitive decision points where you would use an LLM to control the control flow rather than leaving it all in the hands of an agent. So at this time, we we saw where Jowood fit in. And when, Diego was talking about Jowood, we were really excited. Because, we saw Jevo fit in in a very particular niche area. So, here we see that Jevo is a very powerful tool. As Diego was saying, it's a foundational tool that we would be able to integrate into all automation, and it fit right in between a deterministic case and an agent case. So Jev is extremely fast, and most of you guys would have tried it and we were surprised by how fast and How parallel we could make it work and, and how intelligent it was. So we were able to benchmark it around how good it was compared to an LLM. And it was pretty good, not as a full LLM, but it was pretty good at... In terms of intelligence as well. So it falls in between, and, there are so many ways we could use Jell. And, I wanna hand it off to Kevin to take you take, take you guys through a scenario where, we use Jell in, UiPath Maestro flow. Yeah. Thanks, Jeev. So Everybody. Today, I wanna actually walk us through a little bit about what UiPath is building with UiPath Maestro Flow and talk through a little bit of how we would go through and coordinate humans, robots, and agents all together inside of one canvas. What we're gonna walk through today is essentially, you know, essentially a flow that will demonstrate how we can weave all these things together. And the way that we're approaching this is that we're gonna end up having system one agents, which are gonna be able to make quick intelligence happen within our process. Think about things like, hey, is this transaction fraud or not? Additionally, we'll have, system two agents running where we're gonna have more of a methodical, deeper investigation that's going on where agents are going off and calling tools. And we'll see kind of this pattern... These patterns start to emerge where we're able to hand off from one to the next. And so with that, I'll I'll jump into an actual demo where we're gonna demonstrate a account fraud, scenario. And so our demo starts here with Effectively, a transaction coming in for $48,000. And this wasn't something that we were expecting. And so, as I'm reviewing my bank portal here, I'm gonna go ahead and report this as a fraudulent transaction. And so now this would end up submitting it off into our UiPath Maestro flow where this would act as essentially the trigger for this end to end workflow to to go off and start processing. What What we're looking at is a collection of various different nodes that are each gonna kind of, be orchestrated. From the left side of the canvas over to the right. And we're gonna use, essentially a combination of system one and system two models to actually process this, and do analysis if this is fraud or not. And so the first thing that actually takes place is we're gonna go through and get account activity. We're gonna send this off to Jeff to basically pull 60 transactions from the most recent, credit card purchases that were made and analyze each one of those individually, whether or not these are fraud. And this is happening in real time where Jeff's able to process This... These transactions and ask that question of, hey. Is this fraud or not? Within fractions of a second. Now we found that there's about four transactions that recently appeared that it thinks are fraud. Now in parallel to that, we're able to kick off another part of our flow, which is doing more of a semantic analysis on the entire account. We're looking at that one transaction that came in for $48,000, which was flagged. And we're asking a question to Jeff is, hey. Is there enough information here for us to determine whether or not this is actually fraud? And just as we heard Diego talk about this earlier where you have these new kind of paradigms and logic come up, we ended up, finding from this that there's, a scoring that took place, and we needed more information. We were asking the question, do we have enough information for fraud? And for that, what we're gonna end up doing is kicking off a system two agent that's gonna go spin up and run for much larger... Longer calling a bank API and finding that, you know, there's a $48,000 transaction from a payee, basically Creating a new account on his phone, and it was earlier that day that that account got created. So high indicator of fraud. We circle back and ask that same question now to Jeff and ask, hey. Do we have enough evidence now? Jeff agrees, and it finds a 100% sufficient activity. And we would hand that off effectively to another grading model that combines all those transactions that we found with the system payload and then finds that, you know, there is fraud in this particular use case. Square to, you know, one to five, we got 4.97. Now Stepping out of this, you know, typically what would happen from this point onwards is that we'd go off to a human to review. We'd have somebody kind of review both the system one and system two logs and then find that there was, you know, potential fraud and freeze the account. But, you know, effectively, what we're seeing from customers that are going through and implementing JEV inside of their workflows is that they're able to replace the typical LLM calls that are going on behind the scenes. And then in process, we're saving about 50, times, we're we're increasing the speed about 50 times. While also being 7.1, times cheaper, than calling a traditional LLM. Now we... We've gone through and and kinda shown how customers would use this within our technology, but we're also pointing the finger back inwards where we're trying to ask the question of how can UiPath use this within the products and services that we offer to be able to help customers out. And there's a number of different patterns that we're kind of exploring with Jev. And today we're gonna talk about, a re ranking problem as well as We're using inside of email mining. And so to set the stage for for kinda re ranking, effectively, what we're trying to do is run an ad within our retrieval augmented generation within our agents the ability for us to find more appropriate chunks to be able to provide to our agent. And we do that by basically finding relevant documents first and then using, typically an LLM to do this. But we're experimenting with also using JEV to go through and find the various different chunks that make... Are most relevant. Things got really interesting for us. This is, like, the first week of Jell. We had... We got our hands on Jell. And, we have these extremely strong benchmarks and, eval sets that we had for re ranking that we have been running for some time. ## Slides ### 00:00:08 # Guardrails: Jev tripled first-pass recall at about the same precision Every prompt, tool output, and response passes a fast first-pass check; an LLM makes the final call on ### 00:00:35 # Guardrails: Jev tripled first-pass recall at about the same precision Every prompt, tool output, and response passes a fast first-pass check; an LLM makes the final ### 00:01:39 # One decision model, used across product, evals, and security | Area | Decision | Quality vs baseline | Speed / cost | Verdict | | :------------------- | :------------------ | :------------------------------------------------ | :-------------------------------- | :------ | | **Product experience** | Model routing | Higher precision on every route | 8.1x faster (median) | Shines | | | Suggestion ranking | More suggestions surfaced, fewer timeouts | 7x faster ranker p ### 00:02:08 # A simple rule for when to reach for a decision model - **Can the output be listed in advance?** - No → keep a generative model (queries, explanations, tool arguments ### 00:02:40 # Generate when you must. # Decide when you can. ## Thank you - Chinmay Goyal - chinmay.goyal@glean.com - Read more: gle ### 00:03:05 # Code Summit ## AI Engineer ### 00:03:37 # AI Engineer Code Summit ### PRESENTED BY Google DeepMind ## // AVAILABLE # LIVE TRANSLATION Scan the QR code to access real-time translation. [QR code linking to https://translate.google.com/] ### 00:04:01 # AI Engineer Code Summit ## STAY IN THE LOOP SCAN THE QR CODE OR VIEW THE SCHEDULE AT [AI.ENGINEER/SCHEDULE](https://ai.engineer/schedule) [https://ai.engineer/schedule](https://ai.engineer/schedule) [QR code linking to https://ai.engineer/schedule] ### 00:04:36 [Abstract pattern of grey dots with a central lighter concentration] ### 00:05:12 # Your Automation Is Overthinking It ## Chibi Vikram / Engineer UiPath ### UiPath Overview - UiPath is the leader in agentic process orchestration - 20+ years automating ### 00:05:36 # Your Automation Is Overthinking It Chibi Vikram / Engineer UiPath [QR code linking to an unknown URL] ### 00:06:08 # What inspired our talk ### Posttraining & Midtraining > Diogo Almeida, "Dev CEO: I made ChatGPT, now I'm building what's next", AI Engineer World' ### 00:06:36 # What inspired our talk ## Posttraining & Midtraining - **Traditional Software** - Explicit paths written in advance - **Just-in-time Software** - AI ### 00:07:08 # Where Jev fits ## AUTOMATION BUILDING BLOCKS ### Comparison of Automation Approaches | Characteristic | IF / ELSE, SWITCHES (Deterministic) | A TYPED JUDGMENT ### 00:07:36 ### AUTOMATION BUILDING BLOCKS # Where Jev fits | Category | IF / ELSE, SWITCHES (Deterministic) | A TYPED JUDGMENT (Jev SYSTEM 1) ### 00:08:08 # Jev inside UiPath Maestro Flow ## SYSTEM 1 JEV ### Quick intelligence > "The delivery record is missing." A fast, focused judgment: one of the answers you defined ### 00:08:40 # Jev inside UiPath Maestro Flow ## Quick intelligence ### SYSTEM 1 JEV "The delivery record is missing." A fast, focused judgment: one of the answers you ### 00:09:08 # Your Automation Is Overthinking It Kevin Bernstein / Solution Architect UiPath ### Report unusual activity [Screenshot of a banking application showing a "Report unusual activity" modal dialog] ### 00:09:40 # Your Automation Is Overthinking It ## Kevin Bernstein / Solution Architect UiPath ### UiPath Studio: ReviewAccountActivity.flow - System 1 - Jev typed answers, with probabilities ### 00:10:09 # Your Automation Is Overthinking It ## UiPath Studio Workflow: ReviewAccountActivity - **Process:** Every transaction · in parallel - **Performance:** 60 checks ### 00:10:40 # Your Automation Is Overthinking It ## Kevin Bernstein / Solution Architect UiPath ### UiPath Studio Workflow: ReviewAccountActivity.flow Account analysis - up to 3 rounds - ** ### 00:11:08 # Your Automation Is Overthinking It Presented by Kevin Bernstein / Solution Architect UiPath ## UiPath Studio: ReviewAccountActivity Workflow ### Account analysis - up to 3 rounds - ** ### 00:11:40 # Account Fraud Analysis - Demo ## Your Automation Is Overthinking It Kevin Bernstein / Solution Architect UiPath [Screenshot of a UIPath Studio workflow diagram showing steps for account fraud review] ### 00:12:08 # Patterns of using Jev One focused question. An answer software can use. ## RERANKING ### Select Relevant to the query? ## MINING ### Assign ### 00:12:41 # Reranking: which passages go to the LLM **QUERY**: "Can the customer dispute a late-delivery charge 40 days after the invoice?" 1. **THE QUESTION