# Why Wall Street Still Doesn’t Trust AI — Vinoo Ganesh — session 2026-10-14T18:30:00.000Z → 2026-10-14T18:48:00.000Z

_43 transcript lines · 53 slides · source: full recording_

## Transcript

**[00:00:11 · 0]** Hi, everyone. My name is Vinu Ganesh. Excited to be here today. I'm the CEO and co founder of Kepler. This is gonna be kind of an interesting talk because it comes from what happened three months ago.

**[00:00:24 · 0]** So three months ago, I think there's a little bit of an overlap, but I gave a talk at AIE World Fair, really around verification, and why that most institutions need verification in some form of the trust ecosystem of AI. Well, yesterday, most of you saw Jack and I were on stage talking about why Jack, who ran Citadel's biggest book and started Candlestick Capital, still doesn't trust AI. Since that three month period, there's been about a hun... $270,000,000 of VC money that got poured into formal verification layers. A VC actually took Kepler's own pitch deck and gave it to a competitor and produced something that has our same tagline.

**[00:01:09 · 0]** I'm not bitter, just a little. But the funny thing is they copied the wrong part. And so what I want to do is actually give you the practitioners today. You can't just tack verification on at the end. And so, when I gave that chat or that talk a few, months ago, Anthropic had just published an article on how Kepler does what it does and why it's really powerful and and really unique.

**[00:01:32 · 0]** So what I wanna do is get into the technical side of how we do the thing that we do. And ultimately this is driven by one very simple insight. Wall Street still doesn't trust AI. Most people think that's because of this. AI will estimate things.

**[00:01:47 · 0]** It will hallucinate things. A frontier model will make stuff up. And ultimately, it is, and that's a component of it. But the actual core underlying problem is we have a structural problem in this technology that we're all so desperate to use. Meaning, if you actually chat with people that are building rag platforms, most of what they'll say is, well, the models will eventually get good or get better to the point that they will eliminate hallucination.

**[00:02:12 · 0]** That is a nonsensical term. You know another word for hallucination? Alpha. Alpha is how a firm makes money. The creativity that you have and that your analysts have is a mechanism through which you can be a unique firm.

**[00:02:27 · 0]** Otherwise, there should be only one company on Earth. So when you ask these types of simple questions around where data comes from, there is a piece of this that the model will just make things up. And there's a piece of this that your firm's internal data is never actually reflected in this setup. So what are we seeing right now? We're seeing everyone with a checkbook buy everything they can.

**[00:02:48 · 0]** Every rag platform, some firms are trying to build, some form... Some firms are running agent pilots. I had a a customer meeting this morning where someone said they were actually an AI first firm, which means they have one co pilot agent running all this stuff. So why aren't we not seeing the complete switch of fundamental investing becoming effectively, why are we not all Jane Street? Why why has that not happened yet?

**[00:03:13 · 0]** And so the challenge really comes from this idea of in order for you to produce work product that you believe in, you have to trust it, meaning you have to blame somebody who gets it wrong. So we see numbers like this all the time, where 80 of finance execs trust numbers, but then 31% find those errors. So what do we actually need? We need a situation where the reviews that we're getting for AI apps, as builders ourselves, do more than just ragify things. And these are quotes we've actually pulled from specific, I don't know, I don't want to say our competitors, but other people, where things demo super well, and they behave like a really smart intern.

**[00:03:54 · 0]** Or things are right if you just hook up the right structured information, but that last 10% still gets things wrong. So what I want to do is, we're lucky, we have great partners and folks like TurboPuffer, and I gave a similar chat yesterday about this, but how do we actually do the verification? How do we architecturally build a system that does not hallucinate numerically? So let's talk about it. Now, wrong numbers or wrong facts come from three sources.

**[00:04:21 · 0]** Either one, you retrieved the wrong document. So I pulled a document, some random thing from somewhere else. How do you fix this? Well, by retrieving the right document. But that in itself is complicated.

**[00:04:34 · 0]** Is the right document my own firm's internal analysis of what this stock is gonna be worth? Is it, let's say, some analyst who I really trust more than anyone else? That's piece one. Now, failure two. We get the right source, and we pull the wrong number out of that source.

**[00:04:50 · 0]** Let's say we're trying to figure out what the revenue for, I don't know, Palantir was back in 2023. Well, if I look at the twenty twenty five ten ks that says Palantir's 2023 revenue was this, I better make sure that that number corresponds to 2023 and not 2025. So the fix here is really we have a filter that actually checks each citation. But this is still a problem because that filter only applies after the fact. And formal verification doesn't actually help here because the thing has already been produced.

**[00:05:21 · 0]** Three is the mental math. Models don't do math. They predict the next token. Now, for those of you who are gonna say we have deterministic models, great, we do. We also have one CPU cycle that's good at adding two numbers together.

**[00:05:34 · 0]** Why don't I use that CPU cycle? So third failure is, let's use deterministic tools that we already have to actually compute numbers and entities rather than relying on some external tool. So here's what we do. And this is the magic behind Kepler, at least for piece one. There's a whole bunch of other stuff.

**[00:05:53 · 0]** We are architected fundamentally differently. Code writes every number. And what code does and pre processing does is we assign pointers to unique identifiers. Meaning facts. Meaning something like a number in a 10 ks.

**[00:06:07 · 0]** This operates in a giant pre processing and contextualization step. Such that all of this exists going from unstructured data to a structured data store. Then the model is effectively a smart analyst. It figures out how to go about solving the problem, but not actually being the thing to solve the problem itself. So it plans.

**[00:06:27 · 0]** It says, I'm gonna go here and retrieve this source and retrieve this entity and do this thing. Code and math do the retrieval. By actually pulling a number out of a 10 ks at com... Like a search time, that's a hugely expensive token cost. And Anthropic and OpenAI should thank you for paying all those tokens and token maxing.

**[00:06:46 · 0]** But most of your firms don't like that, at least anymore. I know there are people getting bonuses from token maxing for a while. Now, the next piece of this is really once the model has actually read both these pointers, it can't structurally generate numbers. Meaning unless the model actually has a mechanism of dereferencing that pointer, that number or that fact cannot be generated in Kepler's output. Now, we do run a post processing step, which is the verification step.

**[00:07:14 · 0]** And this is the piece that was kind of funny, where the last company just copied the verification piece, but all the structural stuff beforehand didn't make it. So what you as a consumer actually see is you see something like the revenue was this amount to this amount, and you physically can click on these chips and see exactly how we got the information. Now, in my past life, I had an opportunity. I was head of business engineering at Citadel. So I worked across, the firm to figure out effectively what do we need to produce software that helps us generate alpha.

**[00:07:45 · 0]** You know the crazy thing with Citadel? PMs are so talented and so smart, but they work when they're paired with deterministic tools and technology. That's kind of the magical insight behind fundamental investing. Fundamental investors so good at understanding data, but that data has to be right. And so when you build a system like Kepler, what you effectively have to do is look up the right information and then be clever about the information you don't have.

**[00:08:13 · 0]** So let me give you the parallel. Most folks and most buy side researchers or buy side investors try to use tooling or build DCFs on their own, and then they find and search for information they don't have. They schedule GLG or guide point calls. They find experts. They speak to management.

**[00:08:30 · 0]** The same system actually applies here. So what we do is we pick the path, and we store the information deterministically that we can actually store deterministically. When we have information we can't store deterministically, we run similar preprocessing steps to grab information, store that information in a hybrid storage mechanism, which is a combination of turbo buffer, which is awesome by the way, and a bunch of other, mechanisms. And then we ultimately use a combination of re ranking with literally old school, like pointer dereferencing, like dot get methods to pull things into our system. Now, why this matters is a lookup cannot get it wrong.

**[00:09:11 · 0]** Search can. If you're running the same 10 searches with the same query, the probabilistic nature of models means there's inherent entropy. That's actually an opportunity. So that's what we've changed here. Now, how we do this is we break this down and this occurs across the SEC world as well.

**[00:09:31 · 0]** We are frequently throwing candidate documents away. Meaning, we consistently have to look at not only what source is the right source for what information, but also which documents should we stop looking at kind of globally. So, we're an ANN plus BM25 to get back a set of documents, usually up to 200. We do reciprocal. We effectively get 60 documents back, probably the best way to think about this.

**[00:09:56 · 0]** Then we do cross encoders. We try to figure out exactly the query and the actual document chunked together and get those results that way. What that allows us to do is we can search really wide. You know what LLMs aren't good at doing? Searching really wide.

**[00:10:14 · 0]** What they try to do is create a plan and find either the fastest or easiest breadth first search mechanism, actually, depth first search mechanism of actually, like, getting the first document and moving forward. So we allow this to search really wide and then narrow from there. The reason LLMs don't like searching wide is because it's expensive. It's because looking across a bunch of different sources is significantly worse than getting the reward function back with your first hit and going from there. Okay.

**[00:10:45 · 0]** So, I want to talk a little bit about security here because it's super important. Kepler was built with a security first mindset and security first mentality. Now client documents and this is specifically targeted towards the builders here this is a thing that no one's chatting about. No one is actually talking about the security of what OAuth token is used and what user scope scope mechanism of retrieving documents, which is actually pretty scary. So let's talk about how things go wrong in these systems that people are trying to build and how you can fix it.

**[00:11:15 · 0]** First and foremost, you forget to filter the tenant. All of a sudden, Citadel can search for all of Millennium's documents, you will be in a lot of trouble. Second, a caller claims to be another firm, meaning I can claim that I'm impersonating this other entity and querying documents that way. Third, search returns information that's out of date. We've had this problem kind of globally in even Google search results.

**[00:11:38 · 0]** Fourth, Direct Read skips the search rules, meaning you're not actually querying the full corpus of information, or there's not an allow list. You only get back the single document that you did the search. Five, you hit an index that isn't populated. The way to actually solve all these problems, and there's individual things listed here, is actually scoping searches with security built in. And that's kind of another critical piece that I think a lot of people miss when they're trying to build these systems themselves.

**[00:12:05 · 0]** Inevitably, people will start caring about the security associated with querying this information. Alpha is the most elusive thing out there, especially for folks on Wall Street. So making sure you have a good mechanism of actually constraining the set of documents that come back with each query is really important. Okay, now let's talk about verification, which is the last piece here. This is probably what we're known most for, but we do the other stuff I mentioned as well.

**[00:12:32 · 0]** How we do this is by pre processing information and storing it deterministically. We have systems like code that actually write individual pointers. The model can only then read those pointers and grab information back if they exist in the database. If I grab something out of a hash map that doesn't exist, I don't get anything back. And so this combination is what allows us to actually make sure the information that we retrieved actually came from the source that it came from.

**[00:13:01 · 0]** Now, this is a kind of hidden version of what our, source IDs look like, but these are hashes. And these are very simple things that you can do in your own system and your own setup to ensure that you're pulling back facts that in fact do exist. Cool. So that's a little bit about how we do verification. Now, I want to chat a little bit about what this enables, kind of, globally.

**[00:13:26 · 0]** This enables us to verify facts that come out of our grounded corpus of data. Now, you as a builder, or you as a startup creator, or you as someone who is trying to figure out, how do I trust my outputs, really just need to do a couple of things. You need to show people that as you're creating these agents that are theoretically looking for alpha or whatever they'd be looking for, you're using information that is actually relevant and concrete, and most importantly, accurate to the set of people that you're trying to service. So we end up with a contract where citations are either verified, corrected at, search time, or flat out refused. This is crazy if you think about it.

**[00:14:09 · 0]** If a model cannot de resolve that pointer, it cannot print that number out. So that's kind of an insane thing. It means that the whole and not just a number, let's talk about facts. Like, we see these court cases all over of precedents being hallucinated. Well, why can't we create a system that just has the precedents stored deterministically, and allows us to search through that list and not have a random court case that never existed all of a sudden start existing?

**[00:14:37 · 0]** So this this this mechanism and this technology actually becomes fairly globally applicable. And the reason I'm on stage telling you how we do what we do is because I'm terrified that Wall Street never trusts this technology, because of the fear that so many other companies or so many build it at home folks have been, instituting in these organizations. It's it's gotten to the point where the vast majority of finance AI tools right now are being used to format PowerPoint presentations. We still haven't actually built a DCF cradle to grave, despite what most people are saying. And if we had, every single company on Wall Street would have adopted that overnight.

**[00:15:16 · 0]** So this is my fear. My fear comes from a very simple fact. For us to actually get value out of this technology, we need to be able to build technology that people actually can trust and can verify. That isn't just true in finance, it's true in a bunch of different industries. So please take what you need to from this presentation and get people using this technology.

**[00:15:36 · 0]** Otherwise, we're all gonna be out of a job. Now, I wanna chat through what happened when we saw this. So when we went through this whole process, this is actually some of our real production data here. Refusals are when the model does not... Is not able to produce a number.

**[00:15:54 · 0]** Meaning, the model will say, I couldn't dereference this pointer, therefore, I could not produce this number. We saw that drop super drastically. Now the reason that it didn't drop to zero is because models are still probabilistic, meaning it can still make up numbers after the fact during inference. So there is another pre pro... A post processing step after this that we we have as well.

**[00:16:17 · 0]** But my point with this slide is, just by creating a system where the models have to dereference their own pointers, you can actually create a very powerful, very verifiable AI platform. And that, to me, is my ask to all of you. As you go through the building process, as you go through the scoping process, any tool that you use, ask them if you can actually verify the numbers. Ask them if you can see where the sources are, not in terms of some after the fact annotation, but click on the number, whether it's a calculation, whether it's an extraction, or some combination of the two to make sure you trust the insight. And all of this ultimately is going to be the unlock.

**[00:16:56 · 0]** So you as a vendor or you as a builder ask these questions. Does the model actually write pointers? Does it actually create a mechanism for verifying the output? Is every citation source checked? Is arithmetic source checked?

**[00:17:10 · 0]** Do Do you have a mechanism for stopping files that reach clients or customers that it shouldn't reach? And most importantly, what happens to a number that came from nowhere? What happens when a when a fact comes from nowhere? And how do I actually handle these kinds of of hallucinations? So this whole platform is actually to make a very simple statement, very obvious, which is, we can have the smartest models in the world.

**[00:17:38 · 0]** We can hit some crazy cool piece of technology. But we still have the same database problems that we've had from the beginning, which are I cannot if I search through a row and that column, that value doesn't exist, I won't get a retrieval back. We're now out of the search phase of AI. We need to be building infrastructure. We need to be

## Slides

### 00:00:00–00:00:20

# AI Engineer
## NEW YORK

### 00:00:20–00:00:40

# Why Wall Street still doesn't trust AI.

How Kepler retrieves, computes and verifies every figure

### Vinoo Ganesh
CEO & Co-founder, Kepler

AI ENGINEER • NEW YORK
kepler.ai

### 00:00:40–00:01:00

## Why Wall Street still doesn't trust AI.
How Kepler retrieves, computes and verifies every figure

AI ENGINEER • NEW YORK

Vinoo Ganesh CEO & Co-founder, Kepler
kepler.ai

### 00:01:00–00:01:20

# Why Wall Street still doesn't trust AI.
How Kepler retrieves, computes and verifies every figure

Vinoo Ganesh CEO & Co-founder, Kepler
kepler.ai

### 00:01:20–00:01:40

AI ENGINEER • NEW YORK
## Why Wall Street still doesn't trust AI.
How Kepler retrieves, computes and verifies every figure

Vinoo Ganesh CEO & Co-founder, Kepler
kepler.ai

### 00:01:40–00:02:00

# Why Wall Street still doesn't trust AI.
## How Kepler retrieves, computes and verifies every figure
Vinoo Ganesh CEO & Co-founder, Kepler
kepler.ai

### 00:02:00–00:02:20

# "Honestly, I estimated those."

A frontier model, asked where the share counts came from in the dilution model it had just built from a 10-K.

Its first answer had

### 00:02:20–00:02:40

# Why Wall Street Still Doesn't Trust AI

Vinoo Ganesh / CEO Kepler

"Honestly, I estimated those."

A frontier model, asked where the share counts came from in the dilution model it had just built from a 10-K.
Its first answer had looked perfect.

**1 in 3** analyst tasks the best model gets wrong. It never says which.

Source: Vals AI, Finance Agent v2 leaderboard, Oct 12, 2026.

> where did you get your RSU and PSU number
>
> Honestly, I estimated those — I couldn't get the RSU/PSU activity table to render from the SEC filing either. Same problem as the options table. Do you have that section of the 10-K as well? If you can share it I'll plug in the actual numbers.
>
> You're right to call that out — my estimates were wrong. The actual 10-K data is right here:
> **Options outstanding as of December 31, 2025: 127,679,804 at a weighted-average exercise price

### 00:02:40–00:03:00

# Why Wall Street Still Doesn't Trust AI

## "Honestly, I estimated those."

A frontier model, asked where the share counts came from in the dilution model it had just

### 00:03:00–00:03:20

### THE PROBLEM
## Every firm is trying. It works until numbers matter.

- We rolled out a chat assistant.
  Great for emails. **Nobody trusts it with a number.**
- We connected it to our files.
  It read 20 files. **The answer was in file 4,312.**
- We ran an agent pilot.
  It ran 40 steps. **Step 3 was wrong. Nobody noticed.**
- We built our own.
  New models ship every week. Workflows break. **We patch instead of building.**

**88%** of finance execs trust AI numbers in reports without checking them.
**31%** found AI errors had already reached investors or the board.

Source: Workiva, 2026 MidYear Executive Benchmark Survey.

### Why Wall Street Still Doesn't Trust AI
Vinoo Ganesh / CEO & Kepler

### 00:03:20–00:03:40

# Why Wall Street Still Doesn't Trust AI

### THE PROBLEM
## Every firm is trying. It works until numbers matter.

- We rolled out a chat assistant.
  - Great for emails.
  - **Nobody trusts it with a number.**
- We connected it to our files.
  - It read 20 files.
  - **The answer was in file 4,312.**
- We ran an agent pilot.
  - It ran 40 steps.
  - **Step 3 was wrong. Nobody noticed.**
- We built our own.
  - New models ship every week. Workflows break.
  - **We patch instead of building.**

- **88%** of finance execs trust AI numbers in reports without checking them.
- **31%** found AI errors had already reached investors or the board.

Source: Workiva, 2026 MidYear Executive Benchmark Survey.

Vinoo Ganesh / CEO Kepler

### 00:03:40–00:04:00

# Why Wall Street Still Doesn't Trust AI
## Vinoo Ganesh / CEO Kepler

### THE PROBLEM
## 500 reviews: nobody trusts the numbers.

> "[It

### 00:04:00–00:04:20

# Why Wall Street Still Doesn't Trust AI

## THE PROBLEM
### 500 reviews: nobody trusts the numbers.
- "[It] demos like a dream and delivers like

### 00:04:20–00:04:40

### WHY IT HAPPENS
# Wrong numbers come from three jobs we keep handing the model.
Find the source, copy the number, do the math. A wrong number looks exactly like a right one, so Kepler takes all three jobs away from the model.

## FAILURE 1
**Wrong source**
Retrieval missed the document, so the model filled the gap with a number that sounds right.

### FIX
**Retrieval finds the source**
Filings are read as tagged facts and client files are searched in turbopuffer. 3.0M filing facts are citable.

## FAILURE 2
**Right source, wrong number**
The document was found, but the scale ($M vs $B), the unit ($ vs shares) or the copying slipped.

### FIX
**A filter checks every citation**
Each cited number is compared with its source as the answer streams. 32,128 were corrected or refused.

## FAILURE 3
**Mental math**
Growth rates, margins and multiples were worked out in the model's head.

### FIX
**Code does the math**
Compute tools do the arithmetic and register each result. 32.7% of all citations are calculations.

Source: Kepler production event logs, Apr-Jul 2026.

---
### Why Wall Street Still Doesn't Trust AI
Vinoo Ganesh / CEO ♾️ Kepler

### 00:04:40–00:05:00

# Wrong numbers come from three jobs we keep handing the model.
Find the source, copy the number, do the math. A wrong number looks exactly like a right one, so Kepler takes

### 00:05:00–00:05:20

### WHY IT HAPPENS
# Wrong numbers come from three jobs we keep handing the model.
Find the source, copy the number, do the math. A wrong number looks exactly like a right

### 00:05:20–00:05:40

# WHY IT HAPPENS
## Wrong numbers come from three jobs we keep handing the model.
Find the source, copy the number, do the math. A wrong number looks exactly like a right one, so Kepler takes all three jobs away from the model.

### FAILURE 1: Wrong source
Retrieval missed the document, so the model filled the gap with a number that sounds right.

### FIX: Retrieval finds the source
Filings are read as tagged facts and client files are searched in turbopuffer. 3.0M filing facts are citable.

### FAILURE 2: Right source, wrong number
The document was found, but the scale (**

### 00:05:40–00:06:00

### WHY IT HAPPENS
# Wrong numbers come from three jobs we keep handing the model.
Find the source, copy the number, do the math. A wrong number looks exactly like a right one

### 00:06:00–00:06:20

### HOW KEPLER WORKS
# Code writes every number and pointer. The model reads them.
"What was Apple's FY2025 revenue, and how fast did it grow?" Every step sits in one of two lanes, and only code touches a number.

**MODEL** (Language only)
- 1 Plan: Needs FY25 and FY24 revenue, then percent change.
- 4 Read: The model reads both pointers into its prose. It types no number.

**CODE** (Every number)
- 2 Retrieve: Tagged 10-K fact: Total net sales 416,161 ($M).
- 3 Compute: Percent change = 6.4%, stored under a hash-derived id.
- 5 Verify: Filter matches each marker to its registered value.

READER SEES Revenue was **$416,161M** up **6.4%** from FY2024. Click a chip to open the filing line.

[Flowchart diagram showing a five-step process divided into two lanes: MODEL and CODE. Steps 1 and 4 are under MODEL, while steps 2, 3, and 5 are under CODE, with arrows indicating the flow from Plan to Retrieve, Compute, Read, and Verify.]

### 00:06:20–00:06:40

## HOW KEPLER WORKS

# Code writes every number and pointer. The model reads them.

> "What was Apple's FY2025 revenue, and how fast

### 00:06:40–00:07:00

# HOW KEPLER WORKS
## Code writes every number and pointer. The model reads them.
> "What was Apple's FY2025 revenue, and how fast did it grow?" Every step sits in one of two lanes, and only code touches a number.

### MODEL
Language only
- **1 Plan**
  - Needs FY25 and FY24 revenue, then percent change.
- **4 Read**
  - The model reads both pointers into its prose. It types no number.

### CODE
Every number
- **2 Retrieve**
  - Tagged 10-K fact: Total net sales 416,161 ($M).
- **3 Compute**
  - Percent change = 6.4%, stored under a hash-derived id.
- **5 Verify**
  - Filter matches each

### 00:07:00–00:07:20

# HOW KEPLER WORKS
## Code writes every number and pointer. The model reads them.
> "What was Apple's FY2025 revenue, and how fast did it

### 00:07:20–00:07:40

## HOW KEPLER WORKS

# Code writes every number and pointer. The model reads them.

> "What was Apple's FY2025 revenue, and how fast

### 00:07:40–00:08:00

# Look up what is tagged. Search only what isn't.
Job 1, finding the source. Kepler picks the path by how the data is shaped, and ranks only where there is nothing

### 00:08:00–00:08:20

# RETRIEVAL - FINDING THE SOURCE
## Look up what is tagged. Search only what isn't.
Job 1, finding the source. Kepler picks the path by how

### 00:08:20–00:08:40

### RETRIEVAL - FINDING THE SOURCE

## Look up what is tagged. Search only what isn't.

Job 1, finding the source. Kepler picks the path by

### 00:08:40–00:09:00

RETRIEVAL - FINDING THE SOURCE
# Look up what is tagged. Search only what isn't.
Job 1, finding the source. Kepler picks the path by how

### 00:09:00–00:09:20

# Look up what is tagged. Search only what isn't.

## RETRIEVAL - FINDING THE SOURCE

Job 1, finding the source. Kepler picks the path by how the data

### 00:09:20–00:09:40

### RETRIEVAL - FINDING THE SOURCE
# Look up what is tagged. Search only what isn't.
Job 1, finding the source. Kepler picks the path by

### 00:09:40–00:10:00

# RETRIEVAL - FINDING THE SOURCE

## Each search pulls up to 200 candidates and hands the model 10.

Each stage throws candidates away, so the first

### 00:10:00–00:10:20

### RETRIEVAL - FINDING THE SOURCE
# Each search pulls up to 200 candidates and hands the model 10.
Each stage throws candidates away, so the first stage fetches wide on purpose. Missing the document costs the whole answer.

- **Up to 200**
  - **ANN + BM25 in turbopuffer**
    Both run at once, 100 each (3x top_k), zembed-1 embeds the query first.
- **100**
  - **RRF fusion, k = 60**
    The two ranked lists merge by rank, not raw score.
- **20 docs**
  - **Best chunk first**
    At most 5 chunks per document, so one file can't crowd out the rest.
- **10**
  - **zerank-2 cross-encoder**
    Reads query and chunk together and reorders. Up to 25 when asked.

About 4.5 s per search: embed 0.2 s, turbopuffer + Postgres re-check ~2.0 s, rerank ~2.3 s.
**Lesson:** one wide rerank beat several narrow ones, and a bigger pool is the cheapest recall gain.
Source: Kepler engineering measurements.

[Flow diagram showing a reduction from "Up to 200" candidates down to "10" through several stages]

### 00:10:20–00:10:40

### RETRIEVAL - FINDING THE SOURCE
# Each search pulls up to 200 candidates and hands the model 10.
Each stage throws candidates away, so the first stage fetches wide on purpose. Missing the document costs the whole answer.

- **Up to 200**
  - **ANN + BM25 in turbopuffer**
    - Both run at once, 100 each (3x top_k), zembed-1 embeds the query first.
- **100**
  - **RRF fusion, k = 60**
    - The two ranked lists merge by rank, not raw score.
- **20 docs**
  - **Best chunk first**
    - At most 5 chunks per document, so one file can't crowd out the rest.
- **10**
  - **zerank-2 cross-encoder**
    - Reads query and chunk together and reorders. Up to 25 when asked.

About 4.5 s per search: embed 0.2 s, turbopuffer + Postgres re-check ~2.0 s, rerank ~2.3 s.
**Lesson:** one wide rerank beat several narrow ones, and a bigger pool is the cheapest recall gain.
Source: Kepler engineering measurements.

[Diagram showing a funnel or filtering process, starting with "Up to 200" candidates and narrowing down to "10"]

### 00:10:40–00:11:00

### RETRIEVAL - FINDING THE SOURCE
# Each search pulls up to 200 candidates and hands the model 10.
Each stage throws candidates away, so the first stage fetches wide on purpose. Missing the document costs the whole answer.

- **Up to 200**: **ANN + BM25 in turbopuffer**
  - Both run at once, 100 each (3x top_k). zembed-1 embeds the query first.
- **100**: **RRF fusion, k = 60**
  - The two ranked lists merge by rank, not raw score.
- **20 docs**: **Best chunk first**
  - At most 5 chunks per document, so one file can't crowd out the rest.
- **10**: **zerank-2 cross-encoder**
  - Reads query and chunk together and reorders. Up to 25 when asked.

About 4.5 s per search: embed 0.2 s, turbopuffer + Postgres re-check ~2.0 s, rerank ~2.3 s.
**Lesson**: one wide rerank beat several narrow ones, and a bigger pool is the cheapest recall gain.
Source: Kepler engineering measurements.

[Funnel diagram showing a reduction from "Up to 200" candidates down to "10" through several filtering stages]

### 00:11:00–00:11:20

### RETRIEVAL - FINDING THE SOURCE
# Five ways one client could see another's files, each closed in code.
Client documents sit in a shared search index, so we listed the ways a leak could happen and closed each one in code.

| HOW A LEAK COULD HAPPEN                 | WHAT CLOSES IT                                                                                                                            |
| :-------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------- |
| 1. A developer forgets the tenant filter | One filter builder adds it to every query. Org namespaces carry an `org_id` predicate; a personal namespace is itself the boundary.       |
| 2. A caller claims to be another firm   | Org and user come from the signed token and are stored on the chat. Only auth code can create a tenant scope.                             |
| 3. Search returns a deleted or unowned file | Every hit is re-checked in Postgres (tenant, ownership, not deleted) before the model sees it.                                            |
| 4. A direct read skips the search rules | Search and read share one allow-list, so a read can't return what a search would hide.                                                    |
| 5. A query lands on a half-built index  | Index rebuilds go v1 -> v2 with an atomic flip, so queries only ever see a complete index.                                                |

### 00:11:20–00:11:40

### RETRIEVAL - FINDING THE SOURCE
# Five ways one client could see another's files, each closed in code.

Client documents sit in a shared search index, so we listed

### 00:11:40–00:12:20

# Five ways one client could see another's files, each closed in code.
Client documents sit in a shared search index, so we listed the ways a leak could happen and closed each

### 00:12:20–00:12:40

### RETRIEVAL - FINDING THE SOURCE

# Five ways one client could see another's files, each closed in code.

Client documents sit in a shared search index, so we listed the ways a leak could happen and closed each one in code.

| HOW A LEAK COULD HAPPEN                 | WHAT CLOSES IT                                                                                                                            |
| :-------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------- |
| 1. A developer forgets the tenant filter | One filter builder adds it to every query. Org namespaces carry an `org_id` predicate; a personal

### 00:12:40–00:13:00

### VERIFICATION · CHECKING EVERY NUMBER
# Code writes the pointers. The model only reads them.
Job 2, copying the number. Tools write a pointer to each value, and

### 00:13:00–00:13:20

### VERIFICATION - CHECKING EVERY NUMBER
# Code writes the pointers. The model only reads them.
Job 2, copying the number. Tools write a pointer to each value,

### 00:13:20–00:13:40

VERIFICATION • CHECKING EVERY NUMBER
# Code writes the pointers. The model only reads them.
Job 2, copying the number. Tools write a pointer to each value, and a filter

### 00:13:40–00:14:00

VERIFICATION - CHECKING EVERY NUMBER
# Every citation ends up verified, corrected or refused.
The filter always tries to repair a citation before giving up. Here is one example of each outcome.

### 00:14:00–00:14:20

### VERIFICATION · CHECKING EVERY NUMBER
# Every citation ends up verified, corrected or refused.
The filter always tries to repair a citation before giving up. Here is one example of each

### 00:14:20–00:14:40

### VERIFICATION · CHECKING EVERY NUMBER
# Every citation ends up verified, corrected or refused.
The filter always tries to repair a citation before giving up. Here is one example of each outcome.

## VERIFIED
### Exact match
The model cites $416,161M and the filing fact says 416,161 ($M).

READER SEES
**$416,161M**
Shown as written, as a green chip.

## CORRECTED
### Right number, wrong form
The model cites $416.2B. The value matches

### 00:14:40–00:15:00

### VERIFICATION - CHECKING EVERY NUMBER
# Every citation ends up verified, corrected or refused.
The filter always tries to repair a citation before giving up. Here is one example of

### 00:15:00–00:15:20

VERIFICATION - CHECKING EVERY NUMBER
## Every citation ends up verified, corrected or refused.
The filter always tries to repair a citation before giving up. Here is one example of each outcome

### 00:15:20–00:15:40

# Every citation ends up verified, corrected or refused.
The filter always tries to repair a citation before giving up. Here is one example of each outcome.

## VERIFIED
### Exact

### 00:15:40–00:16:00

# Refusals fell from 116.6 to 1.4 per 100 sources in eight weeks.
In production, a refusal is the system catching a citation it

### 00:16:00–00:16:20

# Refusals fell from 116.6 to 1.4 per 100 sources in eight weeks.
In production, a refusal is the system catching a citation it couldn't prove. The rate fell as prompts and tools caught up.

## Refusals per 100 registered sources, by week, 2026

### 10,981 REFUSALS, IN THREE GROUPS
- **3,810 Wrong numbers**
  Value didn't match, number not in the quote, fact never shown, missing cell, wrong unit.
- **5,341 Quotes that didn't check out**
  Quote not in the document (4,542), or passed unverified and counted as a failure (799).
- **1,830 Broken markers**
  Id no tool registered, malformed or nested marker, raw id leaked into prose.

Refusals and registrations are counted as separate events, so the ratio can exceed 100. Source: Kepler production event logs, Apr-Jul 2026.

[Line graph showing refusals per 100 registered sources decreasing from 116.6 on May 4 to 1.4 on June 29, with an annotation for "Citation framework v2 ships mid-May"]
[Box summarizing 10,981 refusals categorized into "Wrong numbers", "Quotes that didn't check out", and "Broken markers" with their respective counts and descriptions]

### 00:16:20–00:16:40

## Refusals fell from 116.6 to 1.4 per 100 sources in eight weeks.

In production, a refusal is the system catching a citation it couldn't prove. The rate fell as prompts and tools caught up.

### Refusals per 100 registered sources, by week, 2026

### 10,981 REFUSALS, IN THREE GROUPS
- **3,810** Wrong numbers: Value didn't match, number not in the quote, fact never shown, missing cell, wrong unit.
- **5,341** Quotes that didn't check out: Quote not in the document (4,542), or passed unverified and counted as a failure (799).
- **1,830** Broken markers: Id no tool registered, malformed or nested marker, raw id leaked into prose.

Refusals and registrations are counted as separate events, so the ratio can exceed 100. Source: Kepler production event logs, Apr-Jul 2026.

## Why Wall Street Still Doesn't Trust AI
Vinoo Ganesh / CEO Kepler

[Line graph showing refusals per 100 registered sources decreasing from 116.6 on May 4 to 1.4 on June 2

### 00:16:40–00:17:00

# Refusals fell from 116.6 to 1.4 per 100 sources in eight weeks.
In production, a refusal is the system catching a citation it couldn

### 00:17:00–00:17:20

### TAKEAWAYS
# Five questions to ask before you trust an AI number.
You can use these on any AI tool, including ours. The last row is where our own guarantees stop.

### 00:17:20–00:17:40

# Five questions to ask before you trust an AI number.

TAKEAWAYS
You can use these on any AI tool, including ours. The last row is where our own guarantees stop.

| ASK THE VENDOR                                                  | KEPLER'S ANSWER                                                                                             |
| :-------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------- |
| 1. Does the model write pointers, or only read what tools wrote? | Reads only. Tools write every pointer.                                                                      |
| 2. Is every citation checked against its source before I see it? | Yes, in the stream: 21,147 corrected and 10,981 refused.                                                   |
| 3. Who does the arithmetic and lays out the tables?             | Code. 32.7% of all citations are calculations.                                                              |
| 4. What stops my files reaching another client?                 | Typed tenant scopes, plus a Postgres re-check on every hit.                                                 |
| 5. What happens to a number nobody retrieved?                   | Not caught by verification. 12.8% of final-answer numbers were uncited; we log them and measure recall. |

Source: Kepler production event logs, Apr-Jul

### 00:17:40–00:18:00

# Why Wall Street Still Doesn't Trust AI

## Smarter models won't close the trust gap.
**Infrastructure will.**

Vinoo Ganesh / CEO Kepler

[https://