# Autoresearch for Kernels — Tejas Bhakta — session 2026-07-01T21:50:00.000Z → 2026-07-01T22:10:00.000Z _17 transcript lines · 10 slides · source: full recording_ ## Transcript And we have this reevaluate print loop REPL, that the agent is interacting with in the original paper that's Python. And the RLM is instructed to operate symbolically on that prompt. So Don't just read the whole thing into your context window. Explore it symbolically. And, even more, you don't even directly explore it symbolically or maybe you do a little bit of poking around. Hi, everyone. I'm Tejas. So I'm gonna be explaining how we make models three times faster with auto research. Previous to this, I actually used to do GPU mining in my dorm room with ten eighty c i's all the way up to working at Tesla on inference optimization for Tesla AI. But first, what is auto research? So auto research is this framework from Andre Karpathy where, you basically set up a framework for an agent to move towards a goal that you define. And all you have to do, basically, is say at the high level what you want it to do, and it will try things as it goes and move back and forth, towards that goal. In actuality, it's really just a while loop. The agent proposes a solution. You have a set up to define what's correct, benchmark it for us. And then you keep or revert that. And you do this in a loop until your goal is met. And so this is very well aligned to GPU kernels. So if you don't know what a GPU kernel is, it's basically a low level operator. In an NVIDIA GPU, this is a CUDA kernel. And this is an operator that's used by the GPU to operate millions of times in parallel. For example, like a matrix multiply or an expert computation. You can verify them for correctness and speed, and that's basically all you need for your auto research framework. So in actuality, there are some caveats here. The auto research framework is really good for, like, picking block sizes and these tiny parameters. But they're also still really bad at the high level idea. Like seeing, I want to use this GPU, and I actually want to pipeline it. And why are GPUs such a good fit for auto research? It's because they're super verifiable. It's not going to come up with these groundbreaking ideas. So it's still up to the human to do that. But the actual implementation is very straightforward once you Once you have the idea laid out. So it is still your job to have good ideas, is what I'm saying. And so the actual secret formula here is, you have the good ideas. Auto research picks out the parameters and everything to verify that it actually works. And moves toward that verifiable goal of it being x times faster and still correct. And you mix that with billions of tokens of your favorite model. And that results in kernels that beat hand tuning. So what are the actual things you care about when you're when you're when you're writing a custom kernel, or you're having your agent write a custom kernel? The three main things you can have are a compute bottleneck, a memory bottleneck, or you just have excessive overhead from, too many kernels being launched. And you can do you can view these things with by profiling with a profiler like NSYS, for example, which is that NVIDIA's profiler. And so this page looks super daunting. But basically, your job as the human is to look at the top here and be like, this is dumb. We are loading 32 ks chunks into context. And we don't actually need to for this deep seek attention, for example. Be doing it every 32 ks instead. So at a high level, all you have to be telling auto research is, this top method is dumb. Let's pipeline it instead. And everything else, like the sizing, the chunk sizing, the context chunks, that all should just be decided by auto research. And so my problem is that I really love cheap GPUs. And so that means like GPUs that don't have ND link, for example, is an example of GPUs you can get for cheaper. But the problem is you don't actually have kernels off the shelf for those. So you have to come up with a auto research framework, as well as a custom harness. So what goes into the harness to make this really good? So one thing you really need to make sure your agent is aware of is the hardware. And so on a b 200, for example, you need to make sure it has context of, the warps it has, t m, t m a. And so if you don't know what these are, these are just low level operators that you have on a specific hardware. And this changes generation to generation. Like H200 won't have TMEM, for example. That's a new feature that came out with V200, which is why you need to have this in context. And so this basically is just like a bunch of MD files in your GIF. So it has context. ## Slides ### 00:00:19 # RECURSIVE LANGUAGE MODELS ## Context itself is the object of computation - **Externalize** — the full prompt lives in a REPL, not the context window. - ### 00:00:50 # AI Engineer ## World's Fair [Background grid of various company and event logos, including "World's Fair"] ### 00:01:18 - Browserbase - WorkOS - docker - aws - neo4j - trust - together.ai - LanceDB - Google DeepMind - Amazon AGI ### 00:01:53 ## what is a GPU Kernel? autoresearch for kernels | @tejasvbhakta Engineering the future of AI [Diagram showing the hierarchy of CUDA components: a CUDA thread forms a CUDA thread block, which forms a CUDA kernel grid. These map to a CUDA core, then a CUDA streaming Multiprocessor (SM), and finally a CUDA-capable GPU.] ### 00:02:17 # Good for sweeps, bad at ideas ### GOOD AT - smoothing corners - picking sizes - tuning parameters ### BAD AT - novel ideas - pipelining - unusual tricks autoresearch for kernels | @tejasvbhakta ## Engineering the future of AI ### 00:02:44 # THE FORMULA ideas + autoresearch + verifiable goal + billions of tokens = kernels that beat hand-tuning autoresearch for kernels | @tejasvbhakta ## Engineering the future of AI ### 00:03:19 # Engineering the future of AI ## Before All chunks kept until the end ## After Incremental reduction keeps only top-k, reducing memory and compute autoresearch for kernels | @tejasvbhakta [Two flowcharts illustrating a process for 128K context, comparing a 'Before' method where four 32K `paged_logit_indexer` chunks are merged at the end, and an 'After' method with incremental `topk reduce` steps before a final reduction.] ### 00:03:46 ## My Problem # I like Cheap GPUs autoresearch for kernels | @tejasvbhakta Engineering the future of AI ### 00:04:18 # B200 (sm100) w/no NVLink HARDWARE SM100 `.agents/skills/sm100/` - `warps`: warp specialization · 4-warp warpgroups - `tcgen05`: 5th-gen tensor cores · async MMA - `tmem`: tensor memory · new accumulator class - `tma`: bulk copy + multicast - `cta-pairs`: 2-SM MMA · paired CTAs share a tile - `clusters`: thread-block clusters · DSMEM · CLC - `nvfp4-mxfp8`: native microscaling · per-block scales - `smem-228kb`: 228 KB/SM · the epilogue budget autoresearch for kernels | @tejasvbhakta ## Engineering the future of AI ### 00:04:57 ### MODEL: DEEPSEEK-V4-FLASH `.agents/skills/models/dsv4-flash/` - `attention-zoo`: hybrid · one type per layer - `swa`: sliding-window attention - `csa`: compressed sparse · compressor + indexer - `hca`: hierarchical compressed · compressor only - `compress_ratios`: the