Not Everyone Can be a Founding Engineer
Karan Ganesan, founding engineer at Luma AI
My next guest is Karan Ganesan, who joined Luma AI as one of the first engineers in 2022, before NeRFs were a household acronym in AI, before Dream Machine had millions of users, and before the company’s recent $900M Series C.
He found the role through a single tweet, joined remotely for the first few years, moved to USA with O1, and spent his first few years as the only engineer building everything from the marketing site to the backend infra and product. We talked about why he bet on the people before the product, what it takes to debug a memory leak that breaks Discord at 6 digit count of users, and his contrarian take on who should actually become a founding engineer.
Ben: Let’s start at the beginning. You graduated in 2020, ended up at Flipkart through an acqui-hire, and were running some of the most successful A/B tests in the company’s history. Why leave?
Karan: Quick context: I studied CS, but I was always the guy who wanted to push the boundaries of visualization, UI, graphics, AR/VR. During college I interned at two product companies in India building on top of AR tech, and one of them, Scapic, ended up acqui-hired into Flipkart, basically the Amazon of India. I spent two years there in Flipkart Labs building the application layer for 3D and AR. If you’ve ever tapped the “View in your room” button on Amazon to see if a sofa fits, that’s the kind of feature I built, but for Flipkart.
The A/B test you mentioned actually became the most successful experiment in the company’s history. Huge surface area, millions of users. But I was hitting a ceiling on innovation. You can’t really push graphics or compute on a phone, you’re capped by the hardware. And I wanted to find the next big challenge. So I was passively looking.
Ben: And then it sounds like a tweet changed everything.
Karan: Yeah. Both my Scapic job and my Luma job came through Twitter. In 2021, a mutual friend tweeted about Luma when they were still in stealth. I saw it, got curious, reached out to Amit, the co-founder and CEO, and we talked. Funny thing, they didn’t hire me the first time. They were looking for deeper, research-level systems people, and the algorithm layer for NeRFs wasn’t built out yet, so there wasn’t really a place for me.
Six months later, out of nowhere, Amit pings me again and says come join. I was the first engineering hire. I didn’t have multiple offers. I didn’t shop it around. I just took it.
Ben: That’s a big bet. You’d never been to the U.S. Luma was pre-product, pre-everything. What actually convinced you?
Karan: It was Amit. He’s the most impressive technical CEO I’ve ever worked with. He did systems engineering at Apple, worked on the Vision Pro lidar sensor, knows how CPUs and memory actually work at a low level. But what really got me was his attitude. If he doesn’t know something, he goes and learns it. If he knows something and you’re wrong, he’ll patiently walk you through why. That’s the kind of person I wanted to build with.
Back in 2022/2023, basically only us and Polycam were doing anything in this space. I read the NeRF paper, talked to the early researchers, and figured, this is genuinely new, and the people are exceptional. That was enough.
Ben: Walk me through year one. You’re employee four or five, no other engineers, and Luma is trying to turn academic NeRF research into a product. What did you actually own?
Karan: Everything. For the first 1.5 to 2 years I was the only engineer. I built the web dashboards, the marketing pages with Amit, the backend infra, the API platform. I’m a generalist, so if you throw me at a problem with enough surface area, I’ll figure it out. Year two is when the team grew from four to twenty, then thirty, and I finally started getting help.
The NeRF era ran through 2024, video-to-3D, text-to-3D, the Unreal Engine plugins, all of that. Then in 2024 we shifted into video diffusion: Dream Machine, Photon, and Ray (we just launched Ray3 about a month ago). Along the way I took ownership of the public-facing API platform, the equivalent of OpenAI’s API platform but for our models, and ran that for two years.
Ben: Let’s talk about the moment that almost broke you. You built the text-to-3D Discord bot, it went viral, and then it didn’t scale.
Karan: (laughs) Yeah. I built the first version in JavaScript. It had memory leaks. Bad ones. The bot blew up, 6 digit number of users in a week or two on the Discord server, and then it just collapsed under itself.
Here’s where Amit was incredible. He sat down with me and taught me how to do memory profiling on a web application. Most web devs never touch this, Chrome has the tools, but nobody uses them because you don’t usually need to. Profiling memory is genuinely hard. But Amit had the systems background from Apple, and he walked me through it. We ended up scrapping the JavaScript version entirely and rewriting it in Python or something.
That single project was huge for the company, it was monumental in showing what we could do, and right after that we raised our Series B with a16z. But the bigger win for me personally was the skill. Now anytime there’s a weird memory issue, I have a baseline ability to debug it that no web developer I know has. That only happens at a startup like Luma.
Ben: I want to talk about evals. Text models have relatively clean ways to evaluate quality. Video is a totally different beast. How do you actually know Ray 3 is better than Ray 2?
Karan: I don’t work on this directly, but I’ll tell you how we think about it. There are baseline mathematical metrics, efficiency, inference time, that kind of thing. They give you a starting signal. But honestly, there’s no replacement for human evals in video.
We have an eval team whose job is partly to set up automated pipelines using VLMs as judges, but a huge chunk is human review. We have technical artists internally. We have a team in LA called Dream Lab whose help test everything. We work with external partners running leaderboards and matchmaking-style evaluations.
And here’s my personal take: eval scores are 90% there, but the last 10% has to come from real customers. A model with a lower Elo on some leaderboard might be perfect for a niche use case nobody on the leaderboard tested for. Evals are representative, not definitive. You still have to put the model in front of users and see what happens.
Ben: You said earlier that some of the early hires actually came through you, your reputation on Twitter. What do you look for when evaluating people?
Karan: In the early days, pre-2023, before Cursor autocomplete and the Opus-class models, we didn’t have AI tooling making engineers faster. So the differentiator was something else. The keyword is agency. Are they curious enough to do the small things that compound into big things? Are they comfortable not waiting for approvals, not waiting for a script manager to tell them what to do?
In a startup, you don’t have an engineering manager to lean on. You don’t really have HR, there are five people, just go talk to whoever. The technical side is you, the CEO, and a few friends along the way. So the question is: do you actually thrive in that uncertainty, or do you need structure?
Ben: That leads me to a question I ask everyone, how should someone think about whether they’re cut out for the founding engineer path?
Karan: Here’s my contrarian, unpopular take: not everyone can be a founding engineer. And that’s fine. You need a sense of comfort with things being unstable, unknown, and unscripted. There’s no engineering manager you can co-depend on. There’s no mature playbook. If someone wants a stable job, doesn’t want to take a lot of risk, that’s a totally valid choice, it just means this isn’t the path for them.
I’ll be honest about my own bet too: I was 50% lucky and 50% deliberate. I went looking for the right team, the right culture, the right kind of founder. We’re a $4B company now, so it worked out. But it could have gone differently, and I think it’s important not to romanticize this role.
Ben: Now that you’re four years in, founders reach out to you for advice on building their early teams. What do you tell them?
Karan: Some of the early Luma people came through me, they saw what I was tweeting about, figured if Karan works there it’s probably worth a look, emailed me, and I’d connect them with Amit. Now founder friends ask me to help them figure out who their first hires should be, what profiles to look for. I do it casually, as a friend, not as an advisor or angel.
The pattern I keep coming back to is the same agency point. In the early days, you’re not hiring for a skill set, you’re hiring for someone who can be thrown at any problem and won’t freeze. Multifaceted problems where you need a lot of context to even know what the task is. People who treat that as fun rather than overwhelming. That’s what I look for, and that’s what I tell them to look for.
Ben: Last question. Looking back at that tweet in 2021, the six-month gap before they came back, the move to the US, the all-nighters debugging memory leaks. Anything you’d do differently?
Karan: Honestly, no. The path looks obvious in retrospect, but at the time it was a single tweet, a stealth company, and a founder I thought I could learn from. I didn’t have a backup plan. And I think that’s the lesson—if the people are right and the problem is real, you don’t need ten offers to make the call. You just need one.