The Story Behind the AI Startup Powering Services You Use Every Day

Intro 00:01
Russ, welcome to StartupSuccess, the podcast for startup founders and investors. Here you’ll find stories of success from others in the trenches as they work to scale some of the fastest growing startups in the world, stories that will help you in your own journey. Startup Success starts now.

Kate 00:18
Welcome to startup success today. We have Russ d’Sa in studio, who is the co-founder and CEO of LiveKit. Welcome Russ.

Russ D’Sa 00:27
Hey, Kate. Thank you so much. Thanks for having me here.

Kate 00:31
Yeah, I’m looking forward to our conversation. I want to get into your background and everything, but I always think it would be most helpful if you give an overview of LiveKit for our listeners to start, just to kind of set the stage, and then we’ll delve into your background and what led to its founding.

Russ D’Sa 00:48
Sure. LiveKit there are a variety of different ways I can explain it, kind of varying amounts of kind of technical depth. The tagline is sort of real time audio and video infrastructure, or streaming infrastructure. The way that I would explain what that means is, if you kind of think about like, what Stripe ended up doing for payments, right? There’s payment gateways and Visa and MasterCard, and there’s all kinds of software underneath there that you have to interact with. Stripe kind of took those interfaces and made a really nice kind of developer platform, instead of APIs that make it easy for a developer to integrate payments into whatever application they’re building. You can think of LiveKit as the same way, but it’s kind of like Stripe, but for communications technology. So another way I kind of sometimes phrase it is, if you think about the Zoom application. Zoom has, of course, an UI that you interact with and you can meet anyone around the world instantly. And Zoom has a ton of technology that they built underneath to make that relatively simple thing conceptually possible. And so what LiveKit is it’s a collection of APIs and infrastructure that is an open source kind of version of Zoom’s technology stack, like this, a similar type of underlying infrastructure that Zoom had to build to power the Zoom application. We build that infrastructure, but we stop at the API level, and we allow anybody else to build an application that has that kind of infrastructure for real time audio, video, communication. We allow you to kind of build an application that incorporates that technology in you know, your feature set or the core way that your application works, that’s what LiveKit is.

Kate 02:45
Wow. Okay, first of all, I’m really glad I had you explain. I wouldn’t have done it the justice that it deserves. Those are excellent analogies. Well, I read it, but when you explained it, it’s much easier to understand. That’s pretty cool, and that’s pretty complicated what you’re doing. So kind of walk us through your background and what led to LiveKit.

Russ D’Sa 03:06
Yeah, we took kind of, well, we definitely kind of sort of accidentally stumbled into LiveKit as a company. But we had this kind of organic path towards how LiveKit started. For me, background-wise, this is probably, this is my fifth company that I’ve started, and so each one kind of gets better and better over time, which is what you’d hope, I guess. And I kind of started off almost in some ways, steeping in startups and entrepreneurship and stuff, just because my dad was kind of early in technology startups when I was a kid. So I grew up in Canada. My dad was an electrical engineer, and ended up kind of joining companies and starting companies through many waves of technology from the 80s all the way through to the early 2000s. And so started off with semiconductors, and then there were GPUs, and then it was high-speed internet through DSL. And then moved to flash storage. And then there were a few other kind of waves after that that he was a part of. So he exposed me to this stuff, brought me to, sometimes to VC pitch events, and he took me to CES when I was 14 years old, and Comdex as well when I was a young kid. And so I always kind of grew up in this environment where I was both fascinated by it and also kind of enamored by this idea that you could have a small group of people that could change the world from their garage right. And build stuff that they might dream about, and make it into a real thing. And so I got into programming very early in my life. I was, I think 12 years old, I started taking these C++ night classes at this local night school with a bunch of HP Hewlett Packard, like engineers. I remember sitting in these classes, and that’s actually the location of that school is where Apple’s campus is. So they demolished that thing to build the spaceship in Cupertino.And so, yeah, my dad had come to Silicon Valley in the early 90s, had been flying back to Canada every month. And that, you know, I’m close to my dad, it ended up, you know, kind of being not a super fun situation to be so far from him and not see him as regularly as we’d like. And so we moved the whole family out in the early 90s. And so I’ve been out here since then, kind of doing all these things, going to all these events, and, you know, just kind of getting exposed to it and so I started my first company right out of school, right out of college. I did my kind of the bachelor’s master’s program at Davis, up a bit north from here, and Y Combinator was something that had just started. They had been three or four batches in and my friend of mine that I’d made in college and I, we applied to YC, ended up getting in for the fifth batch, and started a company back then. That was my first company. So I think I was, like, 22 years old or something like that. No idea what I was doing. And I guess nobody else did either, but a few of us ended up doing quite well from that batch, not myself, but definitely some other folks there. And then, you know, I’ve been starting companies ever since along the way. Fast forward to LiveKit and how that was started, the pandemic had just hit, and I had sold my previous company to Medium. And so I was running product at Medium. My co-founder, we had met in YC in 2007 doing separate companies, and then we started a company together, many years later, sold it to Medium. He was running Engineering. I was running Product. And the pandemic hits and Medium is this 350 person company, and all of a sudden we’re all working from home instead of working from their office in San Francisco. And Clubhouse had come out, and Clubhouse had become quite popular. This really neat, fresh, interesting kind of application. And I had tried it, and I said, Wow, this is really cool, the kind of casual drop in nature of it, I want something like this, but I want something like this for Medium, because 350 people all of a sudden working from home, and you’re not seeing people in the cafeteria anymore, and like having these kinds of, like casual conversations. So I started to prototype kind of a side project I called Water Cooler to kind of be a Clubhouse, but for a company. And I built it in a month, and I used the same stack that Medium had used – this company called Agora, and they’re kind of the incumbent in the real-time audio and video infrastructure space. I didn’t know that at the time. I just knew that’s what Clubhouse was using. And so I took that, built this app, launched it, and it blew up on Product Hunt, and suddenly I was doing interviews with the New York Times and the BBC and the Wall Street Journal, for a side project, which is kind of strange, but also it was a strange time. We’re only four, five months into the pandemic. Everyone’s kind of just clamoring for a way to connect with one another. And so a company, I had a 1300 company waiting list to use this product, the side project. And one of the companies was Pinterest. And Pinterest, they wanted to buy 500 seats of this side project. And said, Okay, I’ve never sold software to businesses before. I’ve always kind of been in, like, the consumer space, and I didn’t know much about how to do that and what the next steps were, so I asked Pinterest, well, okay, what do I do next? I’d love to sell 500 seats. And they said, Well, here’s a vendor security checklist, and we’re going to be potentially discussing company secrets using this software, so where does the audio data go? And I said, Well, I used this company called Agora, and they’re public, and hubbus uses them. And so we’re good, right? And they said, Well, no, actually, there’s geopolitical issues, and like, we’re not, you know, you got to find an alternative provider. So take a look at Twilio or something else. And for a side project, I was looking at these commercial offerings, they were just way too expensive. And so as an engineer, I said, Well, I’m going to look at open source, like, what’s in open source? And it turns out there was nothing, really. There were a few projects, but they weren’t very mature. It wasn’t easy to deploy and scale them. It just wasn’t kind of like enterprise grade. Or, you know, when I describe kind of Stripe for payments, like Stripe is something that is a very robust platform that just did not exist for communications infrastructure in open source. And so I pinged my old co-founder, and who was at Medium, and I said, Hey, can you help me? Like, I need to get this deal with Pinterest. So we started to look at open source some more, and we said, You know what? What if we just build our own? How hard could that be?

Kate 09:43
I knew you were going there, but this is wild. Okay, keep going.

Russ D’Sa 09:47
And so we started to build it, and we used my Water Cooler side project as kind of the reference implementation. And then as we started to test Water Cooler with companies, we realized that, you know, audio rooms for companies is not really like the job to be done. The job to be done is, how do you build company culture in a remote, hybrid world? At that time, only remote. There was no hybrid, but it eventually became hybrid world. And companies tend to be – something I’ve learned over now this is my fifth one – you really have to have conviction in what you are doing and be willing to do it for a decade, call it seven to 10 years at least. To be able to kind of weather the ups and downs of company building is just what it’s what you’re signing up for. And I was not willing to sign up for that for the next seven to 10 years of like, kind of building corporate culture software. Other people will build big companies in this space, I’m sure, and there are some already. But for me, personally, it wasn’t what I was passionate about, but my co-founder and I were much more interested in the infrastructure part of it. And so we said, okay, you know what? We’re gonna just forget the Pinterest deal, shut down this water cooler thing, and let’s just focus on working on this open source project. Like, let’s turn it into an open source thing, because none existed. And that’s eventually how LiveKit was born, is we just started to focus on this open source project. We worked on it for about a year before we released it widely on GitHub, and when we did, it just kind of blew up. So we had a lot of large companies, Spotify and Oracle and Reddit were all starting to deploy it internally and play around with it, and build internal projects with it. And so we started to just have conversations. We had, like, I don’t know, maybe like, 50 conversations with companies and individuals that were using the open source project and asking them, how can we make it better? How can we improve? What can we build for you? And we invariably heard from probably, like 98% of the folks we talked to, What you can do to make it better is host it for us and scale it for us, and we can just give you a credit card and pay for that part, because we love that it’s open source. We love that we can host it ourselves if we want to, but we don’t really want to, at least not right now. So can you build a cloud, hosted version of this? And so that is kind of how the motivation to start a company around it came about where we really were passionate about working on the infrastructure. We wanted to do it for the next 10 to 20 years. And now we had a sustainable path towards doing it by creating a commercial, you know, entity and a commercial product around it and, you know, really following the playbook of of other kind of commercial, open source companies that have come before us, like Mongo and Elastic and Confluent with Kafka, etc. So we raised a round, started hiring a team, and kind of it’s been a wild ride ever since.

Kate 12:54
That is such a wild ride. We’ve had so many founders on this show, over 100 episodes. That is one of the wildest ones I’ve heard. But awesome. I mean, just the different paths you took, and the Pinterest and the 500 seats, and then going the open source route.

Russ D’Sa 13:17
It’s the first company that I’ve started where it organically happened. It wasn’t that I went out to start a company. That wasn’t the goal in itself. That was just kind of incidental or a necessity to be able to move this project forward and turn it into something that we could continue to work on for a long period of time. And every previous thing I’ve done has been forced in a way and has not worked out. And so I don’t know, but I think there’s a learning in there for me, especially, definitely, there’s a learning in that don’t force it, don’t start companies just to start companies. That’s not even why the concept of a company even was invented. It was, you know, it’s, it’s a kind of a container or a grouping mechanism to kind of tie together and align incentives for a bunch of folks that have a mission or a goal in mind that they want to pursue. And so the goal has to come about first and then kind of create the group structure to go pursue that goal. And that’s how LiveKit ended up. Kind of that was a path that we took, and how it went.

Kate 14:29
Absolutely and I love the pivots you made, because I think a lot of founders might have gotten hung up on the Pinterest thing, like too scared to walk away from that, right? I mean, that’s a big deal. And then when you’re on open source, you took a lot of customer feedback, right?

Russ D’Sa 14:48
Another big learning that I’ve had, this one I would attribute to DoorDash. Tony actually went to high school – he was two years younger than me at high school – we grew up in Cupertino. But I’ve started companies in the past, and even worked at places in the past where – Twitter being an example of this, or I guess X is what I should call it now, still not used to calling it that. I didn’t have Elon Musk owning the company I worked for in my bingo card, for sure. But when I worked at Twitter, we built a lot of things, kind of with the mentality of, if you build it, they will come. And previous companies I’ve worked on, it’s the same thing. It’s like you have this brilliant idea, and you think everyone’s going to use it, and you spend like, a year or two, kind of, like working on it and making it perfect, and then you go out there and it doesn’t, you know, meet your expectations in terms of the adoption, but everyone believes that what they’re working on is going to be big, and I think it’s important to have that excitement, but it also is important to kind of root it in reality a little bit. One thing that I heard from the DoorDash founders when they ended up going public is they would just drive to restaurants all around the Bay Area, and just ask people, you know, what problems do you have? Like, what can I solve for you? Right? How can I make your life easier? So they were doing a bunch of that research and really validating what was a problem worth going and solving, or what was a job to be done, or a set of jobs to be done. And that’s something that I hadn’t done in my past companies. And open source was kind of like this conduit towards understanding what were the jobs to be done, what were the problems that needed to be solved? Right? If we put out open source and it didn’t go anywhere, then, okay, we probably don’t have anything to be solved. We were solving a problem for Water Cooler, my side app, but not for anyone else. And then when we went and had conversations with these companies who were early adopters of the open source project, they started to tell us, like, here are the things that we need. Okay, I can charge money for that. Or they were telling us they’d pay us money for it, even better. So we kind of de-risked starting a company around it just by talking to people, absolutely,

Kate 17:04
I can’t tell you how many founders I hear that from. It’s the customer feedback, the pivots, the changes, the iterations they made around that feedback being close to those initial customers. (100%) Yeah, so spot on. Can you share a use case for us?

Russ D’Sa 17:23
Yeah, the use cases are pretty varied, and some of them are very wild. So I mean, I’ll just go straight for some of the most interesting ones. My favorite ones, of course, is that LiveKit saves a life every single week through 911 Emergency. So we power about 25% of 911 Emergency in the United States. And the way it works is there’s a company that goes and takes LiveKit open source and employs our open source server in dispatch centers all around the US. And before September, I’ll tell you what happened in September, but before September, the way it would work is that a 911 phone call comes in, and the dispatch agent can say, Hey, do you want to stream video to me, show me what you’re seeing or what’s happening right now? And the caller, they say yes. They get a text message that’s sent to their phone, and it’s with a link. They tap on that link, and they’re in a mobile browser, streaming video, audio and GPS information. And then the dispatch agent can see what they see, hear what they hear, understand where they are. They can even conference-in emergency services, fire, police, and that’s happened actually before too, where someone will be streaming from a 911 call like a fire, a building being on fire, and then en route, the firefighters will connect to that session and watch the video and figure out how they’re going to plan dealing with the situation that’s going on before they get there. Another interesting story here is that every single week, someone gets a heart attack, and the person that’s with them calls 911, they get a text to their phone, they’re streaming video, they give their phone to someone else that’s there, or they prop it up against something. And then the dispatch agent coaches them on how to administer CPR, and they save that person’s life. This story happens every single week, and it’s going through kind of LiveKit’s technology. Now in September, iOS 18, it’s directly baked into iOS 18, so hopefully you never have to call 911 but if you have to call 911 on an iPhone, the FaceTime button will now be there, and when you tap on it, that will stream video through LiveKit to the dispatch center. So it’s baked into the operating system now. So that’s one cool use case, but there are other really cool ones too. So if you’ve ever completely, well, not totally, on the other end of the spectrum, but a very different one is if you’ve ever had a voice conversation with ChatGPT, that’s also powered by LiveKit. So we have our cloud system, you know, it’s a whole network of servers all around the world. They all mesh together to form, kind of like this network fabric that spans the entire globe. And when you tap on that voice button or wave form button in the app, and you enter into advanced voice or voice mode before it in the ChatGPT app, you connect with an AI model, and so when you speak that’s going over LiveKit’s network to the AI model, the AI model is internalizing it and then generating speech that gets sent back over LiveKit’s network to your device and played out through your device. And you do that back and forth, and you’re having a conversation with GPT-4o or and any one of their other models.

Kate 20:36
Wow. Okay, two great examples.

Russ D’Sa 20:39
Yeah, it’s pretty cool.

Kate 20:40
It’s pretty powerful stuff.

Russ D’Sa 20:42
It was kind of designed to be very general purpose. (Yeah) I think one reason that you didn’t see any open source infrastructure for this back in 2020, 2021, is because in software infrastructure, there’s kind of this pattern that it follows, where when a piece of infrastructure is needed by most or every developer, then you tend to have an open source solution that commoditizes the space. So if you look at databases like there’s quite a few open source databases. So for audio, video infrastructure, the only thing you really used it for for 10 years was like having business meetings. And so it’s like, BlueJeans, WebEx, Zoom and so for like a single purpose infrastructure, the vast majority of developers aren’t building applications like that. And so there really isn’t a motivation for an open source solution to kind of be this commodity thing that everyone can use. But 2020, 2021, you know, peak of the pandemic. All of a sudden, everybody is trying to think of, how do I take these offline things and port them over to the internet? And I need to use cameras and microphones to do it. So every developer is starting to think like this, and so that’s why there was this kind of moment where there’s this gap in the market, and we didn’t see that gap until afterward. We’re like, why did we actually succeed? Why is this project doing? Well, I don’t get it. But it wasn’t until in retrospect that we looked back and we said, oh, okay, that’s why it struck a nerve, and we just kind of hit the timing right. And another lesson about starting companies is people say that ideas are cheap and execution matters. It’s not quite true. It’s ideas matter and execution matters. And in so far as that an idea also embedded within an idea is a timing of that idea too. Timing matters a ton, and a lot of that is luck. This is not something that people want to hear…

Kate 22:55
I’m so impressed because you’re owning you know that in hindsight gave you clarity on the timing and I mean, I love that you’re owning that, instead of saying “we saw that this”, you know, it’s more like looking back. That makes sense.

Russ D’Sa 23:15
100%. Yeah. People actually asked me. They said, How did you think about building this infrastructure to power like give AI the ability to see and to hear, this nervous system, to the brain, where the brain is the model and our infrastructure is the nervous system part. How did you see that opportunity? And I said honestly I didn’t. The honest truth is we built a very general purpose system, and it was able to be used for this AI use case. And it wasn’t until we started to work with OpenAI on it, it was at that moment that I kind of realized, Oh, okay, this is going to be the future. Because if you’re building an AI model, if you’re building a computer that can think like a human, respond like a human, behaves like a human, how will you probably interface with that human? Computers today we use a keyboard and a mouse, and we have to take a keyboard in class, in school, and learn how to type on this thing, and we have to move this mouse-looking thing around. And in the future, when you have a smarter computer, a more human-like computer, it’s the computer that adapts to you. Where the inputs and the outputs are more natural, like human inputs and outputs, which is what we’re doing right now. We’re talking and we’re looking at one another, and we can hear one another. And that’s with cameras and microphones, not keyboards and mice. And so it was that moment where I realized, Okay, like we’re actually building kind of core infrastructure for the future of where computing is going. But I did not see that beforehand. The light bulb went off much later in the process.

Kate 24:57
And I like that you shared that with us. That’s so helpful for all the founders listening. I think that you know sometimes hearing the truth of what really happened helps a lot. That leads me to my last question. We always end the show with this. I’ve so enjoyed speaking with you. This has been really fun. Advice for startup founders listening. Everyone gets upset if we don’t have time to cover this. And so anything you can share?

Russ D’Sa 25:28
Yeah, well, I think (you gave a lot of advice.) Yeah, my biggest learnings, I kind of dropped some of those nuggets, like, during the conversation, maybe to recap, if I were to, like, review, just think in my head, like, what would be the key takeaways. I think the first key takeaway is, don’t start companies just to start companies. Try to take a more organic path, right? I think build a lot of things, experiment with a lot of things, put yourself out there, be in the conversation, be contributing value in places. I would say, like the best way to kind of go along that organic path, but also touching upon the timing matters piece of advice, or the learning that I had to kind of marry those two things together. What I would suggest is take a very broad view of where you think things are going. Like when you’re sailing a ship, and land is, like, pretty far away. You don’t have to be perfect on the trajectory. You can be like, a few degrees off. But the main thing is have an opinion on where you think the land is generally out there, and then just get the boat in the water and start sailing right in that direction. And you’ll learn things along the way. And the tides will change, and the wind will change, and things like that. And you just want to be like, course correcting over time. (Course correcting, right.) You just want to be nimble, and you want to be in the water. You want to put yourself out there. You want to be building and doing stuff and trying to add value, really, just heading in the general direction of where things are going. And over time, you’ll kind of tighten that path, and you’ll end up, you know, with these kind of 10,000 tweaks on your direction over time, you’ll end up, you know, hitting land. land. I’m not sure how clear that metaphor is, but that’s what I would suggest. I think, like for us when we did LiveKit, the thing that we knew, we knew we would eventually come out of the pandemic. It wasn’t like we were trying to just build for the pandemic, but we knew that computing was not going to get more async or slower. It was only going to get more and more real time and faster. And people would not want to connect with each other less. They’d only want to connect with each other more. And the world is getting increasingly global, not less global. And so those are all the trend lines. And so, you know, the land is in this direction, kind of far out there. And so how can we add value in that kind of area, or make it easier for humanity as a collective to build towards that direction? And that’s kind of the path that we took with LiveKit. Just starting off with an open source project and just improving it and adding value along the way, talking to users of it, getting information about how to make it better and better, or what they need, or what’s not here yet that would make their lives easier. And just building those things and continuing to grow and kind of just, you know, sailing, right?

Kate 28:22
Sailing. I love that you have a great way of explaining things. Where can listeners find more information about LiveKit before we let you go.

Russ D’Sa 28:31
Yeah, folks can go to our website, that’s LiveKit.io. They can also follow us on Twitter. We’re @LiveKit, on X. I’m not gonna get used to this! On X, we are @LiveKit. I’m personally @DSA. My DMs are open, so if you listen to this and you want to chat more, I’m happy to bounce ideas around or try to help if you’re using LiveKit or anything like that. So yeah, those are the places you can find me.

Kate 29:00
So great. Thank you. Russ, this was really fun. Thanks for taking the time.

Russ D’Sa 29:03
Thanks so much, Kate, yeah, I enjoyed it as well. Appreciate it.

Intro 29:06
You’ve been listening to Startup Success to make sure you don’t miss out on future episodes. Subscribe to the show and your favorite podcast player like what you hear cap the number of stars you think the show deserves in Apple Podcasts. For more tools and resources for your own startup success. Check out burklandassociates.com. Thank you so much for listening. Until next time.