Aidan Gomez: The Case for an Independent LLM Platform

You and I met when I worked at Index, which is an investor in Cohere, and I've since jumped over to Contrary. I've listened to a few other podcasts you've done, and I feel like a lot of times people are using you as a proxy to learn and figure out what's going on. When you're doing that, I feel like the Cohere story gets a little bit less attention, because people are so focused on trying to wrap their heads around AI more broadly. What I was really excited to do today is to spend a lot more time digging into the Cohere story. As you were immersing yourself in the world of machine learning, there must have been a reason why Cohere became this embodiment of what you wanted to build. What was it about your experience that led you to solving this particular problem in this particular way?

A lot of the time, it feels like we've been standing in the same place for years, and market has been moving around us, and I guess in particular towards us. Obviously, I was part of the team at Google Brain that created the transformer. We saw that success and adoption and excitement. And then starting Cohere, Nick, Ivan and I, the whole objective was to bring this tech, large language models, to industry, bring it to the world and facilitate adoption. Nobody really knew what large language models were. It was a research project, interesting tech, but no real commercial application. But I think we had conviction on the trajectory and the pace of progress, and that computers would get increasingly good at manipulating language. And we would quite soon arrive at a place where computers could speak correctly, they could talk our language, and that that would be extremely valuable for the world, be valuable for consumers, it'd be valuable for businesses.

So when we started up, initially, we didn't really have an idea of what product we wanted to build. We just knew we need to push this tech trajectory forward. And we just focused on building the infrastructure to train large language models on supercomputers using whatever compute we could get our hands on.

Shortly after we started Cohere, about six or nine months in, GPT-3 came out. And that was another huge breakthrough moment where we saw, look what happens when you keep scaling this thing, look at the behavior that comes out. And so that was very validating, gave us a direction that we knew we were heading in the right path. But still, the world didn't really know what the impact of the tech would be like. When I would go into sales conversations early on with enterprises, I would have to explain what an LLM is, why you should care about it. And I would be patted on the back, and, "That's cool. That's an interesting research project you're working on." And so for a long time, that's what it was like. And folks were like, "I'm paying so much money to my internal data science and ML teams. Why would I buy your stuff? We're just going to build it ourselves." And people really believed they could do it themselves without us.

And then eventually market woke up. I think the emergence of dialogue models, models that you could talk to, really changed the way people saw the underlying technology. So consumers actually got a firsthand experience interacting with it. We got direct proof that the experience is magical and highly desirable and highly useful. And so every single board, every single C-suite woke up. And that same group of folks that I was talking to a year ago who thought the tech was cool and was excited about it, but they didn't really know what to buy or whether they needed it or whether they needed us or they could do it themselves. The market completely shifted into, "We need a partner to do this. This is some of the most difficult technology to build that humans have ever pursued." And there became a real urgency in market for exactly what we had and we were building.

So I like to think that we've been mostly preaching the same thing and saying the same story. And maybe we phrase it in the language of today. It's LLMs one day, Gen AI the next day or blah, blah, blah. Who knows what it'll be tomorrow. But underlying that is we know the market will need an independent, not tied to any of the hyperscalers, platform to enable enterprises to adopt this technology in a way that's extremely data private and secure. The world just needs that. And so we're going to keep building it. We're going to keep making these models better and better, making our platform more and more robust. And we will be the partner to enterprises when they're ready to start this project. And then seven months ago, everyone became ready.

So I think it's been a really exciting journey. And yeah, we're in a really exciting moment because now everyone appreciates the tech. Everyone has used it. They felt it. And so we're post awareness, but we're pre the real change. We're still early. We're still within the first year of people waking up. I think the world looks very different two years from now. And this technology is a feature of every single product that we interact with. It just becomes day to day. It becomes an expectation. You expect to talk to an agent that's automated and as smart as a human.

Do you think that it was just the consumer experience with LLMs that woke people up? I remember two or three years ago talking to you about those patting-on-the-back experiences where people are saying, "I think we can handle it from here. Thanks, man." You'd think that business users and everyday consumers would emerge into a technical trend very differently, but it definitely feels like the consumer experience of LLMs was a wake-up moment for a lot of people, even business users. Do you think that's true of C-suites and boards, that it was just a moment that shook everyone out of their sheer hubris that they could do this stuff themselves?

I do think it was. There's two ways to wake up enterprise. One is to work with their competition and have a breakthrough and get them to scramble. The other way is to get consumers, the end users of whatever product or service that they're building, to insist upon this. And I think what happened with this technology is that consumers insisted upon it. They very clearly expressed their intent and desire of, "I want this. This is the most exciting thing I've ever seen. This is a magical product experience. And so I want this everywhere." For LLMs, this technology, that seemed to be the route at which enterprise woke up, which is interesting. I think it's exciting. It's even more exciting because there's broad public awareness. Like the cloud breakthrough, there weren't consumers saying, "I want the app I use on my phone to be running on the cloud." There's something nice about being able to bump into anyone on the street and for them to understand what you do, what you build and how special it is. So I'm happy at how it played out.

It's a huge moment to be standing in just the right place at just the right time, where the consumer sentiment came to where you are. But one of the things that I think is a really difficult position that you're in from a technology perspective is that there are a lot of different use cases. When you're trying to build a business and identify this ideal customer profile, the world is your oyster, and it's very difficult to hone in on what's most valuable. In simplest terms, how do you describe what your product is? Do you isolate it around a particular use case? Do you try and be very general? How do you position your product when you can solve so many different problems?

Yes, we've tried building purpose-specific solutions, like endpoints, right? We have our summarize endpoint, we have classify. What we found is that's more of a mature market buying decision type product. For this technology, what people are most excited about, what they care most about, is how general it is, how instead of us coming to them with a solution to their problem, we provide them a general platform, which are these models. The models themselves are a platform for solving a problem. And they come with their problem and the model can just solve it. That's magical, right? It's this generalist software that can just solve, within reason, any problem that you're coming to it with. That's what the market is valuing and demanding.

I don't know if it stays like that. I don't know how enterprises will think about specializing as their needs become more defined and concrete and dominant use cases emerge. I expect that there will emerge a demand for more specific, tailored offerings. The state of the market today, though, is very much for a general platform of artificial intelligence. You have an artificial intelligent system and you want to be able to bring your problem and have it solve it for you. So yeah, that's the state of play. Super general.

When you think about the interface into those super general models and use cases, you guys have provided an API that people can leverage. We've seen a lot of chat interfaces. There's also a lot of folks talking about AI agents and being able to endow a specific system with a use case to go and do tasks. How do you see the future of the way that people interface with these models changing over time? Is there going to be one that becomes the most dominant, or how does it shake out?

I think agenthood is super important for this category of product, or for these models. The end state is what you really want is some intelligent entity, which can use tools like the internet and the browser to go off and do work for you and come back with that job done. And what we have today is much more transactional. It's a one-step process. It's, "Do this for me." "Okay, I've done it." That's the interface between us and these models. When what you really want is, "Do this for me," the model says, "Okay, I'm going to go do it." It goes out, it breaks that task down, goes out and carries out all the subtasks, compiles them into a solution, then comes back and says, "I've done it." It's much longer horizon, there are more steps in the process of solving these tasks. That framework of long horizon planning and agenthood and being able to go out into the world and carry out actions and do research and compile it, that encapsulates tasks and use cases that are just dramatically more valuable than the transactional one-step process that we're in today.

So that's inevitable. And I think that's the real value unlock, when we have agents that we can send out into the world and they do work for us autonomously. That's when we can really start to automate the interesting stuff.

The problem with them today, I'm super excited that people are exploring it even at this early stage of the technology. But the problem with them today is that if your model is 90% accurate, which is a pretty good accuracy, right? You would be happy for a lot of tasks on that. We're only referencing one step. So each step is 90% accurate. And let's say an interesting problem requires 30 steps. Well, it's 0.9 to the 30, which is an insanely low success rate, right? You could never ship a product with a decimal zero whatever percent success rate. So what you really need is something that succeeds 99.99999% of the time on each individual step and can carry out these super long trajectories. We're not there yet, but like I said earlier, it's inevitable that that sort of system gets built. And there's a bunch of people focusing on making them much more robust.

As these agents get more capable and accurate, do you think that the more transactional interfaces like an API or a chat interface go away? Do they get relegated to these really simple tasks? A lot of people are talking right now about how you build a model and then you can build an API to be able to leverage that model. Does that API business model completely go away for this space?

No, I think it's still a dialogue. Even when these are agents that can go out and do stuff for you, it's still a conversation. You'll still need to provide feedback. It's not like these machines are going to become infallible, right? In the same way that when you interact with a human and if you're managing them and you ask them, "Hey, go do this thing for me," they're going to come back and they'll have made some assumption that's wrong. And you'll say, "Oh no, I meant like this." And they'll have to go back out and fix that and come back again. Unless at the outset, in painstaking detail, you describe exactly what they have to do, which no one's going to use systems like this. So it will still be a back and forth. It'll still be a dialogue and exchange.

So I don't think our current interface of chat and dialogue goes away. I don't think APIs go away. I think those are going to be, of course, that's the way that computers talk to each other. If you're going to build a product on top of these agents, you're going to be using an API that probably looks very, very similar to what we have today. It's just in the background on our side, the companies building these models and these agents, in the background, a lot more is going on. It's no longer just put it into the model, get the completion and return it to the user. It's now put it into the model. The model makes some predictions about what it needs to do, spin up sub jobs, and then give the response to the user.

You talk a lot about how Cohere is really focused on being cloud agnostic. You can operate within a company's VPC. You've got some folks that completely shut it down. They don't even want their employees using something as simple as ChatGPT: Amazon, Apple, everybody limits it. And then you've got these really highly regulated industries, healthcare, government, where there's so many concerns about allowing any of this data to live on an OpenAI server. How does your approach differ from alternatives that are out there in terms of the way that you interact with customers' data?

I think Cohere isn't beholden to any of the large cloud providers in the same way that our competitors are. We haven't taken monster, multi-hundred million dollar checks from anyone. And that's been a choice that's been a fairly dogmatic position to preserve our independence. My thinking is that the market needs an independent. The market needs someone who can play on all different clouds, who can play on-prem, whatever the customer needs, because often customers are multi-cloud, right? And so they can't really have a stack where, "Oh, for my data that lives on GCP, I'm going to use PaLM. For the stuff that lives on Azure, I'm going to use GPT-4." They want a consolidated, simplified stack. And similarly, even if you're not multi-cloud, if you're on one cloud today, that changes, right? Your strategy pivots, you need to, every three years, renegotiate your cloud contract.

And if you're locked in, if you're using a proprietary model exclusively available on one platform, your negotiating leverage is gone. The core piece of your user experience is a core piece of what's driving features inside your product, and it's only available on this one cloud. There's no plausible world in which you switch. And so strategically, you have no leverage. My thinking is that the market will demand an independent player for enterprise. It's just absolutely necessary. And so that's what we're positioned to do, is to be able to play everywhere with any cloud, even on-prem.

You look at this space and you've got a lot of these leading AI companies and big tech companies, and it's starting to feel very much like there is this lining up of interest, right? You've got OpenAI with Microsoft, Anthropic with Google. I know that Cohere has some corporate investors like Nvidia, but you've made it really clear, I think in an interview a few weeks ago, that this is a very normal financial investment. You're not becoming a subsidiary to these companies. You're not getting to cut into the front of the line. At the same time, it certainly feels like some of these companies are becoming very corporate aligned, and they have these incentives with these bigger companies. It is going to force them to probably tip the scales towards their corporate partners. As you think about this intricate web of corporate relationships and incentives, how do you navigate that? Is it 100% to stay away from it and stay above it all? Or do you have to take that into account?

Yeah, I think on this front, it's just happiest customer wins. Money is a moat, resources are a moat, up to some point, but above some threshold, it starts to diminish in terms of its differentiating factor. So Cohere needs to stay above that threshold, but it doesn't really matter if we have $500 million or a billion, right? There's some threshold above which we have access to the compute and the talent that we need to execute. And within our peer group, whoever makes customers and users the happiest will win. I think Cohere's current position is that we don't need that sort of relationship to succeed. We're very focused on delivering value to customers early. We don't do these sorts of once every 18 months, we'll do a big version release. Our Command model gets updated every single week. Users who contribute data, they'll see an improvement on a weekly basis. And so we're just focused on as quickly as possible improving our models and getting a better and better experience for our users. Of course, that is a capital intensive fight, which is why we have to raise significant rounds to facilitate doing that work, paying for those supercomputers. But above some threshold, I don't think capital yields that much. And I think that Cohere sits above that threshold.

One of the ways that it feels like this will play out is continued improvements in performance and capability of the models that you're offering. In terms of staying competitive, certainly compute and having access to those resources is one piece of the puzzle. But another piece of the puzzle that we talk about a lot is data. We know a little bit, at least from things like GPT, you look at the underlying training data, and Common Crawl is a big piece of it, WebText2, right? So you've got the Reddit controversy that's going on as people leverage user-generated content sites for training data. I haven't heard a great answer to this, and maybe it's a stupid question. But when you think about the next five to 10 years and the performance of these models and the data required to improve them, are we effectively pushing the limits or running out of the user-generated data that LLMs are trained on? And if we have more of the output on the internet coming from these models themselves, does that exacerbate the problem? Or is this even a data question?

I definitely think it's a data question in terms of model capabilities and continuing to push up their performance. It's pretty clear that we're running into some very natural barriers, which are the models are getting as smart as the smartest humans in certain fields, right? They're really consuming a very significant portion of humanity's knowledge and starting to exhaust it. And so if we're going to keep up this pace of improvement, we need to take that limiting factor, that bottleneck, away and let these models self-improve. Let these models, either through some sort of competitive dynamic or whatever it is, give them a means to discover new things, new useful things, and teach themselves, as opposed to relying on humans to teach them everything and to spoon-feed them new knowledge.

I think we're hitting that friction point now. And you can see it for foundation model companies like Cohere, we have to go and call up master's students in mathematics and get them to teach the model expert knowledge at that level of proficiency, right? And so I think that really points to the fact that we're starting to exhaust the low-hanging fruit and we're having to go after very scarce resources in order to continue improving these models.

I do think there will be a breakthrough. This is super contentious and a lot of people would disagree with me. I think that within the next 18 months, maybe even 12 months, we will see systems for self-improvement that are very, very compelling. Systems that can offer scientific discoveries. We're already starting to see them in very specific domains, in vaccine design and protein design, these sorts of things, where the models themselves are proposing new proteins or new vaccines that humans haven't encountered before. So I think we'll see language models get to a similar place where they'll be able to push the boundary further, push past the knowledge that they've been given and discover their own. So I think at the moment it's very high friction. I hope that that gets easier with synthetic data and automated knowledge discovery, that type of thing.

But yeah, the pieces about the internet starting to proliferate with model-written text or the models starting to consume their own content, I think that that is superficially a problem. But in some sense, that sort of information is really easy to filter out. Because what you can do is you can quite easily get these models to decide whether a piece of data, this is well studied, active learning, deciding which data is actually useful for you to learn on. Does it tell you something new or does it tell you something you already know? That's actually a super well studied, very old piece of machine learning. And so if there's content out there that was written by you, you know it very well, that's not going to add a lot of novel information to you. And we have really good methods for detecting when that's the case and filtering it out. So I think superficially, some people might say that, "Oh, this is some big frontier problem that we're going to start running into." I think that it's overblown.

When you think about that in the context of Cohere, what do you guys need to do to be able to continue to have that delightful customer experience and always be at the cutting edge of offering the most performant models? What are the things that you think are going to have the biggest impact on your ability to continue to stay competitive?

I think for us, as the product space matures and solidifies, the difference between models is going to close. And we're already seeing that. I think everyone can feel it. The differences between models are becoming much more marginal, and that will only increase, especially since we're hitting these points of friction where the models are truly extremely knowledgeable, extremely capable. The buyer will then shift from a mentality of purchasing based on who has the best performance, they're all within epsilon of each other, doesn't really matter, to the other properties of buying a product, which are strategic considerations. If I buy this product, am I locked into one cloud provider or am I able to go where I want? Does this product offer customization that lets me build unique models on my data, which become ostensibly IP, right? My competitor couldn't reproduce this model because it's a combination of Cohere's model plus my proprietary data. So those sorts of features, those strategic buying considerations are going to become more and more prominent. That's my guess. That's how I see things moving.

It's in line with this idea of the performance gap between models closing. One of the things that's been really interesting to follow is progress in a few open source models as well. You've got a lot of the stuff that Meta is doing with Llama, and you see they're getting fairly close on performance to some of the proprietary models. It feels like AI was for a long time fairly open. Certainly the work that you guys did at Google has enabled a bunch of other things in a very open way. But now there's this red alert moment at Google, where it feels like very quickly they went from being fairly open to being much more closed, much more competitive minded. Many of the cloud providers like Microsoft, they're trying to lock things down, right? There's this argument that because Meta is not in the cloud compute game, they're much more incentivized to be okay with things being more open and just trying to get the best stuff into their own products. How do you interpret this whiplash experience we've seen with who's open and who's closed? There's this joke that OpenAI has lost a lot of the open in their name, right? So how do you think about that?

I love open source. I came from academia. It's the reason that academia was as fantastic a place to be in as it was, and still is, right? There's still, like you say, massive progress in open source modeling. And that's feeding a lot of fantastic work in academia. At the same time, I think this technology is incredibly powerful. Anyone who's still on the fence about that, burden is on them to explain why they think this isn't powerful. And so it can be used for nefarious stuff. It can be used for stuff that we just don't want to support. And so I think there are real considerations around what we want to put out in the open for anyone to download and do whatever they want, literally whatever they want with. That doesn't mean that you need to go to a fully closed source model where no one gets access to the weights of your model. It might mean that academia and these researchers need to take a more mature security posturing.

So similarly, if you're a researcher in university and you're working on zero-day security threats and you're developing software that could be extraordinarily damaging to global economies or compromising for public safety, even within universities, you're operating in a very locked down, monitored, secure environment to ensure that that stuff doesn't leak. I could see that start to develop in the field of AI. And so these most sophisticated models that are being developed in industry are still going to be given to researchers within those institutions, but with much more security around them. So researchers will still be able to do their work, publish about it, but it won't just be openly distributed freely. I think that's probably the compromise that I see. A lot of people are on one extreme, which is just everything needs to be open. "How dare you not share these weights with me." And conversely, "We need the fewest number of human eyes on this stuff as possible. We need to slow this down." I think there's probably some compromise where academia gets to do what academia does best with intimate access to these models, but also with commensurate security around that.

That feels like the most well-rounded compromise you hear. It's also tough, though, when you're running a business. You need to generate revenue and provide something that people are willing to pay for, crafted in a way that creates value that you as a business can capture. When you think about that for your own business model, where does the value creation come from and where does the value capture come from?

I mean, even in a world where open source was at absolute parity with closed source in terms of model quality, I think that might be true for databases, for example. There are still extraordinarily large and successful database companies because the average company doesn't want to actually manage that. They don't want to build their own databases. They don't want to do all those optimizations, keep it stable, robust, reliable. They want to outsource that and focus on the problems that are actually differentiating for them, right? Allocate their resources towards stuff that gives them an edge against their competitors. As this stuff matures, even in a world where open source catches up and is perpetually at the same point as closed source, the world will need folks to take on building robust, reliable infrastructure to build off of. And so that's where I see Cohere playing.

Similarly, for more consumer oriented products like ChatGPT, everyone isn't going to be pulling from GitHub the latest, whatever open source LLM and running it on their laptop and chatting to that. No, you're going to need a consumer company that is developing cool consumer features and customizing these models, blah, blah, blah. So I don't really view open source as a threat at all. There are some security concerns for just public safety and well-being with these models, which can impersonate humans very faithfully, but there's solutions to that as well.

We've talked about a bunch of different things, but there is this consistent thread that runs through it, which is the necessary components of actually creating business value from this technology. You need access to compute. You have to have data. And then finally, you get down to this element of usability. You might have the most compute in the world and as much data as you possibly need, and then you say to your users, "Here's a GitHub login, Godspeed," right? The average person is like, "I have no idea how to do this." I'm sure some folks are more cagey about talking about who they're working with. But from a customer perspective, are there marquee customers that you point to and say, this is really a great example of a use case that's perfectly situated to leverage our technology effectively?

Yeah, one of them is LivePerson, which is a call center software provider, and they have literally hundreds of thousands of people using their software. And that's a very obvious customer support and customer experience. That's a killer application of these models, helping those agents respond more fluently, more thoughtfully, faster and more efficiently. All of that I think is transformative for that whole space. The other big one that I'm just insanely excited about is Oracle. So obviously Oracle is Oracle. They work with quite literally every enterprise on the face of the planet. I've never seen such a large organization move so fast. They're completely transforming everything that they do with AI. And so that has been really jaw-dropping to watch. I think we only announced a month ago, if I'm correct, three or four weeks ago. But the speed at which this stuff is going to hit products, I think it's staggering. I'm super excited about that partnership as well.

A final question for you. If you were to fast forward 10 years from now, what do you think most people would be the most surprised about Cohere?

Things move so fast that a 10-year horizon is just purely fantasy to try and plan and correct around. I can probably tell you in three years what people might be surprised about. The average person, I think they'll be really shocked about these models' ability, the sorts of tasks that they can carry out for us, the sorts of stuff that we can automate, I think will really surprise people. And that's been happening, right? From creative stuff to knowledge work, people are just consistently shocked at how good these models are. I think that continues pretty much ad infinitum, any sort of intellectual task that you think is uniquely human today. I'm not so sure.

Over the span of 10 years, a decade, where did this go? I think there's meta questions around work. What does work look like? Each employee at a company is now probably augmented with a suite of agents, which that person instructs to go do this for me, go do that for me. There are core lines of businesses which are entirely automated, end to end. I think the notion of work, this decade, is probably going to change quite dramatically. And we're going to have to reckon with that. I'm not talking about mass unemployment. I don't really see that happening, but employment changing super fast, the sorts of activities that you're spending your time on. Yeah, I'm interested to see what your and my jobs look like in 10 years.

I haven't seen it put that way. And it's such a good framework to think about abstracting the humanity from work, right? We might think there are these fundamentally human things that we do because we're human. And it's like, actually, those are just complex tasks. And there's a way to abstract that complexity out into a very specialized piece of technology. And when you strip that away, it forces us to reckon with our own humanity and say, well, what does it actually mean to be a person who's working? And what do we do for work?

Yeah, it's pretty exciting to think about. You have moral principles, right? And you have relationships between humans, and those don't get automated away. We don't even view it as work, right? We just view our moral principles as our moral principles and our relationships as our relationships. But I think those can't be automated. And so a lot of focus will return to that. And we'll do a lot more steering of these agents to go do the work when we've decided what they should be doing. Those sorts of higher level considerations, I think, are where we're going to see work turn towards. We're just going to keep leveling up the intellectual hierarchy as models start to become more and more competent and start to bridge those gaps. That's my very rough guess, though. Everything 10 years out is total speculation. So don't listen.