No Priors 🎙️ 109: How do we go from search engines to answer engines?
With Perplexity AI’s Aravind Srinivas and Denis Yarats (TRANSCRIPT)
EPISODE DESCRIPTION: With advances in machine learning, the way we search for information online will never be the same.
This week on the No Priors podcast, we dive into a startup that aims to be the most trustworthy place to search for information online. Perplexity.ai is a search engine that provides answers to questions in a conversational way and hints at what the future of search might look like.
Aravind Srinivas is a Co-founder and CEO of Perplexity. He is a former research scientist at Open AI and completed his PhD in computer science at University of California Berkeley.
Denis Yarats is a Co-Founder and Perplexity’s CTO. He has a background in machine learning, having worked as a Research Scientist at Facebook AI Research and a machine learning engineer at Quora.
EPISODE TRANSCRIPT
ELAD:
In this podcast, we're talking with the leading founders and researchers and AI about the biggest questions
With advances in machine learning. The way we search for information online will never be the same. We're back again to talk about the future of search. This week on the No Priors podcast, I'm excited to introduce our next guest from Perplexity AI Perplexity. It's a search engine that provides answers to questions in a conversational way and hints at what the future of search might look like. Aravind Srinivas is co-founder and c e o for Plexity. He's a former research scientist at OpenAI and completed his PhD in computer science at UCBerkeley. Denis Yarats is a co-founder and perplexity C T O. He has a background in machine learning, having worked as a research scientist at Facebook, ai and also as a machine learning engineer Quora, Arvin. And Denis, welcome to the podcast.
ARAVIND:
Thank you for having us here.
DENIS YARATS, GUEST:
Thanks.
ELAD:
Oh yeah, thanks so much for joining. So the two of you alongside Andy Konwinski created Perplexity around August or so of 2022. Arvin, do you wanna give us a little bit of a sense of why you started this company and what the core thesis of complexity is?
ARAVIND:
Yeah, sure. Actually, Elad, you're our first ever investor who offered to invest in us. So although the founding days, in fact, I remember the first ever idea we talked about in Noe Valley where we're sitting in the open space opposite Martha and I was telling you, oh, it'd be cool to have a visual search engine. The only way to disrupt Google was to not do text-based search, but to actually do it from camera pixels. And you were like, uh, this is not gonna work. You need to think about distribution. And search was always the core motivation for me and Denis and many others of the company. We were just bouncing around ideas and then a SL open air around that time. And then we le uh, left and Denis also was still at Meta. Then he also left and Andy came in to help us sort of incorporate and get the company rolling.
So that's sort of how the company started the space of L and was exciting. Generative, generative models were really exciting and in general we were motivated about search, whether it be a gen general search or vertical search, and we were bouncing around several different ideas. One of the ideas that you gave us was the working on text to sequel and like we were pretty excited about that and started prototyping ideas around that. And I think Denis also like was hacking with us on building like a Jupyter Notebook extension with like, uh, copilot for every cell and then we were trying it with SQL around databases, but it's all like a bunch of non-linear pathways to eventually get to where we are right now.
ELAD:
Yeah, absolutely. And hopefully I caveated whatever feedback I gave with I'm probably wrong, but since, uh, I think I'm often wrong on <laugh> on directions,
ARAVIND:
<laugh> No, I think, I think whatever you said still applies. Search is tremendously a distribution game as much as a technology game.
ELAD:
Yeah, I think one of the really impressive things about perplexity is the rate of iteration. And to your point, you've, you've gone through things like text to SQL copilot for the next gen data stack and I've always been impressed by how rapidly you've just been able to point in a direction, iterate really fast, prototype something, see if it's working and then move on to the next thing. And to your point, you always had search in the back of your mind. Remember even as you're prototyping these things, you were talking about indexing aspects of Twitter or other sort of data feeds and then providing search on top of them. How did you end up building a team that can iterate that rapidly as well as a culture of fast iteration, like other specific things that, that you all do as a team to help reinforce that?
ARAVIND:
Yeah, I'll, I'll take the first part of this question and then also let Denis answer this because he's a big part of why this is happening. We both are basically from an academic background. So in general the culture in academia is to, you have hundreds of ideas and you just need to try them out pretty quickly, run a lot of experiments really quickly and get the results and iterate. So we come from that background, both of us. And so, so that's not really new to us, it's just that when it comes to trying out in products, it it, it's not just a result you get from running an experiment. You actually have to go to users and make them use it and talk to companies or customers or potential people in a company who will be using a product and get feedback. So there's that aspect of operational work that needs to be done to get results for experiments.
And there's this aspect of quickly doing the engineering to get it to a state where you can show it to people. So both of these things had to come together and that's why the company exists and tennis is incredibly good at engineering and recruiting. And so we found our other co-founder Johnny Ho, who's also like a big reason why Perplexity operates this fast and full credit to Denis for helping recruit him. He was the world number one at competitive programming. It's like being Magnus Carson programming. He got pretty excited about all the stuff that Denis was showing him. So it's sort of like few people help you accelerate a lot and Jo Johnny is like one big reason for that. And Denis sort of continued doing that by getting more and more people in the fall.
ELAD:
Are there specific things, Denis, that you look for when you're hiring or when you're screening for great technical talent? Because I mean, one, the way one person put it to me is that you have a team that can build almost anything really fast. And so I'm curious like are there heuristics, are there places that you look for people? Like how do you think about hiring?
DENIS:
Yeah, that's a, that's a good question. I'm kind of like looking for the people who maybe have like this general interest in like applying this emerging technology like L L M, right? So like a lot of people hear about it, maybe they don't have a specific background, but they're like very curious to learn about it and they wanna put like a lot of energy into it. So I feel like that's my number one indicator. So like personally, I would rather like get somebody who has this burning desire to work on these things rather than somebody who has already has a lot of experience and not gonna put a lot of, or like as much effort as other people. I think another concept that we still using to this day, even though it like takes a lot of time, it's we have kind of like this trial periods I would say.
So we basically get to know a person a little bit better and once we see there is like good chance we might hire this person, we then ask person to sort of like work with us for like some time. We kind of like intend to keep our number of employees very small. I feel like it's much faster to operate and because of that each hiring decision is very important, you know, and we wanna like optimize for the best people. Uh, and that's why we kind of like run this trial process where, you know, we basically making very sure that we wanna work with this person. So I think that that's been helpful.
SARAH:
Is there anywhere you've been surprised as you've made hires where you feel like, you know, the signal was wrong or the trial surprised you?
DENIS:
Yeah, there has been few exceptions obviously. I think that's the part of it, but I feel like it's definitely at least like comparing to my prior experience, it's definitely a smaller chance where you're gonna get surprised, normal interviews that, you know, big companies run, you know, you have like four or five meetings, like 4 45 minutes each and then you basically make decision after that. Sometimes it works, sometimes it doesn't. But I feel like the way we do things, it's gives us like much more confidence that we're gonna make the right decision and, and I think it's obviously it's very useful for us to get the signal, but I also feel like for candidates it also makes sense to, uh, it's useful to make better decision as well, right? So they can understand do they wanna work on things like we do, do they wanna work with the base that we do?
I think like one important thing for us and like many candidates sort of like don't wanna maybe do this is the energy we put in into perplexity. It's kind of like work-life balance. Maybe not the ideal, but that's the only way to sort of like beat competition. It's sort of like a trade very fast and um, to great things. So that's why we kind of like meet you have this alignment at the beginning. Yeah. If you don't have a clear idea of like, you know, which product you're going to build or which market you're going after and you still wanna be giving yourself a shot at success, uh, iteration speed is the only thing that you can hope for. And so we decided, okay, like until we actually figure out our business and product, we'll make sure that this is non-negotiable. Like they, they people should iterate really quickly with us.
ELAD:
Yeah, I feel like the only real advantage that a startup has relative to incumbent is speed, right? Because incumbent has more people, they have more money, they have more distribution, they have more product. And so really the only thing you have is speed in the early days of mobile, you know, I was working at Google and I I really helped get that team up and running and then I started a company that Twitter bought on a Twitter. There was this weird meme internally that the only people that you should hire for mobile were people with prior mobile experience, which I thought was a really dumb idea, right? It was more you just find great engineers and they'll figure out how to write for iOS and stuff like that. How do you think about that in the, in the ML world because there is this sort of dogma right now I feel that people believe that only somebody who's trained an L L M before can work on an L L M center company or things like that. So I'm just sort of curious, uh, you know, how you think about that.
ARAVIND:
Yeah, actually my thinking about this was already pretty clear because of having seen how open AI went about this. If you look at the people who did G P D three, eventually they went on to start anthropic. All of them are basically from physics background, the C E O dios, so physics PhD and Jared Kaplan is a physics professor. He wrote the scaling loss paper. So they basically open AI really succeeded in bringing these extremely talented physics people who wanted into machine learning and, and they came into the sort of language models in scaling and that paid off well for them. And similarly, their whole engineering team, if you look at an engineering team is just Dropbox and Stripe people all like software engineers, solid people who can build infrastructure frontend and now they're doing AI work. So it's always been clear to me that in order to work in AI, both research and product, you do not need to already have been in ai.
And that's being shown clearly with Johnny Ho, our co-founder. He was not an ai, he was a competitor, programmer, a trader. He had worked at Cora for a year, but he's as good as anybody can get in picking up new things. So the other thing also is that LMS are sort of in this weird territory where the people who use the LMS for building stuff understand it better than the people who actually did great in dissent and train these models. Like you could find a PhD student at Stanford, at Berkeley who would know a lot about how to train the model, but they might not be the best person to build a product with it.
DENIS:
Yeah. I wanna like quickly add to this point. So I was um, at early days at Facebook, a research in Menlo Park, the office, right? And at that time, and that, that was honestly one of my reason to do PhD is there was like kind of this like very exclusive culture.
So if you like you, if you don't have a PhD, you're not gonna be a research scientist. And I didn't like that too much, so that's why I decided to do PhD later on. But uh, turns out through my experience, the best research scientist, also very good engineers, like very good engineers and we've noticed like through Deep Mind and open AI is just like the, the companies that made them this the most progress over like last six years or five years were companies where they're like extremely good engineers and it's kind of like I didn't like from the beginning this I guess view from like academics, that's just like if you engineer, you're probably not, you know, either like smart enough or you're not going to like great things, but turns out it's actually the other way around. So that was, uh, also I think like motivation and I feel like you don't need to be, you know, this like very impressive like academic with PhD to to great things.
ARAVIND:
Yeah. This also goes back to the thing Denis said earlier that you wanna find the people who really wanna get into ai, uh, rather than who are already in ai. And every, every company that's Scott in BIG has done this in their early days, including Google. Like they got a lot of systems people compiler, Jeff Dean was a compilers and systems person and then they, they got an SLOs, like a systems professor at, they got all these amazing people and they told them, Hey, you know, guys, like we, we are having the most interesting computer science problems to solve here and we can scale it up. So come work on search. So it's not like you have to go, the few people they hired from information retrieval or search background, but most of the celebrity researchers that they have right now are just, uh, people who wanted to get into that space for the first time.
SARAH:
It's funny because now the now the desired pedigree isn't the PhD from whatever, you know, academic lab because the industry lab's filled with physicists and engineers and people not necessarily from the domain made the most progress. And so if you want somebody who's worked on, I, I hear the argument, you don't actually need it. Like you want people who are really smart and motivated, but a lot of companies are looking for somebody who has, you know, experience with x billion parameter training runs and those people come from now open AI and deep mine and such. So it's quite, quite funny very quickly how, uh, the pedigree changed. What's been the area of steepest learning curve with both of you coming from these like research engineering backgrounds?
ARAVIND:
For me it's like how to run company. That's mostly the what I'm doing here. I'm not doing much, uh, le core engineering, tennis, <inaudible>, so that was not easy at all, but I've had the opportunity to learn from many good people including you out here. So if at all there is an easy way to do it. It's like getting advice, rapid advice from people. The other thing is also like when you're making a mistake, like being super brutally honest with yourself and listening to feedback and quickly course correcting it, that was also like very new to me.
DENIS:
Yeah, I guess for me probably it w it was, you know, building a team, kind of like organizing everything in terms of like who does what, how to prioritize things, what to work on. I think like being a small team, it's even more important to identify this one thing that you wanna work on and like put all your focus on it.
Early on we were like had this sometimes made this mistakes that we're trying to go after, like several things. That's why <laugh>, even though we're like six months or like seven months old company, we actually built so many things. We had like, you know, Twitter, we had like, uh, Salesforce integration, we have like HubSpot integration, like many other things that we never like released. One of the interesting thing is just like you, you have to have very precise focus and and just like go after it. Yeah.
ARAVIND:
Also as leader, uh, if you wanna lead the company and if you cannot do everything, the people will lose faith in you, right? They think you don't, you don't really have any clarity and that's why you're making them do one new thing every week. So you, you have more responsibility to sort of get it right and think more clearly.
And even if you're wrong, like do one thing wrong and at a time and course correct rather than doing five things at once and seeing like which one went, it's, it's difficult for us coming from the academic background for this particular thing because in academia you're basically taught to hedge. You have like one first author project and like three or four co-author projects and like one of them might become a big hit and might change your career, whereas that's not how you should do startups. Like you really have to focus and iterate multiple times on one thing or a few things. So that took us a while to quickly learn.
ELAD:
Yeah, makes a lot of sense. I think everybody goes through very similar paths as they start a company. You know, one thing that I think is really interesting right now is we're, we're at this point where consumers are really starting to wake up to how machine learning and AI is changing search. And to your point, you all were thinking about this actually before chat G P T and before the Sydney News and before being integrated all these things. So you were quite early to realizing that yeah, you know, this is gonna be a really core piece of search. Uh, perplex City's mission I I believe is to create the world's most trusted information service. How do you think about important product points around factual accuracy bias and presenting aggregate information for users and things like that?
ARAVIND:
Yeah, so I guess you're, you're also from a PhD background, right? So when you write your first paper, the thing your advisors teach you is you only have to write things that you can actually cite anything else that you write in the paper, is your opinion not not a scientific fact. And so that's sort of stuck with us pretty closely and that's sort of why we did the first version where it's citation powered search. So for factual accuracy we, our first step towards that was making sure you can only say stuff that you can cite. This is a pretty subtle point here. It's not just that we want to retrofit citations into a chatbot, that's not what perplexity is. In fact, it's more like a citation first service that it'll never say anything that it cannot say. So if people have tried to play with it as if it's like Chad JP p t where like tell me who are you or like things like that.
And even for those questions it would still go to a search engine, pull up stuff and come back with an answer. It's not gonna say I'm perplexity, I'm like a bot design, how are you doing? Or something like that. This is because of our obsession about factual accuracy. Like even if it doesn't have a personality or or a character in it, we don't care. We want, we only care about the other thing which is obsession about truth and accuracy. Yet the second point you mentioned about aggregation of things when when you mash up multiple sources together, you might end up hallucinating. Like for example if there's multiple people, the name Elad Gil and like one of them is a venture capitalist and now some o some others is like a doctor or let's say even if it goes to your own LinkedIn and things like, oh the Elad Gil who did a PhD in biology is different.
Elad Gil from the venture capitalist. Because someone might think that, right? It's pretty unusual background. So then that it might end up coming up with some entertaining or funny summaries that collate different sources together. We still haven't thought of a proper fix to this, but one thing is obviously as language models keep getting better, they're gonna understand these things, these subtleties even better and we are already seeing that. And second thing is we are giving users the power to remove sources that they think are irrelevant. Just like how you can curate sources in Wikipedia. So we sort of working towards this accuracy and the bias issues step by step at a time but I, I feel like it'll take more iterations to get this truly correct and I also don't think one LM will just magically solve this problem. You need to build an end-to-end platform where users can correct the mistakes of an lm and that also means you need to design the platform where the incentive is right for the user because this cross will be used in the other way where users can use it, hide information. So we haven't really thought through all these issues thoroughly, but we are committed to sort of figuring these things out over time.
ELAD:
That makes sense. I guess in addition to that or maybe related, you've done an impressive amount of research and reinforcement learning. What's unique about the way the perplexity uses reinforcement learning and how does it tie into these plans?
ARAVIND:
We like our leche like reinforcement learning from human feedback where we use the contractors to, we collect feedback from the users on whether they like the summaries, the completions or not. And like we use contractors to do the sum ratings themselves and these days even LLMs can be prompted to do the work, the contractors do. Andros written a paper on that. So all these things are getting really very efficient to do. So that's sort of how we have been thinking about greenforce and learning right now, but we haven't gone beyond that to think of like agents and browsers and things like that. We'll probably focus more on the first part for the next at least six months to a year.
DENIS:
I want to add on this aspect full blown like a lhf is you know, definitely something we can look into that but there is like several many steps that you can have in between that's significantly can increase your quality. So for example, I mean even using something like a rejection sampling, you know like discriminate on top, it's kind of you wanna like shift, you know maybe you have a several samples from your L L M and then you can can rank some of them using different model peak only dose. It is in a sense kind of like one step of reinforcement learning but it helps a lot. It's very effective.
SARAH:
You guys have talked about how trustworthiness is more important to you than like, I don't know, personality, the ability to play with a bot. Like do you believe in chat as an interface? Like where's the line between chat and search given perplexity does support for example, like follow ups and things that are more conversational?
ARAVIND:
We think chat UI is the future. People are using it pretty heavily. At the same time, if you can try to get the answer right in the first attempt, you should, right? Like you have a responsibility to save people's time. It's not like Google doesn't do some kind of chat implicitly. Like if you go to Google you always get related questions, follow ups people also ask for and things like that. This is sort of implicitly making you click on it and you know like you get a follow up question that you sort of get an experience without the chat ui. So I feel like it's more like whether it makes sense for the particular experience you wanna provide for your end user. And in our case it does often you do not get the answer you want in the first attempt and you shouldn't feel the burden to get it.
Also like sometimes you could ask a question by just asking for the keyword and then you might realize like, okay this is actually what I want to ask for. I've seen this in live and people use it perplexity but they just couldn't know how to phrase this question the first time so they use multiple attempts to get to the right question. So there are things like that whereas the questions get more complex, the chat UI makes a lot more sense than Google's ui but for like obvious questions like we just wanted to know whether it's uh, gonna rain in Bay area the next one week, why do you need to keep asking more, right? Like you just get the answer, the first attempt and we want to support both these experiences in perplexity
SARAH:
More broadly. I'll ask a a dangerous question but what do you, what do you think the future of search looks like, right? Five years plus out? Do we still have monolithic horizontal providers if the players change, do we get more embedded apps like contextual search as a feature in different places? Are there agents that do things for you? Like what do you think it looks like?
ARAVIND:
I think there's this phrase that's becoming popular. I think Carati was the first one who tweeted it and Satya Nadela is also using it, it's called answer engine instead of a search engine that directly tries to answer your question instead of providing you a bunch of links or just snippets from the first link. So we believe in that like perplexity is the first conversational answer engine. Truly the first, I think nobody built it before us. I believe answer engines will become its own segment, market segment. Like just like you have a default search engine that that'll be a default answer engine over time if these things really work and the burden of getting ranking and search right will reduce in the sense answer engines can do more heavy lifting than search engines. As these things get really good and like way fewer hallucinations and even, even if they do hallucinate, people can still go and click on these links.
They will eventually prefer this experience over the regular like 10 links or 20 links UI that Google has. So that, that's something I'm pretty confident about and, and I think the sort of asking follow up questions will become more of the norm. The number of queries in perplexity that go to at least one follow up has been increasing ever since we released the chat ui. So that that will keep going up. People will get used to the sort of experience where they're encouraged to ask one more question and they're okay with not getting the answer right away so that that'll happen. The third thing is like actions, people will be more deliberate in what they search for and try to execute an action on top of the search results they consumed. So that's definitely likely to happen. It's already happening. If you go to Google and book a flight, you just like fly from SF to Seattle, you just directly click on the book button.
So that's gonna happen more frequently in the chat UI to, and this will become an assistant more than just a search or answer engine. And I also think the fourth thing is there will be some sort of like much fewer traffic to the actual content site. Like very few links need to be consumed uh, in perplexity. In fact, like we only, we don't even cite more than filings. It's, it's a deliberate decision. Like a lot of people ask us, well can you add 10 links or 20 links? Can you just show all the links together? You put the summary at the top but you also put all the 10 links. The usual, I want both. It's a decision just, we just made that no, no, no, that's not the right experience for you. You actually need to feel the difference here. So we only made only three to four links or like in fact the first version shipped it to like three to four links I think and like 50 words sum reason we expanded the num summary more. So I think we, we just have fewer tabs open, we'd only open the tabs that we really need so that the, all the sort of behavioral change in the consumer is likely to happen, whether it be through US or Google itself doing these changes. It's unclear but this is where it looks like it's trending towards.
ELAD:
When do you think we're gonna have that transition from uh, almost like a pull versus push model, right? Because I think right now you go and you ask for certain information and you ask for it repetitively, you know, I've been checking the weather every day versus a world where you have agents that are effectively understanding your intent ongoing and providing you with an information in a sort of a push base way. How far away of a of a world is that? Is that a year away? Five years away, 10 years away?
ARAVIND:
I feel like we can do it now. In fact like Google now was an attempt to do that. If you keep checking for scores of a favorite football club a automatically give you a push notification for their next latest match that you might have missed. They tried some of these things already. So I feel like we can do these things even better now with language models. So yeah, I think it'll happen in a year more than five years.
ELAD:
And and do you think it's gonna be fragmented in terms of each, and I know this is all uncertain, right? It's kind of predicting the future is never correct, but I'm just curious how you think about, is there gonna be like a agent for your Google Drive or an agent for your GitHub or an agent for your email or is it just sort of consolidated eventually into one central service?
DENIS:
I mean it's, it, it's getting pretty clear, you know with Google the first few results, like at least like my pattern of using Google, I see like first two, three results I basically skip most of the time because it's here or like some ads and that's like not ideal experience obviously for the user. And it's also kind of like why it might be very hard for Google to fully launch this system like answer engine because it just like breaks ization strategy for them. A as Arvin mentioned, you know there's gonna be like fewer clicks and Google wants you to click on those things, right? So that's how they make money basically. I think there has to be this like new paradigm where you kind of like you get your answer quicker but then maybe you can monetize it better on it maybe now like you can help user to make the decision faster and then you can show like much more targeted and accurate ad. So then you know like if user clicks it, it's gonna be maybe they're gonna buy something or or whatever. So there is gonna be a few clicks but each click is gonna be more expensive. And I feel like overall user experience has to be better because you just, you just get what you want. You don't have to do any extra effort
SARAH:
In this era of fewer clicks. Right? If the summaries and chat that's just giving you the answer, how does that impact the relationship of search or answer engines with publishers, right? Does that remove the incentive to publish information on the internet? Does it become more adversarial? How do you guys think about that?
ARAVIND:
Probably not, I feel like, so it's sort of like whether you cite a paper or not sort of thing, right? Like you would cite a paper if the paper was really good, it's actually gonna bring back the whole concept of page rank even more. But the, but the concept of page rank was inspired by academic citations from what I've read, like a very important paper tends to get cited. So when you're in the sort of citation based search interface now the better content the A publisher has, the more likely it'll get cited by an L L M unless humans figure out answer engine optimization are LM optimization, which which I hopefully they don't invest effort into, but in general it's unlikely that it's as easy as seo, which is keywords because LMS are gonna be much smarter in understanding relevance to a query. So I think it's just gonna incentivize people to publish higher quality content in order to get cited by an L l M powered answer like CK or things like that try to do like you wanna own your content and you publish it and make sure it's high quality and you have your own like set of subscribers.
So people put a lot of effort into that more than writing tweets. So something like that is likely to happen with this interface two, but it's unclear exactly how to make all this monetize its scale like you know, the click based ad engine that Google has. And I think it's super interesting problem and it's amazing that many companies are trying this at the same time. So even if one company figures this out, like others can also like benefit from,
ELAD:
If you look at a lot of the biggest consumer services, they took a while to figure out monetization. So for example, Google's first attempt was literally to sell search results so they got paid per thousand search queries that they'd syndicate to others and then they built out an enterprise appliance, a literal piece of hardware that you'd install at enterprises to do search and inside the enterprise. And then eventually they came up with ads and really realized that that was their feature path for monetization. And then similarly, you look at YouTube and people said that would never make money and Facebook would never make money and all these things would never make money. And then of course they monetize eventually how are you thinking about monetization or is it simply too early and it's more about just getting market share and then you know, at that point you can iterate on the right model.
ARAVIND:
Yeah, we have multiple thoughts about this. Obviously just like Google was paid to sell the search results for thousand queries, like, I mean Bing has APIs like that too and people ask at least more than thousands of people have emailed us or messaged us in various different ways to ask for a perplexity as an API and we haven't done that yet, but that's an obvious monetization strategy. And the other thing is obviously prosumer of this where like we are already sort of beginning to see our Chrome extension pick up rapidly heading towards a hundred thousand users and extensions like Grammarly have the sort of free version and uh, prosumer version which has more features in them. So they're ways to do that through the browser extension as a productivity assistance sort of thing. We already see some kind of search pilot every time you're on a site you can a, you can ask it to do things for you.
And then there's the whole, as we keep getting more and more traffic onto our site, like say hundreds of millions of people come to our site at one point, eventually that becomes a ripe ground for serving ads. But we need to not make the mistake that Google did of combining ads into the core search product itself and figure out an alternative pathway like Facebook did and, and that might work out better for us. Subscription based search has been tried by other companies and that's something that chat G P T is also trying. So we, we don't know yet if it's high margin enough and so if a bigger bemo like Google or Bing just put out the same or like even 80% as good as you for free, then you're never gonna make it as a subscription product. So we are likely to stay away from that pathway, but we don't know yet.
And the final piece is like if perplexity becomes like something that a lot of people want to use for their own internal data, their links or their bookmarks or their company, and if we can make it easy for them to build that and become more like a platform which everybody can use, then that's likely to lead monetization to. So there are like so many different rollouts possible here that we don't know yet which one we'll actually go for, but in the, in the short term, we are more focused on growing the product, the users, the traffic in fact improving the whole experience. Like there's, I feel like Google and Microsoft will pretty much do the same thing we have right now as well as us. So we as and as we discussed in the first part of the podcast, like we need to operate with more velocity, ship more things and stay ahead of them in terms of the core value of the product itself.
ELAD:
Mm-hmm <affirmative>. Yeah, that, that makes a ton of sense and I think, you know, to the point before there, there's probably lots of paths to monetize once you have a lot of usage and so it's more just, you know, figuring out what's native and natural to the product. You've mentioned earlier that one of the things you learned from academia is fast iteration and I feel like most academics I've worked with are almost the opposite. You know, like I actually feel like there's a lot of pre-planning and a lot of discussion and there's, there's less of a bias to action and so I've been very impressed by the bias to action you all have. What advice do you have for researchers who are now deciding between an academic path or joining a company research role or starting a company? What advice would you give them given that you've gone through a recent transition that's similar?
ARAVIND:
Yeah, firstly we both share, uh, Peter Appeals also Denis' thesis advisor and my advisor and he's a pretty different academic from the others. He pushes people to get results pretty fast. So that's a reason why we both are like that. And Denis also worked in an industry where he can't operate that slow. So for the advice to academic researchers, I feel like it's super hard to be an academic researcher right now, especially a PhD student is one of the worst chops you can go for in a time when AI is so hot and so highly paid and you can do a startup or join a startup, you just sort of give up all that and the buzz every single day you see for on Twitter or other news journals and still focus on trying to build a future. It's more of a mental thing, I would say more than like picking the right problem.
Like even having the composure and the maturity to sort of stay poor and like work on hard problems is super hard right now. Very few people in the work can have the ability to do that. To be very honest, I only came to the United States for doing a PhD because there was no other way for me to come to the United States. Like I couldn't take a loan for a master's or something like that. So PhD is fully funded and like sponsored. So that, that was the biggest reason I actually wanted to come here more than doing a PhD. If I had the ability to get a job in industry, I've gone for that. So there's other reasons you might wanna do a PhD, but okay, if all these things are sort of not a problem for you and you still wanna do something, I think it's best to look for alternatives to the transformer alternatives to like language models that sort of radical directions than trying to improve them because there's so much incentive for the existing companies to do that.
A lot of people think open air has no incentive to improve G P T, the core architecture or some, or like the model or something that's far from true. Like even if they have a lot of money, they would want to make it more efficient and train even bigger models that make better use of compute. So I feel like it's best for people to sort of look at places are kind of controversial or radical and, and that would mean even questioning transformer itself, which is actually one of the best research problems to work on. That's what I would work on if I were to do a PhD. I would work on trying to be write the next transformer paper.
DENIS:
Fully agree with this. I think I would probably wanna also add at least like from my experience, I think it's best to not go to PhD right after undergrad and kind of like spend at least like a couple of years in industry.
I think that goes to my point to me like the best researchers are those who can, who are also very good engineers or like you can get this like valuable experience of being, you know, a good engineer and then it's, it's basically gonna like propel your PhD. You can do things faster and especially, you know, like now AI becomes a little bit more like engineering than it used to be, right? So it's, it's just like the skills, uh, essential either you, you won't be able, especially if you wanna work on like a alums or like large scale stuff, you have to be very good engineer. But uh, yeah if you still want to stay on like very academic side where you just like think like very tip ideas, then I agree with Arabic and you probably wanna completely ignore LMS and just do something very radical. There are some other ways to do stuff that's still useful and pretty different. Like I think Stanford has some students doing this like state space models and flash attention. Uh, there's some really good papers like that coming out and if flash attention's already being used to improve efficiency of lms so you can sort of do such things or you can go work on video or video generation, stuff like that, that's just still out there. Yeah,
ELAD:
Makes a lot of sense. Well Denis, Aravind, this was a great conversation and it's really exciting to see how the progress you're making on an important area. So thank you so much for joining us today. Thank
ARAVIND:
You. Thank you for having us here.
DENIS:
Thank you for having us. Thanks.