Fuqua Insights Podcast: How Do 700 Million People Use ChatGPT?

Intro 00:03

Welcome to Duke Fuqua insights, a Podcast where we explore faculty research and the actionable takeaways for business leaders at every level.

Jenny Laurence 00:15

Everywhere you look, people are debating how generative AI will change our jobs, the economy and society, even with all the debate, one big question has been hard to answer, how are people actually using it? A new study led by Ronnie Chatterjee, professor at Duke University's Fuqua School of Business and the chief economist at OpenAI gives us a clearer answer by analyzing more than a million ChatGPT conversations, his team uncovered patterns in where AI is already making an impact in our lives, at home and at work. I'm Jenny Lawrence, an MBA student at Fuqua, and today I'm joined by Professor Chatterjee to break down this important research. Thanks for joining me, professor.

Aaron Chatterji

Great to be here.

Jenny Laurence

So to start, I'd love to hear about how this study came together with so much speculation about AI's future. Why was it important to study how people are actually using ChatGPT right now?

Aaron Chatterji 1:01

Well, we really want to think when it came together. I had to go back to my graduate school training and my work here at Duke University. So I did my PhD out in Berkeley in the bay area at a time of a lot of technological change. And it wasn't an accident that as a young economist studying at a business school that I got really interested in how technology was influencing corporations, but also society. And I learned an approach at Berkeley that I think has carried me through my academic career, which is really being data driven in my research, which is, you know, one of many ways to approach work with something that really resonates with me and asking this question that one of our professors would ask all the time at Berkeley, which is, what's going on here, you know, very, very, just sort of like, sort of grassroots question about what is going on here. And as students, we learned this idea of looking at our data, and one of my colleagues called it having a cup of coffee with your data, really understandinChatGPT and the idea of writing a paper that could be for an academic audience, for the National Bureau of Economic Research, but also kind of reach the media, like we've been covered, you know, on CNBC. I think the view did a segment on us, which was kind of funny. The idea was like, Okay, let's just try to figure out what the use cases are, how people are using it, how high level demographics correlate with that usage. And that is sort of the genesis of what became the paper of how do people use ChatGPT, which, again, the title kind of says it all. And I'm really blessed to have a team of fantastic economists and data scientists on my squad to help me do that. You'll see them represented both in the CO authorship, of course, but also in the acknowledgments and the people who've helped us on the paper. So that's kind of how it got started. I got to credit my PhD training and also all my work at Duke over the years and a great team at OpenAI

Jenny Laurence

Yeah, incredible. I mean, I have to say it's great that research is out there. I've definitely seen it, you know, all the way down through my LinkedIn, day after day. So it's been great to see everyone's reactions to that. So one of your headline findings is that most ChatGPT Use isn't actually for work, but for everyday, personal tasks. I mean, Did that surprise you at all?

Aaron Chatterji 3.40

You know, it's so interesting because it did not surprise me, or maybe it surprised me in a different direction than a lot of the commentary has been. And I'll explain it like this. We're looking at the consumer ChatGPT. That's the one that I used, you know, before Duke had an account using my personal sort of credit card to pay for the subscription. So for personal use, I thought, gosh, well, this is going to look like a consumer app. You know, I think about the most popular consumer apps like Tiktok or Netflix. If I told you 30% of what you did on Tiktok or Netflix was for work, you'd be like, Wow, that's really high, right? Give it, right? So for me, the idea that 30% on this, like, amazing consumer product is actually for work, was actually surprising, maybe in the opposite direction, in the sense that, like a lot of people, I think, thought, oh, that's that's not that much. It's actually a lot. If you think about traditional consumer products, we didn't do this study on the Enterprise part of chat, GBT, so you can think about, when you join a company, they might sign up for an enterprise account, and you'd be using chat in that context. If you're using it for coding, you might use Codex. If you are building apps on top of our API, you'd be doing that, right? So for API, for Codex, for enterprise, that's not part of the study. So a lot of the work stuff is, let's just call it to be discovered and researched on. This was the consumer idea. So the idea that 30% of work, if I'm surprised, it's I'm surprised it's so high, it's amazing how productive people can be with consumer application. But that's what we do research, right? That's super fun. And I didn't know exactly what the norm would be, and that was what was exciting about the results.

Jenny Laurence

So the study introduces a useful framework of asking versus doing. Could you explain those categories a bit and what they reveal about when, whether people see ChatGPT more as an advisor or more of a task-completer? =

Aaron Chatterji 5:14

I really have to credit David Deming, who's our co-author from Harvard, with helping us think through this. You know, economists in general are trying to build models, ways of thinking about tools like ChatGPT, and, you know, for a lot of people who are non economists, they say, Why do economists spend so much time thinking about frameworks, writing on all these Greek letters? It's so we can come up with a clear language to explain what's going on. And so David is fantastic at this. He is, you know, a scholar who's very well known for his work in labor economics. But one of the things that really inspired me to want to work with him is his work on soft skills and how important they are in the economy. And he's built a whole framework and taxonomy for thinking about the impact of soft skills. So when David and I started working on this with the team, David said, you know, I really think there's a framework to think about how people use ChatGPT And what he came up with, and the rest of us were persuaded by was asking, doing and expressing. And what was nice about that, it's like, it's very parsimonious. There's three categories, I'm sure. We're, you know, leaving things out, and we're not covering things as well as we could. But it's a nice parsimonious framework, and it generally describes the kinds of things I think any user of ChatGPT kind of resonates with when I think about asking and doing, what's interesting is the asking messages still make up a higher proportion. Although doing is growing pretty fast, I would say, for people who are ChatGPT nerds or watching AI closely as we release more agentic capabilities, and frankly, as people learn more how to use the agentic capabilities, you might see the doing tasks start to increase even faster. But right now, people are using ChatGPT as an advisor to support decisions to inform choices. I think it points to a role for AI in our society and in our economy. It's a little bit different than what's sometimes discussed in the press. The idea here is like not about sort of augmentation or automation in the way it's often discussed, but people are making choices and decisions, and they need help. They need advice. And those could be professional decisions. They could be decisions at home, and they're asking ChatGPT The way they would ask a friend or a co worker or a mentor, what do I do here? And give me some give me some thoughts on this, rather than, let's say, outsourcing whole cloth or for ChatGPT to write something or doing a fully agentic end to end workflow. And so that's what asking is. And I think for a lot of people, it really resonates with how they're using ChatGPT today. I would say the reason we do research, the reason we track this over time, it could really change. And given how fast-moving this technology is, I can do this study in six months, and maybe we'll find different proportions. I think I'd like to keep these three categories, though, because I think it'd be nice to track them overtime.

Jenny Laurence

Yeah, no, absolutely. I mean, it's interesting also, because you know, the efficiency that you can gain, instead of having to ask your friends and family, and, you know, working through whether what they're saying is correct or not. At least you have this

Aaron Chatterji 7:43

Jenny, don't fact-check your mom. Okay, this is not, this is not what ChatGPT is used for, not a recommended use. Hear from the chief economist first, but I will say you've picked up on something I think that is really important, but somewhat nuanced for folks, is that you can create a lot of economic value by helping people make better choices. Yeah, and a lot of people are looking for AI's impact, and they're studying the GDP numbers every quarter, or the employment numbers every quarter, and they're saying, Ah, you know, when is AI going to show up? Really important. We should be watching that stuff. As an economist who also worked in government. That's really important. But actually where ChatGPT can really matter and other AI tools is creating what economists call consumer surplus, which is essentially what you've described, which is, look, how much would you pay if I turned off ChatGPT to have it back, right? And if you're making better decisions, if you're able to get to things more quickly, maybe you didn't have the right person in your network to ask, Hey, what should my concentration in my second year of MBA be? Right? Now, you have good friends on that. But like, there could be questions that you don't have the right network, right think about a kid who's the first in Stanley to go to college, and might be asking, Hey, how do I make this decision? That kind of consumer surplus for a product that you know, for a lot of people, is free, that doesn't show up in GDP, but it can be really, really valuable. A couple economists from Stanford, led by a team led by Erik Brynjolfsson, a fantastic economist, and they estimated the consumer surplus of chat, GPT and AI as like, $100 billion a year. So like, that's an example of something that's really important economic impact. Economic impact but doesn't show up in the headline statistics, which is important

Jenny Laurence

Yeah, absolutely. Although that being said, I will still be going back to my mom before.

Aaron Chatterji

Oh, definitely. There's only some advice that mom can give and never go to ChatGPT for that, agreed, agreed.

Jenny Laurence

So your study also highlights demographics. And so I just want to touch on that for a second. Younger people, women, lower income, countries are leading in adoption. So how might these patterns influence global competition and access to opportunity? As you talk about

Aaron Chatterji 9:28

yeah, their rates of adoption are increasing, I should say, and there's still gaps, but the rate of change has been really, really fantastic. And to your point on the demographic trends from what we can tell from sort of typically female names, typically male names, the gender gap has closed as well. That's really important, and I was really gratified to see that for two reasons. One is, regardless of what the data shows, just having reliable data to see where these things are is really important for society. People are making big decisions based on their perceptions of who's adopting AI and whether it's going to create inequality across a lot of different dimensions. Those are, like, super important discussions. People inside OpenAI are very passionate about them, but unless you have the right data to make those decisions about who's actually using it, you really can't make good decisions. So first, regardless what the data said, I was really excited to at least produce that result. I knew it was going to be very valuable when I saw that the disparity hit closed the team. We talked about it a lot, and we said, look, we really got to make sure that this is a durable, robust result as this paper goes through review process and other things, I'm sure we're going to continue to do that. But when we found that, we said, okay, let's make sure that you know what we're finding is real. Let's do it the best way possible. Let's be transparent about how we did it. But then the idea that this gender gap looks like it's closing was really interesting to us, because a big push from open AI is to democratize AI, and the idea to get more people to use it, and if you get more people to use it, the user base grows. Hopefully, those demographic gaps likeChatGPT, heavily male technology industry, more highly educated people, the early adopters of a lot of tech products. And if you believe that artificial intelligence is super powerful, you want to make sure more people have access to use it. And seeing that happen was really gratifying, and I think it's gonna be the ambition of a lot of people across the world, frankly, to make sure most more people have access to these tools. It's something I work on a lot, but I'm sure there's gonna be a lot of partners. And to be frank, I mean, universities are gonna be key in that as well. Places like Duke and Fuqua as students are getting to use the tools in school, which is really exciting.

Jenny Laurence

And just to clarify, is it also is the benefit of seeing this wide range of use also supporting the reduction of bias in kind of the search terms and the kind of way that the data is being presented through ChatGPT.

Aaron Chatterji 11:33

So the more people who are using it, the more user feedback you can get. We survey our users, and so if your user base is more diverse, all of a sudden you're getting more feedback from a more diverse and representative group of people. So because our population of users looks a lot like their general population now, like, that's a good thing about having, like, a massive platform with like, 700 million, like, weekly active users, it sort of looks like the world. Then you can ask people questions and you get answers that are representative, right? Which is really interesting. How can AI be more useful to you? Well, if you have a platform that's skewed in one direction, you might get answers that actually aren't representative of the whole population. So having the diverse platform, and you know, you see this in some of our data, you know, at least, sort of people with male names and female typically female names, are using the platform a little bit differently. And if you see those things, you can design a b

Jenny Laurence

I know we kind of are talking more about consumers, but just pivoting away slightly for a second as business leaders are preparing their organizations for widespread AI use. What do you think they can take away from this research?

Aaron Chatterji 12:30

I'm really glad you asked, because, I mean, Jenny, you know, this is an MBA student leading the Tech Club. I mean, yes, AI has these amazing applications in our consumer lives, but when you go work at the company, after you graduate, you're going to be using it at work too. And how you use it at work is going to depend not just on the patterns of your use, on your consumer sort of plan, but how your organization has made choices about how they're going to integrate AI into their operations. And I feel like 2024 the year I joined open AI, was the year that every board of directors was coming to their CEO, and usually their CTO after that, and their CIO and saying, what's our AI strategy? And what you're seeing is a rapid deployment of AI across enterprises. But there's lots of different work to be done to figure out how to get return on investment ROI from those enterprise implementations. And so you see, I do a lot of work with companies that are using AI. You see a lot of variation in how they're using it,

Jenny Laurence

Although I will say I'm talking to my peers after their internships this summer. It does seem like across all industries, everyone's been, you know, dipping their toes into working out how to apply AI to their various teams and departments, yeah, which is, you know, phenomenal to kind of learn from.

Aaron Chatterji 15:16

It doesn't surprise me. And I would say Jenny, for the people who are listening, who are in that stage, second first year business students, Fuqua, otherwise, like you all might be the generation that brings this stuff into your organizations. Like, when I was more junior in my career, like, I remember, like, some of the storage solutions, like Dropbox and Box and, like, a lot of that was starting with people using it personally, and then all of a sudden you're bringing it to your CTO or CIO, and be like, hey, like, we could use this, like, inside our company, right? I have a feeling that for a lot of you coming to the organization, you'll be able to do things with AI that maybe your manager is going to be surprised by. And then the question is, does that bottom up approach result in wider adoption in your organization? And I'll also just plug like being someone who's older than the average MBA student in my organization, I learn a lot from the people who are sort of AI native, who've done this, who's used i

Jenny Laurence

Yeah, I love that digital, native generation.

Aaron Chatterji

I mean, totally it's given how young the employee base is at OpenAI, there's a lot of people to learn from.

Jenny Laurence

So final question about productivity, and I guess the business side, has there been any concern from your team about quality over quantity when it comes to boosting productivity?

Aaron Chatterji 16:37

Oh, for sure. I mean, if you think about even just thinking about putting out writing or content inside organization. A lot of white-collar work at labor economists will tell you, and this is what most of our MBA students are going to do. A lot of it requires writing. And so writing memos, writing slack messages, right? And once you start to use AI tools to assist in your writing, and we find a lot of this in our study, writing is like the most popular kind of work-related use case, people are more likely to be critiquing or editing their work rather than just writing something from scratch. But writing is a big part of anybody's job, if they're listening to this as an MBA student right now. And if you are gonna do that kind of work, what is the quality of the writing you gonna produce? The model can produce really cool writing, but if you produce that in such excess quantities that you're not really thinking about the quality, or you start sending many, many more Slack messages that actually might not boost

Jenny Laurence

Well, I think it's also fantastic about ChatGPT is that you know you can speak within your own voice. You know you can write something that's definitely yours, and then with the edit, still comes through your natural voice, which I think is, you know, a fantastic way to make sure that people maintain their kind of unique style.

Aaron Chatterji 18:08

I think so too and like, what's really fun is that when you have that style, you can do things that you never did before. So for example, if you're a person who's more of a poet than a quant, right, you can write like a data driven memo now, with little help from ChatGPT, in your authentic tone. And before, you might have been insecure about that said, Oh, I'm not really that kind of person. I don't know how to present the tables or talk about the data in the right way, but now you can apply that, and that can be really powerful. I think that's, you know, that's one of the upsides of AI use at work that people probably don't think a lot about. They think, I think a lot of the downsides, which is important, but this is going to be lowering barriers to entry for a lot of people who maybe didn't have the skills to write that very data. Driven document data driven document before, or the very sort of data driven person who doesn't have the right language to write the narrative, that's going to help those people unlock a tremendous opportunity that they didn't have before. And that's exciting to me

Jenny Laurence

Yeah, and me. So I'll wrap up with this final question, going back to the consumers. You know, we now have this huge foundation of data from your research, can we expect to see a Spotify wrapped s summary of our usage of ChatGPT in the coming years?

Aaron Chatterji 19:08

See now you're giving me good ideas. I think there's so much interest, and we learned this from this study in terms of how people use ChatGPT. I think doing it at the right interval, like and giving people some like, timely Insights is definitely part of the plan this paper sort of put like, you know, sort of mark in the ground and say, Here's how people are using it today, knowing very well that, you know, in a year or six months, it could be really, really different for the academic audience. I think understanding the asking, doing, expressing is going to help them think about where is AI going, and what has it meant so far. But for folks who are paying attention to this in real time our team, it's incumbent on us and other teams at OpenAI to be releasing timely insights to help people understand where this technology is going. It's both important if you're planning a career in business, it's also just important if you're thinking about the impact of AI on society, and when you think about like, why I joined OpenAI as a chief economist, it is to develop indicators and forecasts and metrics to help people think about the future in a world where AI is really powerful.

Jenny Laurence 20:00

Cool, absolutely. Well, you guys heard it here first. I'm looking forward to seeing that summary, asking, doing, expressing, I definitely don't want to know what I what I'm, what I'm searching for. Ronnie, thank you so much for your time. Really, really appreciate learning more about the insights and looking forward to seeing more of that corporate side as well.

Aaron Chatterji

Jenny, thanks for having me. It's an honor.

Outro 20:26

Duke Fuqua Insights is produced by the Fuqua School of Business at Duke University. You can learn more at fuqua.duke.edu forward slash podcast.