Please enjoy this transcript of my interview with John List (@Econ_4_Everyone), the Kenneth C. Griffin Distinguished Service Professor in Economics at the University of Chicago.
John’s research has led to collaborative work with several different firms, including Lyft, Uber, United Airlines, Virgin Airlines, Humana, Sears, Kmart, Facebook, Google, General Motors, Tinder, Citadel, Walmart, and several nonprofits. For decades, his field experimental research has focused on issues related to the inner workings of markets; the effects of various incentives schemes on market equilibria and allocations; how behavioral economics can augment the standard economic model; early childhood education and interventions; and, most recently, on the gender earnings gap in the gig economy (using evidence from rideshare drivers).
His research includes more than 200 peer-reviewed journal articles and several published books, including the best seller he coauthored with Uri Gneezy, The Why Axis: Hidden Motives and the Undiscovered Economics of Everyday Life, and his new book, The Voltage Effect: How to Make Good Ideas Great and Great Ideas Scale.
John was elected a member of the American Academy of Arts and Sciences in 2011 and a fellow of the Econometric Society in 2015. He received the 2010 Kenneth Galbraith Award, the 2008 Arrow Prize for Senior Economists for his research in behavioral economics in the field, and was the 2012 Yrjö Jahnsson Lecture Prize recipient. He is a current editor of the Journal of Political Economy.
Transcripts may contain a few typos. With many episodes lasting 2+ hours, it can be difficult to catch minor errors. Enjoy!
Listen to the episode on Apple Podcasts, Spotify, Overcast, Podcast Addict, Pocket Casts, Stitcher, Castbox, Google Podcasts, Amazon Music, or on your favorite podcast platform. You can watch the interview on YouTube here.
DUE TO SOME HEADACHES IN THE PAST, PLEASE NOTE LEGAL CONDITIONS:
Tim Ferriss owns the copyright in and to all content in and transcripts of The Tim Ferriss Show podcast, with all rights reserved, as well as his right of publicity.
WHAT YOU’RE WELCOME TO DO: You are welcome to share the below transcript (up to 500 words but not more) in media articles (e.g., The New York Times, LA Times, The Guardian), on your personal website, in a non-commercial article or blog post (e.g., Medium), and/or on a personal social media account for non-commercial purposes, provided that you include attribution to “The Tim Ferriss Show” and link back to the tim.blog/podcast URL. For the sake of clarity, media outlets with advertising models are permitted to use excerpts from the transcript per the above.
WHAT IS NOT ALLOWED: No one is authorized to copy any portion of the podcast content or use Tim Ferriss’ name, image or likeness for any commercial purpose or use, including without limitation inclusion in any books, e-books, book summaries or synopses, or on a commercial website or social media site (e.g., Facebook, Twitter, Instagram, etc.) that offers or promotes your or another’s products or services. For the sake of clarity, media outlets are permitted to use photos of Tim Ferriss from the media room on tim.blog or (obviously) license photos of Tim Ferriss from Getty Images, etc.
Tim Ferriss: John, welcome to the show. It’s nice to see you, sir.
John List: Thanks so much for having me, Tim.
Tim Ferriss: Now, I find your story endlessly fascinating, which means, of course, I have an embarrassment of riches in terms of materials and questions. We’re never going to get through all of it, but I was struggling with finding a place to begin. There’s so many options, but I thought I would just mention for folks, you have eight kids, is that right?
John List: That’s correct.
Tim Ferriss: Eight kids.
John List: Eight kids.
Tim Ferriss: Your grandpa, dad, and brother, all truckers, if I’m not mistaken.
John List: That’s correct. Proud truckers.
Tim Ferriss: Proud truckers. I’m just adding a little bit of color here and there, because certainly, we’re going to get into many different nooks and crannies. I thought I would just start with asking you to describe what a clawback incentive is and how you have used clawback incentives with your kids. And so incentives in general, what other incentives you have used and then we’ll use that as a jumping off point.
John List: The clawback incentive is an incentive scheme that actually reverses the way we think about traditional incentives. So in incentives, I’m the Chief Economist at Lyft. The way I incent my workers there is I have them work really hard all year. And then at the end of the year, they receive a bonus if they do a good job, that’s great. Now, what the clawback incentive is, is at the beginning of the year, I give them the incentive, but then I tell them if they don’t perform, I will take away the incentive. And the idea behind the clawback is I want to, first of all, show them the money. And then secondly, I want to invoke loss aversion.
Loss aversion is something that psychologists have taught us, goes as follows. If you own something, you really, really, really are hurt if you have to give it up. So the idea here is give somebody some money. That will induce them to work harder. And when they do work harder, everyone’s better off because they get to keep the money. And my firm is better off because they work harder. Now, where I’ve used that, I’ve used that in Chinese manufacturing plants. I started a pre-k school about 12 years ago. I use that with the teachers. So at the beginning of the year in September, beginning of the school year, some of them get the bonus. And then I tell them, “If your kids don’t perform at the end of the school year, I will take some of it back.” And lo and behold, what happens? They work harder and the kids learn more.
Now, for my own kids, I’ve also done the same thing. Any good scientist will use his or her own kids as subjects. So I’m no different than that. As you mentioned, I have eight kids. Two of those kids happen to be twins. So I have the perfect treatment and control group. Now, some things like my oldest daughter had a very hard time going potty on the toilet. So what do I do? I give her an incentive, let’s say, a doll or something else. And I tell her, “If you go potty on the toilet, you get to keep it. If you don’t, the doll is going to go and sit in a different room. And you’re not going to be able to play with the doll.” That’s the clawback incentive.
Tim Ferriss: And what were the results?
John List: The results are just remarkable. Whether it’s school teachers, whether it’s workers in a manufacturing plant, whether it’s students, whether it’s my own kids, just across the board, what happens is if you use the clawback, people work harder. And everyone’s better off in the end, because they’re more likely to get the reward, and you’re happier because they’ve worked harder. So you’re happy to give them the incentive.
Tim Ferriss: So hearing of your kids, and the twins in particular, I have to imagine an unfair share of the experimental load lands on the twins. But you could disabuse me of that notion if I’m missing. But could you describe other ways that you have used incentives with your kids? Any stories or examples that come to mind?
John List: No, absolutely. So first of all, you’re right that whenever I need a really good control, that will fall on the twins. But to be fair, I have enough experiments. I think about using experimentation to incent inputs. For example, when the kids are in third or fourth and fifth grade, I tend not to incentivize outputs, because outputs are something that a person has a hard time controlling. But what they can control is the set of inputs. What I mean by inputs is how many books you read or how many hours you spend studying. Those are things that the child, or the worker, or whomever can control.
Things like how well you do on the exam. A lot of exams, for example, are graded on a curve. So you can’t control how well other students do in the classroom. So it’s not really fair to give you an incentive based on that kind of output. So look, my incentives role from in the classroom, to on the baseball field, to on the soccer field. But all the time, it’s using input incentives with my kids. Now, with my workers, whether it’s at Lyft, or I used to be the department chair here at the University of Chicago’s econ department. There, I can’t really observe inputs. So I have to reward outputs. And in the end, it’s the organization is affected by the outputs, not the inputs.
Tim Ferriss: Could you give an example, and don’t worry, folks listening. I’m going to expand beyond the family system. But I like to start with something that many listeners will identify with and then we’ll move in all sorts of directions. Are there any specific examples that you could give of incentivizing and changing behavior or molding behavior with your kids?
John List: So let’s think about studying. When I think about a clawback, at the beginning of the week, I will tell them, “If you work intensively for an hour each night, Monday night, Tuesday night, Wednesday night, Thursday night, Friday night, you can keep this incentive that I will give you on Monday morning.” Now, that incentive can be a pack of baseball cards for my boys. It could be a pass to the movies for my daughters, but the key is give them the incentive early in the week. And then they will perform all week. For example, an hour of study in math every night is something that happens all the time in my household. Then at the end of the week, they end up keeping the incentive because nearly every time, they meet the goal.
Tim Ferriss: So I have a question about the clawback incentive and harnessing the loss aversion. And part of why I was looking forward to chatting with you is that I think it’s minimizing it to say sort of, along the lines of, say, Freakonomics. And I know you’ve had a lot of involvement with professor Levitt and have been featured regularly on the Freakonomics Blog and elsewhere. There are many assumptions that we make or beliefs that we hold, which turn out not to be terribly supported by data when you actually run experiments. And as an example, when I was doing prep for this conversation, I read about, and I might get the specifics slightly off, but your examination of matching donations with charitable organizations. And asking them if matching works, the answer is, of course, yes.
Do you have data to support it? No, not really. But looking at, say, one-for-one matching. You donate $1, we match $1. Or $1 to $2, $1 to $3, and it turns out matching does work, but the ratio is less important than just the matching mechanism itself, it would seem. And we can certainly explore that more. And if I screw it up, please let me know. On the harnessing of loss aversion, as an employer myself, or as someone who’s hoping to have kids soon, I’m probably not going to catch up with you unless I adopt a lot.
But I imagine in my mind that workers will respond well when they’re given a bonus at the end of their performance review period, whatever that is. But that over time, maybe in the long term, and not in the short term, if people are given something that is taken away, that that might breed resentment or some type of learned helplessness where they’ll just not attempt to put forth their best effort. Is that grounded in any of the data in any way, or are there any, I guess, what I’m asking is, side effects?
John List: No, absolutely. That’s a great question. When you think about using behavioral nudges or you think about using behavioral incentives, one very important issue that you always need to recognize and keep in mind is, will it keep working over and over and over again? And then secondly, does it have any side effects? So in the case of the clawback, you could imagine that you run it one time and it works. And then when you try it a second and a third time, it depreciates. It really doesn’t work that well. Or you could imagine that you hire a bunch of school teachers like I did down in Chicago Heights and give them the clawback. And within three months, they all quit because they hate it. It’s just too stressful and it’s too burdensome on them. Now those are two very important considerations for any incentive that we talk about.
What we’ve found so far is in terms of the depreciation it does depreciate a little bit like most behavioral nudges, but not a ton. So on that side, you don’t have to keep creating a new mousetrap, that the real mousetrap and the first one, the clawback, works reasonably well, and it has over time for a long time. Now, the side effect one is also sort of interesting because what you find is that many workers actually like the clawback because they view it as a commitment device. And what I mean by that is most things in life are, you have to exert effort today and you receive benefits in the future. So think about climate change. Think about your own health. You don’t really want to go to the doctor. You don’t really want to go to the dentist because it’s a cost now and the benefits that are in the future. Think about why people drop out of school. As a 16-year-old, there’s a lot of cost now. And the benefits aren’t for a long time.
As humans, we do a really bad job at those types of issues. And that’s why we have problems like climate change, and you don’t take care of your health, and too many people drop out. What’s nice about the clawback is it takes that problem and turns it around because it takes the benefits from the end and brings them to the beginning. So you get the good stuff right away. And that serves as a commitment device for many people. So some people in our experiments will actually pay real money to have the clawback, to get to use the clawback. In the end, there are some, of course, that hate it. But there are enough that love it, that on that dimension, it actually turns out to be a doubly good incentive. Because you not only are better as the owner, but the workers like it more too.
Tim Ferriss: Question about the Chinese manufacturing, speaking to the durability of the incentive. And, again, I’m getting probably the specifics off and you can correct me, but it seemed that it worked on the order of something like six months, not on the order of weeks, at least in the example that I read. And maybe that was just the sampling interval, so maybe it lasted for years for all I know. But my question is what was the reward period? Does that make sense?
John List: No, it makes perfect sense. So what we did in the Chinese manufacturing plant is we had weekly reward periods. So every Monday morning, they were given the bonus and then that cleared on Friday afternoon. So our experiment lasted six months where we would move people in and out of clawback incentives. We’d put some people in the standard incentive. The standard incentive, of course, is work all week, and at the end of the week, you get your bonus. And we were always comparing whether the clawback does better than the standard.
Tim Ferriss: And could you speak to inputs versus outputs and what your metrics were?
John List: Yeah, absolutely. So in that example, all of our metrics were outputs. So the output in China was the number of non-defective items that came off the manufacturing belt. And each team had a goal. And that goal was, if it was met, you got to keep your bonus. If the goal wasn’t met, you had to give back your bonus. So in the manufacturing plant is in many of our experiments that involve workers. We are looking at outputs. Now, when I talk about my kids, I want to incentivize an input, but it’s really hard to do that in the workplace.
Tim Ferriss: Yeah. That makes sense. That makes a lot of sense. I mentioned donations earlier. And I’d love to segue to tipping. And I know you’ve spent more time than most looking at tipping.
John List: Do you want me to tell you the vignette about donations and then how I can pivot that into tipping?
Tim Ferriss: That’s perfect. I would love that.
John List: Perfect. Okay. My research agenda in charitable giving actually started back in the late ’90s. And a lot of people ask me, “Why did you get involved in working on the science of charitable giving?” And a lot of times, they assume it’s because of my altruism. And I tell them, “That’s not the case.” What happened was I was an assistant professor at the University of Central Florida in 1998. And assistant professor means you’re the lowest on the pole. Nobody gives a damn about you. You’re just trying to do the best you can. So I’m sitting in Orlando, Florida in my office, minding my own business, and I hear a knock on the door. I open my office door and there’s the Dean of the College of Business standing in my doorway.
So I say, “Dean,” his name was Thomas Keon. “Dean, Tom, how can I help you?” He says, “John, I want to start a Center for Environmental Policy Analysis. And I want you to help me start that center. And your first job is going to be to raise money.” So I’m like, “What? I’ve never raised any money before. I’m a truck driver’s son. What’s going on here?” So he said, “Think about it and come back to me.” So I thought about it for a few days and I went to the Dean’s office, and I told him, “This is a great opportunity and I will do it if you satisfy two conditions. One condition is I want to run the fundraiser as a field experiment because I want to learn about why people give, what keeps them committed to the cause. And two, I need some up front money.”
And that’s what I learned in those two days is I went out and talked to a bunch of fundraising specialists. And they told me, “The key is to have up front money.” So the Dean gave me $5,000. Big whoop, right? But importantly, he gave me a big list of potential donors who I could ask to give money to the Center for Environmental Policy Analysis. Okay. Now I’m in business. My research agenda has always been to use the world as my lab. And if you’re ever trying to accomplish something, do it scientifically. So you can not only figure out what works and why, but then you can get it right the next time.
So I did this fundraiser. And what I first learned was I talked to dozens of fundraising gurus, practitioners, strategists, et cetera, et cetera, et cetera. And they all had rules of thumb. They all said things like, “Use a three-to-one match rather than a one-to-one match, because a three-to-one match will raise more money.” When I say three-to-one, I mean for every $100 somebody gives, the charity matches it with $300. That’s a three-to-one match versus a one-to-one. All these great anecdotes. And each time, I would ask them, “Can you give me the data behind these gut feelings?” “No, I can’t.” And I said, “Well, how do you know they’re true?” And every person said, “Well, that’s just how we always do it. That’s how my boss did it.” Now, I’m thinking, “Holy cow, this is two percent to three percent of the GDP of our nation is charitable giving.”
Two percent or three percent of wallet is giving of money. Forget about the time. Add volunteerism to that and you might get seven percent of GDP. So it’s a very important sector. But when I learned that it’s mostly built on anecdotes, I said, “Man, we’re in business.” Because now, I can use this as a research agenda to explore generosity. And I can help charities use science to raise more money. So I’ve been at that, for now, close to 25 years. Partnering with charities, helping them raise more money, helping them leverage science, and you end up learning some really interesting facts about people.
For example, when you look at what drives a man to give versus a woman, a man is driven a lot by the tax advantages that charitable giving gives you. That’s called the price effect in giving. Whereas women are driven more by altruism and social pressure. So now, these become two sort of important features that you might say, “Well, that affects charitable giving, and that’s about generosity, but what about other parts of society like tipping?” Can we take what we learn about the generosity to the Sierra Club or the United Way, or what have you, and apply those elements to what happens in the world of tipping somebody, whether it’s an Uber driver or tipping a waiter, or what have you?
And it’s sort of interesting because sometimes you can and sometimes you can’t. One example is when you look at the charitable giving data, so I have the IRS data. So I have everyone’s tax form from like 2008 to 2017. So I can explore all kinds of different cuts of those data. And I can tell you that as it turns out, at each income percentile, so if you’d say the average income earner in the United States is like $58,000. If you look at all people who make that amount of money, women give a lot more than men in that income bucket. So you look across all of the income buckets, and the fact is that women in every income bucket give more than men to charitable causes.
So now you’d say, “Okay, that must be also true in tipping.” It’s not. In fact, it’s just the opposite in tipping. So you can say, “Well, where’s John getting his tipping data from?” I was the Chief Economist at Uber back in 2017. And TK, Travis Kalanick, came to me and said, “John, you need to help us keep the drivers in Uber because there’s this crazy “Delete Uber” campaign, #DeleteUber campaign, that’s killing us on both the driver and the rider side.” And my solution back then was, let’s roll out tipping on the app.
We rolled out tipping on the app. My team at Uber was called Ubernomics. So team Ubernomics helped roll out tipping. And what we find there is, across the board, men tip a lot more than women. So there’s sort of this interesting dynamic that in charitable giving, where you do it face-to-face, or you do it to be recognized, women are giving a lot more than men. But remember, on the Uber app, what happens is you do it anonymously and you don’t do it face-to-face. And in that case, men actually give more than women in tips.
Tim Ferriss: What are some of the learnings around increasing frequency or amount of tipping from your many experiments?
John List: There are some interesting facts about what happens in that Uber car. A first fact is that if your listeners want to join the one percent, a lot of times people say, “I want to be in the one percent.” Now, in this case, only one percent of riders tip every trip. That kind of blew my mind.
Tim Ferriss: That’s wild. I never would’ve guessed that.
John List: Yeah, because in the charitable giving game, like nine out of 10 people give to a charitable cause at least once every year. But when you think about the Uber experience, only one percent of people tip every time. And interestingly, roughly 60 percent never tip.
Tim Ferriss: Wow.
John List: Now, in the middle, of course, are the sometimes tippers. So we did this big data, just this big data grab from our experiments. So now, we’re talking about 23 million observations, a big data set. And we can look at a lot of averages. And what you find are things like men tip female drivers more than they tip male drivers. That’s a fact, especially when the female drivers are younger. So what happens is the older female drivers, say 60 years old and beyond, those female drivers are tipped the same as the male drivers in that age group. But the big difference is with the 21 to 25-year-old drivers, where females earn about six percent more in tips than the males do. And that it’s all driven by male riders, which is sort of an interesting fact. Now, I’m going to let your imagination, Tim, go with that about why that happens. I don’t want to go into that here. I’ll let your imagination go. But there are all these facts like this —
Tim Ferriss: I don’t have to go imagine very long. Continue.
John List: But there are all these kinds of facts like this. The average trip, only about 15 percent of trips are tipped. And conditional on tipping, the average tip is about 10-12 percent of the fare. So you do have things like a bigger bill is a bigger tip. That’s what we have in restaurants too, where people tip 20 percent. But the big thing about tipping is people tip a lot more if it’s face-to-face versus when it’s anonymous. So the way we did tipping at Uber was you don’t get asked to tip until after the ride is done and you’re removed in space and time from the driver. And we did that on purpose because TK said, “John, I don’t want tipping to be a tax on riders. I want it to be something that is truly done for exceptional service. We want to separate it in space and time.” And then that’s what we got is roughly 15 percent of trips are tipped.
Tim Ferriss: Now, if I think to my restaurant experience, so I grew up working in restaurants. And the checks have changed over time, if we’re talking about computer-generated bills / checks. And one of the changes, I don’t know when this started, and I don’t know if it’s supported by data, I would hope so. It certainly has affected how I tip, but it has been checkboxes where it’ll have, rather than making you do the math, it will say 15 percent, 20 percent, 25 percent. There’s a checkbox and there’s the amount and then there’s the total. And that raises the question for me, of what types of user interface or flow experiments worked well, if you’re able to speak to it.
And I think you are, because I remember reading about you getting a call from Jeff Bezos and one of your conditions for possibly engaging with Amazon being the ability to publish your findings. Just to give a snapshot of that, you didn’t take the job because there were going to be trade secrets, they wouldn’t be publishable. However, you helped introduce someone who was hired and I think still works there and he sends you a, what is it? A Christmas card every year with his net worth at the bottom, something like that, which is outstanding. I hope he sends you some chocolates too.
John List: He better send me a bag of cash, the way Amazon stock has been doing. Yeah. When they gave me that offer, the stock was $7 a share.
Tim Ferriss: Oh, oh. So TK was open, as I understand it, to you publishing. So could you share some of maybe design, experiments, or changes or tweaks that turned out to be effective? It doesn’t have to be Uber-specific, but it certainly could be.
John List: Early on, Amazon, Jeff Bezos came to me and asked me to be their Chief Economist and oh, boy, oh, boy, oh, boy. Whenever I go back and look at that contract, it was a contract that was heavily laden with options and Tim, Tim, Tim. So if I could go back, I still think, I can honestly say this, I would do the same thing today. I did give up millions of dollars, no doubt about that. But in the end, I’m a scientist and I’ve always thought that my calling was that to use the world as your lab is a great opportunity to make the world a better place. And I really felt I could help Amazon. I really did. But in the end, they just couldn’t get over the hurdle of I needed to publish in academic journals as a scientist, because I not only wanted to solve Amazon.com’s problems, I wanted to solve the world’s problems.
And I certainly get it, that their basic premise was look, we want you to come in and help us make a ton of money. And at the time he was talking about something called the cloud, which I didn’t even know what that was. In fact, he goes, “John, the two next big things are going to be the cloud and the economics team and you’re going to be the guy who hires all the economists.” And he says, “Let me tell you about this cloud thing.” He goes, “You’re going to help me price the cloud.” Like, I didn’t even know what that is and I just pretended I did. I honestly didn’t know anything about it. So, okay, so TK calls, Uber calls and Uber says, “We understand what happened at amazon.com,” because of Jeff Holden. Jeff Holden was at amazon.com back then and then he ended up leaving for Groupon. He was at Groupon for a while, and then he went to Uber.
And when TK wanted a Chief Economist, he ended up talking to his exec team, Jeff Holden’s one of them, and Jeff said, “There’s this guy who we really liked at Amazon and he wouldn’t come because of this restriction or constraint that he said, he’s an academic and he wants to publish some of the work.” And TK said, “Well if he’s good, let’s bring him in and he can publish some of his work.” And to his credit, TK stayed true to their promise. So not only around tipping, we have two academic papers about the economics of tipping, questions like: how should you do the presets? At the bottom of your app, you have zero, $1 or $2, or should it be one, two and four, or should it be 0 percent, five percent, 20 percent?
We have all those configurations and the bottom line is you want that. You want there to be a choice. You don’t want to have people fill in the number. You want there be a box that you can check, but make sure the boxes aren’t too high, because if they’re too high, you’re going to get a zero, but also make sure they’re —
Tim Ferriss: Too high in dollar amount.
John List: Too high is $50, 100, 150.
Tim Ferriss: No.
John List: Or 25 percent, 45 percent, 70 percent. A lot of times people will say, “Those are out of my range, so I’m going to do zero.” So you don’t want to do too high. You don’t want to do too low because then you might be leaving money on the table for the driver. So it becomes an art and it’s an art that you can only figure out with field experiments. And a lot of those five percent, 10 percent, 20 percent, those are individual-specific, so that doesn’t fit with just everyone. Sometimes the low, it means high for somebody else or a really high set of presets is low for somebody else. So just like the price in the various other pieces of the algorithm, an optimal tipping scheme will take account of the consumer’s traits and how they’ve tipped in the past. So another line of research that we worked on at Uber was what I call the economics of apologies. And as you know, it was pretty rocky at Uber when I was there. In fact, I think there’s a new show coming out called Super Pumped or something.
Tim Ferriss: Yep. Yep. So not only was I one of the earliest advisors to Uber starting in 2009, 2010. So I was along for that ride and watching things very, very closely, pun intended, but I actually also am friends with the, I don’t know if he’d be considered the creator, but he’s certainly producing and writing, Brian Koppelman and David Levien, Super Pumped, or adapting it for screen. I’m not involved with that at all, but he’s a good guy. So yes, based on the book. So there’s never a dull moment in ridesharing.
John List: Yeah. Can you tell them that when the thing comes out, that I want Matt Damon to play the Chief Economist? Is that too much to ask, Tim?
Tim Ferriss: Well, you know what? They have worked with Matt Damon before, so I will put in the request.
John List: All right. All right.
John List: So let me tell you this story, quickly, about apologies. Is that okay, Tim? Do you want to hear that?
Tim Ferriss: Yes, please.
John List: So this is a perfect example of, I think, the leadership of Uber and how they were open to science. So I’m scheduled to give a keynote talk at our big annual meetings. They’re called the American Economic Association meetings. So these are annual meetings that take place every January. Okay, great. So I’m on the schedule to give a keynote talk alongside a guy who ends up winning the Nobel Prize in Development Economics named Angus Deaton. He’s there to criticize field experiments, and I’m there to be the supporter of field experiments. So I get picked up at my house here in Hyde Park, Chicago, because the meetings are downtown Chicago. I just happened to get lucky that year. The meetings rotate to a different big city every year. So the Uber car picks me up and as usual, I’m still working on my slides, but I look at my watch and it’s roughly noon and I’m supposed to be on stage at 12:30.
So the driver starts taking me down there and we get about three-quarters of the way down, I look up, look at my watch, and I’m like, this is perfect. I have to just finish this last slide. I’m going to walk in, these slides are going to be great. Next time I look up, I’m in the front of my house, back in Hyde Park.
Tim Ferriss: Oh, no. Oh, no.
John List: And I’m like, “No. What the blank is going on here?” And the driver said, “Well, the app blinked and Professor, you were so busy working, I didn’t want to disturb you. I thought that you changed the destination because you forgot something at home.” I said, “No, take me back down to the conference.” So, okay. So I get there, I run in. Everything turns out fine. But that evening, when I got home, I ended up walking out to our garage that’s separated from my home and my wife always knows when I go out to the garage, it means somebody’s going to have Hell to be paid. So I call TK and I say, “Hey TK, let me tell you about what I think of your blankety-blank app. And guess what, TK? I will never take an Uber again. I am just going to simply take a Lyft.” I said, “TK, you can call your Chief Economist a Lyft loyalist.”
Tim Ferriss: Oh, wow. He must have loved that.
John List: Yeah. So I said, “You know what, TK? The worst part about the story is that I never received an apology.” And he says, “Well, John, we haven’t gotten to that yet.” And I said, “We have now.” So what my team did is we explored in big, big data sets how much business does Uber lose from bad trips. Now here you don’t want to do an experiment because I don’t want to just give people bad trips on purpose.
What I have to do is I need to find, for example, Tim, and then I need to find Tim’s statistical twin in the data. And I need to find when Tim gets a bad trip, but at the exact same time and over the same route when his statistical twin gets a good trip. And then I can look at, in the future, say for the next 90 days, does Tim, who got a bad trip, spend less money than Tim’s statistical twin? What happens is you do. In fact, millions and millions and millions of lost revenue happened because of bad trips. So now I showed that bad trips cost Uber hundreds of millions of dollars. Now what are you going to do about it? That’s where the science comes in. So we ended up developing a scheme where we did various apologies within an hour of when a person had a bad trip. One apology could be simply a basic apology in an email, “Tim, we’re sorry about your bad trip. We know we should be doing better. Please accept our apology.” That’s one treatment.
But another treatment is the exact same apology with a $5 coupon that, “Please take this $5 coupon for your next trip.” What we find is we can undo a lot of the bad stuff of the bad trips, but nearly every time you need that cash coupon. Words just don’t matter, but a little bit of cash with the apology ends up undoing about a third of the bad stuff. So now we’re talking tens of millions of dollars and true to his word, TK could have said, “Look, that’s a great trade secret. We don’t want anyone else, including Lyft, to know about that.” But TK let us write that up as a scientific paper, and now it’s in an academic journal and we can all learn from it about the economics of apologies.
Tim Ferriss: I have to give TK credit because you think about the, and you obviously, but as the fierce competitor he is, it’s all the more impressive to me that he would’ve allowed that. And you think of the potential ripple effects of that, not just within ridesharing, but outside of ridesharing. I mean, it hopefully translates to a better customer experience in different sectors.
John List: Absolutely.
Tim Ferriss: That’s part of the potential consequence of being able to publish.
John List: No, you’re right. And I think that TK really got it, and Logan Green does too and John Zimmer now at Lyft, we have great papers coming out of there too. But they totally get that we’re not only here to change the face of transportation, but we also have another gem called big data and a lot of information that we can use and we can leverage to help some of societal’s problems be attenuated. And that’s the great thing, I think, about firms and corporations and organizations today when they’re willing to do that, even though it’s not in their financial best interest to do so.
Tim Ferriss: Yeah. I mean, it’s an incredible opportunity. I remember way back in the day, they ended up, I think, cutting this short, but when there existed the God View and the ability to look at all this data and you could really learn a lot about human nature and they would publish on your blog and elsewhere, all sorts of correlations that were endlessly fascinating. But when you have a big enough data set and you’re dealing with individuals and you’re able to identify not only their behaviors, but how their behaviors change with different incentives and UIs and so on, I mean, you’re really just looking at this gigantic Petri dish called humanity and human nature at the end of the day.
John List: You’re spot on.
Tim Ferriss: Let me ask you, if I can find this quickly enough looking through my many notes here, about what you have learned about, if that’s the right way to phrase it, human nature. And I want to bring up car dealerships. So I believe, and please correct me if I’m getting this wrong, but “The Behavioralist Meets the Market: Measuring Social Preferences and Reputation Effects in Actual Transactions.” This is in The Journal of Political Economy, 2006. But the quote that I really liked, and this tells you I’m kind of a Hobbesian at heart, I guess, “Very few people will not screw each other,” List says. “There are very few nice people out there.” Could you just elaborate on that and flesh out how that came about?
John List: No, absolutely. So the paper you bring up is actually near and dear to my heart because one reason why I got involved in economics to start was back in the day, in the late ’80s, I was a baseball card dealer. So what that meant was I went to these large conventions to buy, sell, and trade things like Hank Aaron, Barry Bonds, or Ken Griffey, Jr, whatever baseball cards, and what I noticed was everything that was happening in that convention was not always in accord with economic theory. And I thought that what I was learning in the classroom could actually be tested at these baseball card conventions. So I started to do it in the early ’90s. That’s one of the very early work in economics that uses the world as the lab and it’s exploring things like, why do people trade?
Why don’t people trade? Why do people negotiate the way they do? Why do dealers give women higher price offers than men? Et cetera, et cetera, et cetera, discrimination, gender pay gap, et cetera, et cetera. So along comes the early 2000s and what I notice in economics is that there is an explosion of research on something called social preferences. And what that means is we want to measure how much people care for other people. So clearly they do. Clearly giving to charitable causes is two and a half percent of GDP in America. So clearly people are giving to others. But the types of estimates that these studies were showing is that you’ll give up half of your wealth to some anonymous stranger. And I started to wonder, how far can you push these lab results, if you go to a market where people are buying, selling, and trading for their livelihood?
So what I did was I set up an experiment where everyone in the experiment knew that they were taking part in an experiment. Guess what happened? They all were really nice. And they all were exactly like the literature in economics that they should be. They’re all Homo be nice to everyone rather than Homo economicus. So then what I did is I observed those same people in the marketplace when they didn’t know they were being observed, doing the exact same thing that I had them doing, the experiment. Guess what they do now? Totally screw the other guy over and they’re about maximizing their own incomes. So what happened to all this generosity and all of this social preference, I’m being nice to other people, the market, and the competition, and the anonymity that I’m doing this and if nobody can detect me, I’m going to actually do it.
So then I added a twist to that paper. What I did is I had them act in the marketplace and they did not know that they’re part of an experiment, but now what they sell can be graded by a professional certifier. In there, you find that they start to be nice again. So what’s happening is that they care about their reputations and whenever you can check them on their reputation, they’re nice. Why? Because they care about the long run and they want repeat business. But when you can’t check them on their reputation or you can’t check the quality of the goods they sell, they outright screw people over.
Tim Ferriss: This makes me think about one of my sensitive spots with respect to Uber and ridesharing in general, which was how, and I’m not sure of the right kind of bias label to apply to this, but you would see in the news Uber driver this, Uber driver that, assorted bad behavior, which statistically was a very small percentage. But the implication was that somehow in the early days, especially, that ridesharing was more dangerous or lower quality than taxi rides. But the fact of the matter was there was no data capture with taxi rides. There was no accountability, no rating, no nothing, but the costs and the events, the adverse events, were largely invisible. So it was just a very unfair comparison.
Let’s hop from car dealerships, going to a place that is much more meta because I admire your ability and anyone’s ability to test assumptions, even if they lead you to some very unexpected, surprising, or even uncomfortable conclusions. And I have, as you can see, and people who are on video, I’ve just piles and piles of paper in front of me.
John List: You are a scholar.
Tim Ferriss: Well, I play one on the internet, and let me read a line from you, and I would love for you to explain what it means. Here we go, just a few sentences. “A first step to rise up my critical thinking hierarchy is to practice theory of mind. People have a really hard time doing this, but once you do, you avoid all kinds of silly mistakes at home, in the workplace, and on the streets.” So there are two terms that I would love for you to define, and then you can expand it in any way that makes sense: critical thinking hierarchy, and then theory of mind, practicing theory of mind.John List: Critical thinking is a bit like creativity. You know it when you see it. It’s very hard to define what critical thinking means. Now, in my work, I’ve written a recent academic paper on critical thinking. I view it sort of in two camps.
One camp is how you use data and how you gather data to make logical decisions. A lot of times, people make fundamental errors when generating data. For example, if I really think Uber is a great idea early on, every data point that I get back that says it’s a bad idea, I put on the sidelines and every data point that comes to me that it’s a great idea, I say that data is right. That’s called confirmation bias. And it’s one of the most important impediments to figuring out if your idea is good or not. So we have biases in how we generate, accumulate, and interpret data. That’s one kind of critical thinking. The other kind of critical thinking is more abstract and this is where theory of mind comes in. So theory of mind means how well do you put yourself in the shoes of another person? So there’s a theory in economics called game theory. And game theory was started by a really old mathematician who’s now unfortunately dead, named John Nash.
Tim Ferriss: John Nash. Yeah.
John List: John Nash. That’s the name of my dog.
Tim Ferriss: The bar scene in A Beautiful Mind.
John List: Exactly. One of my favorite movies, A Beautiful Mind. Even though the bar scene got it wrong.
Tim Ferriss: Oh, really?
John List: Yeah. They messed up the Nash equilibrium. The reason why is because at least one of them should have gone for the blonde.
Tim Ferriss: Oh.
John List: So an equilibrium is, given everyone else’s choices, would I change my choice? And if you wouldn’t change your choice, that’s an equilibrium. If you would change your choice, then that’s not an equilibrium. Remember what happened in the bar scene? They all went for the brunettes, even though they all wanted the blonde. So that was a definition of not an equilibrium, but anyway, we’ll cut Ron Howard a break.
Tim Ferriss: Yeah. Don’t let facts get in the way of a good story now.
John List: Exactly. How could you write that bar scene at the Palmer House? So now theory of mind ends up being super important for the abstract part of critical thinking because your next move needs to be, in part, determined by how you think that person will react. And if you can put yourself in the shoes, whether it’s do I lower my price on Uber and how would Lyft respond? Or should I spend more money in ads? If I’m United Airlines, how will Southwest respond? Et cetera, et cetera, et cetera. All of these kinds of examples. Now theory of mind is how well do you put yourself in the shoes of another person and figure out then what your optimal action should be?
Now you see Tim, when you start having kids, you’re going to have to use theory of mind to say, “Okay, how am I going to get my kid to pay attention on the soccer field?” or “How am I going to get my kid to pay attention in second-grade math, if they’re really not interested at all, because they’re doing math at a seventh-grade level?” And you need to put yourself in the shoes of whether it’s your student, your worker, your collaborator, your adversary, whomever, and you need to then backward induct and act in an optimal way. That’s theory of mind.
Tim Ferriss: I have, not surprisingly, many questions about this. So the next question that leaps to mind for me, assuming that theory of mind is a valuable skill with tangible benefits, how do you develop it? And I suppose if we wanted to speak broadly, a lot of folks would say, “Well, it sounds like you’re practicing empathy.” But then on perhaps the other side of the spectrum on the more concrete side of things, I would say, “Well, if I just gave you a thousand bucks and made you play online poker and set a max amount that you could lose per game to ensure a certain number of repetitions, you would probably get better at predicting how others, or at least get better at attempting to predict how others will respond, giving certain gambits and strategies and so on.” How would you suggest people develop this skill? Or are there different approaches that you might suggest people take or books or resources, anything, for people who want to get better?
John List: Absolutely. So what I would tell the listener is to start with what an economist calls the primitives of decision making. And what I mean by that is try to figure out what are the preferences of that other person, what are the beliefs of the other person, and what are the constraints of the other person? Because all choice will be governed by these three primitives. So for example, if I know that somebody adheres to the social norm of never, ever tipping, then if I’m trying to do something to get them to tip, if I put myself in their shoes, regardless of what I do, they’re never going to tip. So I should not spend a lot of money trying to convince them or induce them to tip, because what is governing their tipping behavior is the strong social norm that they will never tip. So that’s called a constraint in a person’s world.
So I think the first step is to try to figure out what are the primitives of the person’s decision making. And after you figure that out, you then say, “Well, what’s my optimal action given that the person operates in this way?” A lot of times it would be how much money do they have? Do they have the mental capacity to actually solve the problem that I’m giving them when I play poker? I think poker’s a hard game because it’s really hard to randomize. So if you’re playing against really good players, you need to, first of all, decide what is the fraction of times that you’re going to randomize?
When I say randomize, I mean, what fraction of hands are you going to bluff? And then if it’s, let’s say, 10 percent of the time, you then need to figure out which 10 percent of the time. Now, humans are terrible at that. Humans are terrible at doing something in a random way. So what I do when I play poker, I use the second hand in a clock to say, “Well, if I’m going to bluff, say, 10 percent of the time, then I look at the second hand and say, ‘If it’s at a certain point in the clock, I’ll bluff. And if it’s not, I won’t.'” And then that helps me because it’s an exogenous machine that is not susceptible to my own biases.
Because when you ask somebody to, for example, create a random sequence of coin flips, you can ask a really smart person to say, “Create 10 coin flips in a row.” And what they’ll do is they’ll say, “Heads, heads, tails, tails, heads.” They’ll have nearly exactly five heads and five tails over the 10, and they won’t have enough runs. Because sometimes you tail, tail, tail, tail. But when a human mind creates it, they don’t do enough runs and they do too close to 50/50. That’s just how we’re built. So you need an exogenous force to help you with things like that in poker and anything else.
Tim Ferriss: Do you play poker often? Have you played for a long time?
John List: During the rage I played and I ended up —
Tim Ferriss: Did you say the rage?
John List: Yeah. The rage to me was —
Tim Ferriss: Your rage or just a — okay.
John List: Yeah. You had this rage of the explosion of —
Tim Ferriss: The World Series of Poker on television and all that.
John List: Yeah, the World Series was starting. Phil Hellmuth was on, Ivey, these guys. And I ended up flying to Australia to give a keynote in a place called Alice Springs. It’s way out in the boondocks of Australia. I don’t know if you’ve ever been there.
Tim Ferriss: I’ve never been to Alice Springs. I’ve been to Australia, don’t recognize Alice Springs.
John List: So I’m out there and there’s a casino there. And it so happens that the conference is at the casino. So as the keynote speaker on, I think it was a Friday night, they were going to take me out to dinner. Wine and dine me, that’s typically what you do with the keynote speaker. So around 1:00 that afternoon, I said, “Well, the dinner doesn’t start till 8:00. I’m going to join this poker tournament.” Thinking I’m going to be out, I won’t make the next table, et cetera. It turns out that I get on this magnificent run and I make the final table. And by this time, it’s 8:00, so the conference organizers are standing around saying, “Come on, John. We have to go.” I end up making the final two. It was an Aussie woman and me. It was kind of an out-of-body experience, but I won the bracelet.
And it wasn’t that I was very good, but I did a little theory of mind. I’m sure I got lucky a few times on the flop. Maybe on the river, got lucky once or twice. But I ended up kind of retiring there. A little bit like Michael Jordan, let’s retire on top. So I got my Australian bracelet and I haven’t really played that much since. I just don’t have time to play. But it’s an interesting game that I think people undervalue skill. Because I think poker’s a lot more skill. Obviously, there’s a ton of luck, I get that. But there’s a lot more skill than people realize. It’s really a great game. Especially if you have a long-time series of tournaments, the best end up winning. And I certainly wasn’t the best, so I kind of retired after that bracelet.
Tim Ferriss: I am fascinated by poker. I’ve actually interviewed Phil on the podcast. He’s hilarious. Very smart, very aggressive guy. And I recall also really enjoying — I recognize this is blackjack and specifically, I think it was single-deck or maybe two-deck blackjack, in Bringing Down the House, which was later made into 21. Let’s hop to scaling. So the new book is The Voltage Effect: How to Make Good Ideas Great and Great Ideas Scale. Why write this book? You have many options with how to spend your time. You have many different projects. I don’t know how on earth — you honestly have your hands in so many things, we haven’t covered five percent of it. Why write this book?
John List: It’s a good question about — we should always, always, always evaluate how we want to spend our time because opportunity cost of time is very expensive. I was growing tired — on the one hand, when I go and give lectures, a lot of the audience, especially the younger audience, will ask me a question along the lines of, “We’ve been fighting poverty for decades. We’ve been doing study after study after study to fight poverty. How come we’re not making a dent in it?” And you can substitute poverty out for discrimination, public education, what have you, and we haven’t really made a deep impact, even though scientists have been working on this for a long time. Now that, coupled with an experience that I had in Chicago Heights — so in Chicago Heights, I started a preschool from scratch with my friends, Roland Fryer and Steve Levitt. Steve Levitt, Freakonomics author. Roland Fryer’s a famous economist at Harvard.
We start a preschool for three, four, and five-year-olds, and it took a lot of shoe leather. And we end up running this for four years. And I’m on the ground day to day to day running it, and I get great results. And then I go to scale it and I talk to policymakers and they tell me, “John, that’s a great little study, but it’s not going to scale.” And I ask them, “Why?” And they say, “It doesn’t have the silver bullet.” And I say, “What do you mean silver bullet? What does that exactly mean? I’ve been doing social science research for 25 years, and I’ve never heard this argument that my idea won’t scale because it doesn’t have the silver bullet.” And then they say, “Well, we just don’t know about this, but every time an expert tells us they have a great idea, it always fails to scale.”
So then the third part of this trifecta was my wife and I are sitting down the street from the Uber headquarters on Market Street, we’re sitting in a cafe. And I’m telling my wife about the Chicago Heights project and how it’s so frustrating. I’m telling her about all these people who always ask me, “Why aren’t we making a dent in poverty or discrimination?” And then I’m telling her about scaling at Uber. And she says, “You know what? For the next part in your career, you should take on scaling.” And I said, “What the heck? Let’s do it.”
And because all of these threads in my life, the White House, creating the Early Childhood Program, working at Uber and now Lyft, all of these had features of scaling. And then when you come to the realization that you only make big change at scale, you start to say, “Why don’t we have science around scaling? Why is it the case that people write a book that says something like, ‘Move fast and break things,’ or, ‘Throw spaghetti at the wall, and if it sticks, good,’ or, ‘Fake it till you make it.'” You say, “Where’s the science behind that?” And there isn’t.
So my chore back then when I first started saying, “I’m going to take scaling seriously” is basically I want to do the science of using science. And I’ve created science in the past, but we haven’t created it in a way that is meant to scale. What we do is we say, “I want to do a great idea in the Petri dish. And then after I’m done with it, I move on to the next idea. I don’t take seriously from the beginning that if I want to change the world, somebody’s going to have to scale this. And I need to understand the features that are important when scaling in the Petri dish itself.” So all of that came together and I said, “Look, we’re going to change the world by understanding the science of using science.”
Tim Ferriss: I want to first just give voice to an observation, and that is thinking back to your story of the onstage debate, the pro field studies camp represented by you. And then I suppose the criticisms of field studies on the opposite side, it strikes me that one of the enormous benefits of field studies, and again I don’t know what I’m talking about, so please correct me if I’m wrong, is that you can, in some respects, get around the replicability crisis that is found in basically every nook and cranny of science, because you don’t have to write grants or raise funding for a tiny pilot study that will never be replicated. If you’re working with an Uber, you have incentives aligned, you have incredible longitudinal data, and you have millions of data points. I don’t know. I just wanted to lob that out there.
John List: No, Tim, you’re right. When you think about the credibility crisis or the replication crisis in science, it really goes along the lines of you had a camp of people do lab experiments. And a lab experiment is you bring a group of sophomores into a classroom, and you have them do a task that they typically don’t do. And that’s the experimental approach in psychology. And it was the experimental approach before I came into economics, it was largely do things in the lab. And what was difficult there is that if you wanted to generalize those results to the extra lab world, a lot of the features of the extra lab world that might matter in decision making, are set at different points of the dial in the lab. So the first obvious thing is the world is not filled by rich white kids who are in the lab doing the experiments at the fancy universities.
The world is filled with truck drivers and all kinds of other people. That’s where it starts, but it doesn’t end there. Because remember, anonymity or whether somebody’s watching you, or whether you have experience in doing something, or whether people opt in or out of a market, people in a market tend to be the winners. The ones who you don’t observe may have tried and they’re long gone. So there’s selection into markets, too. So in the field, you have not only the right people, but you also have the right set of circumstances. And what you’re getting to is right, that when you can partner with a government — I work a lot with governments, I work a lot with firms. But when you can get big data quickly at scale, you can, in a very fast way, find out: did that result in Seattle on green cars?
A bunch of people in Seattle love green cars if you’re a Lyft consumer, but they really don’t like it that much in Akron. You can quickly find these things out. And then you scale where you should scale and you try something new where you shouldn’t be scaling. And I think that organizations, whether they’re for-profit or non-profit or governmental, these are unique opportunities that we’re just scratching the surface. And one of the external benefits, what economists call a positive externality, is that it’s going to be a lot easier to replicate. And the firm has a really strong incentive to replicate because they don’t want to waste a bunch of money.
Tim Ferriss: Absolutely. So let’s come back to this mention of the silver bullet. I’ve read that you have said, “Scaling is not a silver bullet problem. Rather, it is an Anna Karenina problem.” Could you explain what you mean by that? And then at some point, we don’t have to do it right now, but I would love for you to map out a success case and then a failure case or a semi failure case, and do an autopsy on both, if that makes any sense. But let’s begin with, “Not a silver bullet problem. Scaling is rather an Anna Karenina problem.” What on earth does that mean?
John List: That’s right. So the typical policymaker argues two things. One, that the result in the Petri dish won’t scale. And what they really mean by that is that there will be a voltage effect. So what I mean by voltage effect is you have a great result in the Petri dish, but when you scale it up, you turn that mountain into a molehill. Great result in the original research. Scale it up, you get nothing. That’s the voltage effect. Now, policymakers will say that truly scalable ideas need to have the silver bullet. Now, they’re exactly wrong there. What I found in my research and what I talk about in the book is that the silver bullet is really like if you have this one great characteristic you can scale. It’s like a best-shot technology. Scaling is actually a weakest link problem.
And what I mean by weakest link is — so go back to Tolstoy. Tolstoy starts Anna Karenina, one of the best first lines ever in a novel. “Happy families are all alike. Each unhappy family is unhappy in its own way.” Scaling is identical to that. The way I think about scaling is scalable ideas are all alike. Each unscalable idea is unscalable in its own way. So now you can say, “Well, what do you mean?” And what I found is that there are five major reasons why ideas don’t scale. And that’s what I call the five vital signs in the book. So your idea is scalable if it doesn’t have one of these flaws, you see. So it’s a weakest link. When I say weakest link, think about airport security. The airplane is only as secure as the weakest link. Think about your automobile.
The automobile will work, but it breaks if you have a flat tire. Or if a piston isn’t firing, that’s a weakest link problem, because it only works as long as each of the features is at a certain level. That’s how I want you to think about the Ana K problem. So now once you determine does your idea have these five vital signs — let’s go through an example to help bring these vital signs to life.
Tim Ferriss: Yes, please.
John List: Let talk about one of my favorite scientists, Jonas Salk. Salk is responsible for the polio vaccination. What Salk does is he first does, like a lot of great scientists do, he tries it out on his own kids. He finds it works.
Tim Ferriss: Skin in the game.
John List: A lot of skin in the game. And then he finds that he can replicate that result. So those original results were not a false positive. That’s vital sign number one. Does your idea have voltage? So Salk found that it does. It’s not a false positive, his initial result. Now what Jonas Salk does is he tries it out on a lot of different kids. What he finds is it works for every kid. So now he knows that the audience, what I call know your audience, or know your slice of the pie, he finds, “Wow. It works for all kids.” That’s vital sign number two. Understand how big of a piece of the pie you have or know your audience. And be true to yourself about measuring, “How big can this market be?” Now here’s where Salk enters thin ice, and that’s understand the situation in which your idea can work.
Think about COVID. COVID vaccinations have worked, but it’s really hard to get the shot in people’s arms. So like a lot of medications, medication adherence matters. And getting people to take the vaccination matters a lot. The medicine’s good, according to science. Now, how do you get it in people’s arms? That’s where this COVID vaccination scheme that we’re up to right now has failed. It hasn’t been able to deliver. This is where Salk is brilliant, because what Salk does is he leverages the healthcare system to deliver the vaccination. Now, how does he do that? Here’s what happens.
Tim, you don’t have kids yet, but when you have kids, you’re going to be in the delivery room, kid’s going to come out, everyone’s going to be happy. You’re going to make sure your wife is healthy, the baby’s healthy. And then the doctor’s going to give you a playbook. And the playbook is going to say, “We’re going to give these vaccinations today. You’re going to bring your baby back in six months, it’s going to get a checkup and some new vaccinations. You’re going to bring your baby back in 12 months, same thing. 18 months, same thing.” The polio vaccination occurs during that time period. Any parent is always going to do well by their kid and make sure that they bring their kid to the doctor checkups. So it’s not an extra cost and you just naturally get it.
So now you’ve overcome a really hard problem about the situation here. Now, in Chicago Heights, the situation that I have is I hired 30 really good teachers. If I want to scale that thing up, I need to hire 30,000 good teachers. And if I have to do that all in the same input market, say, I have to do that all around Chicago Heights, I’m screwed, because I can’t replicate the quality of the original 30 good teachers.
That’s what I mean by properties of the situation. The fourth vital sign is understand spillovers, or understand unintended consequences. In the Jonas Salk case, he’s again, got that in spades because the polio vaccination has these great externalities. Once you get it, you can’t give polio to somebody else. The fifth vital sign is understand the supply side. And what that means is how much is it going to cost you to provide it. With medications, the first feature of a medication is most of our expenditures are up front. And then after you expand a bunch of money on R&D, you have what’s called economies of scale.
To actually make the vaccination or make the pill it’s really, really cheap, but you also need to get it into the people’s arms. And that’s where we’re failing now with the COVID vaccinations is it costs a lot of money. Whereas in the Salk case, we’ve leveraged the healthcare system so it’s free, because they’re going to come in anyway and your baby just gets that vaccination. So that’s kind of a running example that is really useful to know, because it checks all the boxes. And as you know, it’s scaled with great brilliance.
Tim Ferriss: Could you expand on the unintended side effects, the spillage, maybe give some examples of where things have gone well or gone sideways?
John List: Absolutely. So one thing that is invisible to the entrepreneur or invisible to the firm is an unintended consequence, or what happens after I push my idea out there. Now, one example of an unintended consequence, what happened to us at Uber? So remember, summer of 2017, we have this idea that we are going to add tipping to the Uber app. And in doing so, we beta test it. What we find is we go to a market and we allow five percent of the drivers to be able to receive tips. So what happens? They receive tips. They work more and they make more money per hour. So it looks great. When five percent of the drivers have it, their wages go up and their labor supply goes up. Win, win, win. Now, when you roll tipping out to every driver in that market, guess what happens? They see tipping, they work more, but the market dynamics, because everyone has tipping, they undo the tipping effect.
In fact, more drivers are working, but they’re driving around with an empty car more often, because so many drivers are driving that the extra tip effect is exactly offset. So that’s an unintended consequence. Now, when we scaled it up, drivers don’t make any more per hour with tipping, they exactly offset it. And you didn’t see that when only five percent of the drivers had tipping. That’s an example of an unintended consequence that caused all the good stuff of tipping to go away. Now, on the other side, sometimes you have good stuff. Think about Facebook. If only five people in the world have Facebook, it’s pretty useless. It’s not very valuable to me. But if every one of my friends and every one of their friends and every one of my family members is on Facebook, there’s something called network externalities, that as more and more people get it, Facebook becomes even more valuable. So you need to pinpoint, does your idea have these really cool network externalities in this case? Or at scale, the good stuff becomes even better. These are things that I’m asking people to measure and take account of.
Tim Ferriss: It makes me think of Andy Grove at Intel, and I think he used to refer to it as paired incentives, but for any incentive — and I’m sure I’m getting the details wrong here, but he would ask: “What is the unintended side effect?” What is the sort of perverse behavior that you are — and I mean, perverse as an undesirable behavior that you are also incentivizing that is not explicit, right? And you should measure both. Let me make this just purely self-indulgent for a second. Let me lay out a challenge and you’re the perfect person to ask. So I have seen this and heard of this across ridesharing apps. So I don’t think it’s unique to anyone. I have certainly run into this. If drivers are penalized for canceling trips, but when they accept a trip, they realize, “Oh, wait, that’s a really short ride. I want an airport ride or something like that.” They will sometimes turn in the opposite direction and just drive away to force the rider to cancel. And then the rider gets penalized, and I’m wondering, presented with that issue, how you’d think about or how you have thought about mitigating it or solving it?
John List: No, no, you’re absolutely right. So when we did a deep dive into Uber wages across men and women, what we found is something really surprising. We found that men earned about seven percent more per hour than women as an Uber driver. And that really surprised us because wages aren’t negotiable. All men and women have the same rate card, and what a rate card means is you’re paid the same time and distance. And women receive more in tips than men receive. So we were really surprised and one reason why is because men strategically reject trips and women accept all trips. So a bad trip on Lyft or Uber, you’re right, one type of bad trip is I have to drive a half-hour to pick the person up, and then I only have a five-minute trip. One way to get around that problem is to pay the driver for time that they spend from when they get the dispatch to when they pick the customer up. We do that now at Lyft. And that’s an important reason that you can lower rejection rates.
Now, there are still cases of cancellations. What we do is there’s a fine for drivers. In fact, there’s a fine for riders who do this too, right? There’s a cancellation fee of $5. And I think using that mixture of actually paying drivers to go pick somebody up and having the relevant fines, right, taking money away, clawing it back, is kind of two instruments that we find can be successful in lowering exactly what you’re saying because it’s irritating to sit out outside in the cold. I’m in Chicago here and it’s freezing. I sit out on a corner waiting for a driver and I see them going the wrong way because they don’t want to pick me up. I’ll feel a lot better if I know they’re being penalized versus if they just go to a better trip. Right? And you’re right. The best trips are, I’m going to deadhead to the airport.
Tim Ferriss: And you also can’t use your app while they’re attempting to force you to cancel. Right? So I remember staying with a friend in Malibu once and there’s PCH and it’s a mess. Right? And if the drivers are not paid for their transit time to the pickup, then more often than not, I mean, you would confirm the ride and then go back to doing what you were doing, knowing that it would be there in 12 minutes and then check 10 minutes later, notice it’s canceled. So you can solve that by paying them, the driver, for the transit time. What if it’s a situation where it’s like downtown Chicago or downtown Austin or wherever it is and they’re just looking for longer trips, so they start going in the opposite direction? How would you address that?
John List: So they’re not supposed to know that until you get in the car.
Tim Ferriss: Yeah. I wonder why — I mean, it does happen to a fair number of people. I just wonder —
John List: What drivers usually do is they call you and say, “By the way, where are you going?” And then that’s how a lot of times they find out. So there, if you — one way that we tackle that is if we have undersupply for short trips, we end up giving incentive bonuses for drivers on number of trips. So for example, you say you get the ride bonus of $500 if you take 50 trips this week. The only way you can take 50 trips is if you do a bunch of short trips. So there are ways to incentivize, outside of the rate card, there are ways to incentivize drivers for short or long trips, and those are all at the disposal — and Uber and Lyft are constantly doing these things.
Tim Ferriss: Yeah. So I’m going to plant a — I’m going to provide some foreshadowing for a question to come, just so it can slow-bake. And so I want to talk about the four secrets to high voltage after launch: scalable incentives, marginal thinking, optimal quitting, and building culture. So I’m going to set that on just keep warm for now. I have to ask you though, because you, in a sense, alluded to this, but we didn’t get into the details. So you received your PhD. You went then to sort of sell yourself on the academic job market. This is ’96. Applied to 150 jobs. Usually, the hit rate is around 20. So one would expect around 30 interviews. You got one. So there’s that. And then you were able to sort of alchemize your career trajectory and here you are, right? Obviously in a very different position. And my question is what did people miss about you? Or what did you miss or what mistake did you make or what did you lack? Why did that happen?
John List: Yeah, it’s a really good question. I’ve been doing the post-op for over 25 years now. And I think it’s sort of two parts. One part is academia is very hierarchical and the way the funnel works is that the top schools, Harvard, University of Chicago, Princeton, MIT, et cetera, we put out about 20 to 30 students per year on this market with PhD. So we put out 25 students this year with a PhD from the University of Chicago, but we only hire back one or two. So the unwritten rule in academia is where you get your PhD, the ranking of that school will be much, much higher than where you get your first job. And then where you get tenure will be much, much lower ranked than where you get your first job. That’s just sort of how the funnel works. So when you start at the University of Wyoming, which is — I’m not sure what the ranking is, but it’s certainly not in the top five schools.
If you follow that rule, those top schools, they’re not even going to open my envelope. I didn’t know this, but when I applied to the University of Chicago, I’m quite confident that my application envelope was never opened because they saw the return address was Laramie, Wyoming. Okay. So I don’t think it was anything that I did wrong per se. Although it is true that people were very, very skeptical of my field experiments in the early ’90s. Everyone said, “You’re an idiot for doing it. We think you seem smart. Why are you throwing your career away? If you want to do experiments, you do them in the lab. What is this field experiment stuff?” So you’re right. I applied to 150 schools because I want to be an academic. I didn’t want to be a truck driver. Nothing wrong with a truck driver, but that was my outside option, let’s be honest, right? If I don’t get an academic job, I’m back in Madison, Wisconsin driving a truck. So I applied to 150 schools and I still wear that today. I got 149 people who told me you’re not even worthwhile talking to in an initial interview. And I saved every one of those envelopes that said, “Thanks, but no thanks.” And if I ever need motivation, I can look at that box of envelopes that says, “Go fly a kite list.”
Tim Ferriss: You can laminate them and put them in the floorboards.
John List: Yeah. Yeah. University of Central Florida worked in it. It ends up working well, I think too, because what I see is at the University of Central Florida, I could take chances and I can continue to take high variance plays. And what I mean by that is take risks in the research, where if I started at the University of Chicago, it might have been harder because I might have wanted to march to the same beat of every other drummer around me. And I think, University of Wyoming and as a grad student, University of Central Florida, these sort of relieved the social constraints of having to be like everyone else. And I could just be me. And I think, in the end, it really helped a lot. And that was kind of the silver lining of my approach to academia, I think.
Tim Ferriss: I have to ask you, and this may be a dead end and we can cut it if it is. This is going to show you just how much I don’t know, which is not going to come as a surprise to anyone listening. But when I think about the cold reception that you received based on the field studies and the sort of directionality of real-world experimentation. People are like, “You seem smart. This is stupid. Why are you being stupid?” Kind of reception. It reminds me of a documentary that I recently watched. It’s a NOVA documentary on fractals and Benoit Mandelbrot. And I’m wondering because part of that documentary discussed the wide spectrum of applications for fractals, lots of applications to physiology, cardiology, I mean the extremely wide-ranging. Is there any application for fractal geometry or fractal mathematics to economics? Is there any intersection there?
John List: I think there absolutely is. And I think as we move to higher-order computing and as we move to a higher-order understanding that humans have of how to apply mathematical principles to real-world questions, I think some of the big breakthroughs will be in that area, and I think part of it is going to be how do you solve these problems and get what’s called a closed-form or an analytical solution rather than have something be, well, anything can happen because when you deal with humans — the sciences are great because we have laws and we can talk about these laws because they’re physical constants. They apply everywhere that we at least understand they can exist. These laws apply. In economics, when you talk about predicting human behavior, we don’t have quantitative laws. And what we have are things that on the one hand are kind of obvious, like one of our big laws is the law of demand. And basically what that says is when price goes up, quantity demanded goes down. Well, of course, if price goes up, you buy less.
And then we don’t even have a physical or quantitative estimate because that’s called an elasticity, but the elasticity depends on the good and the circumstance, et cetera, et cetera, so we always have to measure it. I think when you start to add fractals in other aspects, it gives us a shot at least to try to be more scientific. And I think going down those dimensions can be useful. I can’t imagine an application right now, but that’s the fun part about the question, right? Is that there are possibilities. Back when I started doing field experiments, everyone said in part they’re dumb because I’m going to a baseball card convention to gather data, and they said baseball card collectors are weird and the results won’t generalize. And that might be fine. They might generalize or they might not. But what I argued was the tool will generalize. And when we get to work with firms or governments or public education, we can do large-scale field experiments there just like hard scientists, and then we can say something causal about the relationships that we observe.
Tim Ferriss: Yeah. Well, and also if you’re working — I mean, obviously, the reference here is within the halls of academia, but if you’re working in private industry also, that weird group could actually generalize to a pretty larger, a much larger subset of the population that is equally weird as a standard deviation from whatever the hell normal is. So you can have a great business focusing on weirdos.
John List: Exactly. As long as there are enough weirdos, you sure can.
Tim Ferriss: Yeah. So let’s come back to the four secrets to high voltage after launch. And I would love if we could explore these through an example or an illustration, whether real or hypothetical. Scalable incentives, marginal thinking, optimal quitting, and building culture, would you mind expanding on those?
John List: The back nine of the book is really meant for the manager or the decision maker who has scaled an idea. So at Houston, we have launched and after we’ve launched, what we have to think about is how can we make sure or at least give our idea its best chance for high voltage? So the back half of the book really draws upon my experiences in government, in firms to draw out what are the constant threads that people make mistakes around. And the first chapter is how to think about incentives that scale. And I think one of the big features that we’ve learned in behavioral economics is that using framing or loss aversion, like the clawback incentive, which is essentially give somebody an incentive up front and tell them you will take it away unless they perform. That’s called loss aversion, or it is built on the concept of loss aversion.
And also non-financial incentives. I think a lot of times we undervalue both behavioral incentives and incentives that don’t involve cash. So things like norms, things like peers, like how much does my coworker earn? That’s interesting information. Or what are their benefits or how many days off do they get? Or if I perform in a certain way, am I adhering to the norms of my company? In any case, should I tip or not? In many cases, if we can leverage the insights from behavioral economics and leverage what we’ve learned about non-financial incentives, in many ways, these types of incentives have the highest ROI because it’s really cheap to give non-financial incentives by definition.
Tim Ferriss: Could you give an example of a non-financial incentive that you’ve seen work well?
John List: I think one example that you can think about with non-financial incentives is what we’ve been doing with governments around the world. So think about the UK Behavioural Insights Team or in the Dominican Republic, I’ve helped the tax authorities raise more money through getting people to pay their taxes. In each of those cases, we’ve used messaging such as, “70 percent of people have paid their taxes on time. Why haven’t you?” That’s a type of message that works. It gets people to pay their taxes. In the Dominican Republic, we have a paper titled “The $100 Million Nudge.” So the Dominican Republic has a real problem with getting both individuals and their firms to pay their taxes. So what we did is we went to them and said, here’s a suite of behavioral interventions that are all non-financial, things like “Just a reminder that if you don’t pay your taxes, you might have to serve jail time.” Things like “Just to remind you, if you don’t pay your taxes, we can make your name public and let the world know that you haven’t paid your taxes.” Messaging like that, we raised $100 million more in tax receipts, which is 0.2 percent of GDP in the Dominican Republic.
Tim Ferriss: Boring — maybe I don’t think it’s boring, but the tactical or logistics question here, how was that messaging delivered? Was it billboards, newsletters, direct mail? How was that actually conveyed?
John List: Not boring at all. These are direct letters that the Dominican Republic, the IRS sent. So they send a letter every year to people who haven’t paid their taxes. All we did is told them, “We think we can do better than your letter.” So we had them send the letter to a group of people and then we augmented their letter with a sentence or two. And they always remind people to pay their taxes, but we used a little bit of non-financial incentive and we made bank for them. By the way, the DR should have a statue soon of me in a park named after me, I hope, but if there are any people from the DR listening —
Tim Ferriss: Yeah. Arm-in-arm with Matt Damon.
John List: Exactly, exactly. So that’s the first chapter in the second half. The second chapter is about thinking about making decisions on the margin. So a lot of your listeners who have taken economics 101 always hear this, right? “Make decisions on the margin.” And what we do terribly in the classroom as economics professors is actually explain what that means and give an example of what it means. What does it actually mean to think on the margin? So one example that I use in the book is to think about what we do at Lyft. So I’m sitting in a conference room at Lyft and the driver acquisition team comes in. The driver acquisition team is a team that is responsible for recruiting new drivers. We need to keep putting new drivers in the funnel because we want to grow our supply.
What they present to me is a slideshow that has here’s how much money we spent on Google for ads to get new drivers. Here’s how much we spent on Facebook for ads to get new drivers. And then they say, “On average, we had to pay $500 per driver on Facebook. And on average, we had to pay $600 for drivers on Google.” And then they say, “Because of that average, what we’re going to do is we’re going to spend the next tranche on Facebook ads.” And I say, “Whoa, whoa, whoa, whoa, whoa. I don’t care about the average in the last half-year. What I care about is how much did we have to spend to get the last handful of drivers, the last 10 or 20 or 50, on Google and Facebook?” They say, “Well, let us check.” They go, and then they come back and say, they say, “Well, on Facebook, it was 700 per driver, and on Google, it was 350.” Then I’m like, “Well, wait a second. Don’t you think we should be moving money, then, from Facebook to Google?” And they’re like, “Yeah, but we didn’t think about the marginal driver.”
What was the last driver and what would be the cost for the next driver, that’s marginal thinking. And I had this in the White House with superfund cleanup and we talked about how we should be spending money across hazardous and non-hazardous waste sites to clean them up. Averages in some cases are very misleading when you really should be thinking about the last one and the next one, rather than a big average. Okay? So that’s marginal thinking. Now the third one is quitting. And here, this gets a little bit personal because let’s face it. Really, the only reason why I went to college — I’m a first gen kid. I went to college because I wanted to be a golf pro. And I was offered a partial golf scholarship to go to UW-Stevens Point, University of Wisconsin at Stevens Point.
And I went up there. This would’ve been the fall of 1987, and I started playing on the golf team. About midway through that fall season, we had a weekend off and I went back to Madison, Wisconsin, and I happened upon a bunch of players. One guy who was named Steve Stricker. He’s been on the PGA Tour for years. One is Jerry Kelly. These are guys I played against in high school. There were a few years in front of me. Steve went to University of Illinois. Jerry Kelly went to Hartford to play hockey and golf. And then they both made a ton of money later, but it was really a hard lesson because I watched them on the driving range and I said, “Wow, these guys are really a lot better than me.” I didn’t realize they were that much better than me. But practicing confirmation bias, I said, “But I’m better than them around the greens.” So when their scores come out, I’m going to show them.” They went out in the morning. I went out in the afternoon and saw all the scores, and theirs were just a lot better than me. And I ended up doing a bunch of data crunching, and I realized that everything I was told as a kid because, remember, I grew up in Wisconsin and that’s Vince Lombardi country. If Vince Lombardi is a venerable coach of the Green Bay Packers, when people win the Super Bowl, the trophy is called the Lombardi trophy.
So I was raised in the shadows of Vince Lombardi, who famously said, “Winners never quit and quitters never win.” And my parents to taught me that time and time again, “Johnny, you have grit, you stick with it.” So it was really hard for me to say that dream is done. I’m going to have to quit that dream. I’m going to fulfill the golf scholarship. I went and played, and I had a very successful college golf career. But from that day on, my dream was to become an economist. I was still going to invest in golf, but it wasn’t going to be all in on golf. It’s going to be all in on golf and all in on an econ and learning about economics and how I can use it as a trade when I “grow up”.
But for me, what I see in the business world and in government is that people don’t quit enough. They don’t quit enough because they’ve been taught not to quit. They also don’t quit enough because they don’t understand or appreciate the opportunity cost of time. When you think about the person who moves to a different job or they move to a different apartment, nearly every time they’re doing it because something bad happened in the workplace or something bad happened in the apartment. That’s all well and fine, but you should also be moving when you get better opportunities.
Tim Ferriss: Yeah.
John List: And you should always be looking at your opportunity set. We don’t do that. We don’t look at our opportunities until our own lot in life is soiled. And that’s too late most of the time because a lot of opportunities have come and gone. So that’s called opportunity cost because we don’t realize that if I’m working on this idea with all of my might, that there’s an idea out there that I can’t work on.
And we tend to neglect that opportunity cost of time. So we did this large-scale experiment. I helped Steve Levitt design it and run it that had people flip a coin. They had a tough decision in life, whether it was a job or relationship, an apartment, whatever, if it came up heads, they need to change. If it comes up tails, they don’t in our Freakonomics experiment. And what you find is, when you track them over time, is that most of the people are much happier that they made the change.
So they didn’t realize that there were greener pastures out there until they were sort of forced to do it, so that’s the optimal quitting chapter. And the reason why, again, people don’t like to say the Q word. So what I say is call an audible, right? We always call an audible in football. We always pivot. When people pivot or call an audible in football, they’re glorified. But when you say “I’m quitting,” you’re chastised. So I think part of this is just reframing when you quit, and we should understand that it’s pivoting or calling an audible in your life. That’s the optimal quitting chapter.
Tim Ferriss: How would you give a pep talk to someone, could be a CEO, could be a student, could be anyone really, who’s dealing with, let’s just call it a data set or just a life experience, that is not black and white? And not to imply that your experience is black and white, but if you go into the gym for basketball practice and you’re standing next to Michael Jordan, you’re like, “Okay, there’s not much debate here at some point, like that is just a superior player, and maybe I should choose a game where I can be number one or number two or something like that.”
There are many life circumstances or business circumstances where it appears to be like a 51 percent, 49 percent type of situation. And they’re like, “Well maybe if we continue to split test and iterate,” and I’ve read of all these success stories could be survivorship bias, but nonetheless there are early studies of like pivot, pivot, pivot, and then, “Oh, my God, now you have Twitter.” What do you say to those people? Or can you think of a real-world example where it was tougher to kind of determine whether to quit or not?
John List: You’re right. In many cases, we sort of have these tweeners where the idea shows some signs and just enough to keep me going, and the job shows some promise just enough to kind of keep me going. So I always say that you don’t want to quit unless you know you have a good option to go to. And that means that you should periodically, say once a month, look at your options and make sure that the option is real, and also make sure that it is along the lines of your comparative advantage.
So there not only needs to be something that you can go to, that something exists, but also a comparative advantage as what are you good at? And what are you passionate about doing? The long run is filled with success only if you can wake up every morning and be passionate about something that you’re good at. That’s your comparative advantage. So I would say never move or rotate unless you know that what you’re rotating to, you’re good at and you’re going to love, and also that it’s a real option. Otherwise, there’s no reason just quitting to quit.
I think that there are a lot of tweener cases where it could still work out. The problem is that every time the Olympics comes, when this is aired, the Olympics will be on. And there will be a story every night about the person who persevered, the person who had to go and work at the grocery store, and then they drove in the forklift, in the basement of a cheese factory. And lo and behold, that person has become a great bobsledder. And then, the mom is on there saying, “I love Johnny.” And the dad is on there saying how “We knew Johnny would never quit.” And these are great feel-good stories, and that causes us to want to persevere.
But where is the story for the billions of people who went down the hole, and they kept going down the same hole every day to a mine that had no gold in it? Where is that story written? I’ve never read one. So we don’t realize that there are billions of people who have tried and tried and tried, but they keep hitting their head against the same wall, and they never succeeded. That’s lost opportunity right there.
Tim Ferriss: Yeah. Yeah. It’s a harder story to use on the cover to sell magazines.
John List: Yeah. Nobody wants to buy that story because that’s a story of many people’s lives. Nobody wants to read about their own life. Right?
Tim Ferriss: Building culture, let’s touch on that.
John List: The culture chapter was really fun to write because it allowed me to sort of reflect on what happened at Uber and also reflect on the research that I’ve been engaged in for over 20 years, which is along the questions of why do women get paid less than men in doing the same job? Why do people discriminate against one another in markets? What are the underpinnings for that discrimination? And how can we think about diversity and inclusiveness in a way that a firm really wants to do it?
And they’re doing it for the bottom line not just to get moral cookies because in the end, we need, we want both. We want both morality to tug at it, but we also want to make sure the firm is doing right by their shareholders because that’s what a lot of times people are going to make decisions around.
So the chapter in a way begins in Brazil. We did some work amongst Brazilian fishermen in two very distinct communities. One community had fishermen who had to fish in teams. And in those teams, they had to go out in the boat, the ocean’s very rocky, they all go out together, they bring back their catch, and they all share it. And those as villages are also built on community when it’s outside of the workplace too. So just like this brilliant utopia of everyone cooperating and getting along.
Now, inland, there’s a similar village, except those fishermen go to a lake and they all work in a solo way, soloists. And what happens there is when they get to the community, they also act in a soloist form. So what’s kind of cool is that you observe there that the workplace itself bleeds over to society. And in the one society, you have a lot more of the great public goods being provided. In the other society, it looks more selfish. So we did a bunch of experiments in these two societies. And lo and behold, we find that what happens in the workplace really spills over to the community.
So bringing all that together, what I say in the culture chapter is, as a firm or as an organization, we should want to build our own kabuchu, that’s the society that is built on community spirit. And we should do that from the very beginning. Very subtle things like letting people know that wages are negotiable in our job advertisements. That seems kind of innocuous.
If you say wages are negotiable, well, what does that do? What it does is something interesting. It gives women sort of the license to negotiate because what we find in our data is when you say in your job ad that wages are negotiable, women will negotiate as much as men, and they will end up with similar wages. But if you leave that sentence out, women will not negotiate at all really, and then they start with lower wages.
So right away, just your job advertisement, it leads to a pool of applicants that can differ but also the wages might differ. And if you want to scale a culture of equality and you’re paid your marginal product, and we’re not going to let subtle things happen that lead to very different wage profiles, we can do that from our very beginning of our firm. So the idea is there are a lot of little nuggets in that chapter about how you can build your kabuchu, and those are the tips in the chapter on build your own culture.
Tim Ferriss: I don’t know if this is going to tie in culture. It certainly doesn’t have to, but I am looking at some of your favorite things and I see StubHub, could you please explain why StubHub is on your favorites list?
John List: One of my favorite apps is indeed StubHub. StubHub is great because it’s the ultimate market for seats to events. And I can wait till the last minute, I can do a model about what prices do. I’ve bought Super Bowl tickets on StubHub. The other night I went to the Bulls game. I sat courtside at the Bulls game. Wait till the end. There are a few courtside seats for 350 each. My wife gets to hug Benny the Bull. I don’t have to make plans till the very end. It’s the ultimate market that’s based on supply and demand.
And then, there’s a little bit of behavioral economics in there because I’ve sold on StubHub too, and it’s kind of neat because I have to try to figure out how should I price? And if my tickets aren’t selling, how should I depreciate my prices if I really don’t want to go to this event? So it’s economists’ playground here, StubHub is. But plus, it’s great in many cases for the customer but also for the seller. I don’t like StubHub’s fees that much. But nevertheless, they’re taking their bite from the apple as well, and I’m still getting some surplus, so I love StubHub.
Tim Ferriss: I was just thinking, having had a number of podcasts now interviewing people about Web3, it’s a somewhat controversial term, but let’s just make it more specific. NFT marketplace is where all of the transactions recorded can be reviewed on the blockchain. I would imagine that that would provide the incredible playing field for behavioral economists who have the ability to crunch data like data scientists. I would imagine you could probably pull all sorts of fascinating behaviors, also see bad behaviors. I’m sure people are doing wash trading and things like that. But because it is all preserved and there’s this self-similarity in the blockchain and the sense you have all these nodes that have, in effect, copies of these ledgers, you could really do some fascinating digging, I would have to imagine, but that’s just me.
John List: No, you’re right a hundred percent. And just thinking about fintech in general, we’ve thought hard about how we can be players there. When I say we, I mean my team, and we have a few irons in the fire.
Tim Ferriss: Now, question for you, is this your university team or is this like a private kind of SEAL Team Six for profit team that you have assembled?
John List: Combo.
Tim Ferriss: Combo?
John List: So it’s a little combo platter because what we have is we have great PhD students here who want to work on cutting-edge data science, and why wouldn’t I look there, right?
Tim Ferriss: Yeah.
John List: And not only because there’s a wealth of data, but also because there are a lot of big juicy apples that are hanging pretty low. And it’s going to be really important, not only for consumers, but also for regulators. Regulators are thinking very hard now —
Tim Ferriss: Oh, yes, they are.
John List: — about what they should be doing. And in many cases, they’re flying blind, and they’re flying blind because people have not taken seriously a ton of what are the learnings from the data and what are the social costs or potential — back in the day, in 2008 financial crisis, we learned a lot from that and a lot about the system and some of the weak points. This comes back to the weakest link problem. A lot of these examples are, “You’re as weak as your weakest point.” And if that implodes, you’re done. And it’s really trying to understand questions around that. So we’re dabbling a bit, Tim, in fintech but nothing to talk about just yet.
Tim Ferriss: Well, I’ll take that as a teaser. I look forward to when you can discuss more publicly —
John List: The next book.
Tim Ferriss: — the next book. So speaking of books, I do want to ask you about a specific book, and I will admit right up front that I’m embarrassed I haven’t read this book, and that is Wealth of Nations by Adam Smith, which I know you’re a fan of. Part of the reason I haven’t read this book is the same reason that I had some misgivings about this NOVA documentary I watched about Mandelbrot and fractals because the documentary was made more than 10 years ago. And I know a lot has happened in the intervening 10 years, and I don’t know what has been disproven or what has been updated, what has been amended. So my question is, if one were to read Wealth of Nations by Adam Smith, what caveats or preface would you provide so that they are not misled — I don’t want to say misled, so that they read it in as informed and contemporary as a way as possible?
John List: Yeah. I would probably say don’t read the original, right? So my advice is don’t read it. And the reason why is because, A, it’s painstakingly written. Smith had this feature that a lot of writers back then had to make a point that could be made in a paragraph, ends up being 25 pages of words that you have to look up what a lot of them mean. But the true beauty is exactly as you pointed out for an economist. The true beauty is I sort of know how the profession has evolved and how economic thinking has evolved.
And for me, The Wealth of Nations is so beautiful because it set off — he’s a father of economics, and it set off his level of thinking, must have been the deepest of any philosopher I’ve ever read. And I can appreciate the beauty because I know a lot about economics and where it’s gone since then. He talks about specialization as an example, specialization in the pin factory. And he goes through this long and tortuous and arduous explanation of the pin factory. I’m like, “Adam, just get through it. This is crazy.” So I can jump and see the beauty about what was known.
But to me, the beauty is a lot of that was not known, and he had these gems. So I think it would be incredibly difficult for a person, who is not well versed in economics, to read Adam Smith because it would be a ton of head scratching, “I think he’s trying to get at this issue, but I don’t know, it’s kind of that one.” And maybe I should just read the Theory of Moral Sentiments. So I think for the person who is not an astute and expert reader, I would say go to the Moral Sentiments because there you’ve got kind of the behavioralist at work.
Adam Smith is one of the early behavioral economists, and he talks about the moral hector on one shoulder, et cetera, et cetera. And you see a lot of the underpinnings of behavioral economics, a lot of the underpinnings of behavioral economics are common sense. And then, you can kind of see, “Wow, he had that idea back then.” It took him a long time say it. He’s verbose in the theory as well. But nevertheless, I would probably say the reader would direct there. And then, after you take several econ classes, then go to The Wealth of Nations.
Tim Ferriss: Right. So you’re not reading an ancient medical textbook and coming up with phrenology as the cutting-edge treatment or diagnostic tool. Another approach that I might take is to read, as a starting point, the Wikipedia entry on Adam Smith and the Wikipedia entry on Wealth of Nations because there will be footnotes and references and also mentions of different controversies or updates or factual inaccuracies or whatever it might be. So I might start with that and then jump to Moral Sentiments.
Well, John, this has been so much fun. People can certainly learn more about the latest book at thevoltageeffect.com, that is effect with an E, E-F-F-E-C-T, Voltage Effect. Subtitle, How to Make Good Ideas Great and Great Ideas Scale. You’re an excellent teacher. I have really enjoyed this conversation, and you’re very, very good at weaving together examples and illustrations with principles, which I just think is so critical if you want anything to stick, which makes me all the more interested in studying what you did with this preschool. So another conversation for another time. We didn’t even get to dig into the soccer team that you have at home, these eight kids, we’ll go further with that maybe in another conversation. But is there anything else that you would like to say, any closing comments, requests of the audience, anything at all that you’d like to add before we close up for this first conversation?
John List: I just want to say thanks, Tim, for having me. The book does cover my work with the Chicago White Sox. I built a draft model for them, and I call it Moneyball infinity because it takes a Moneyball idea and the data science idea to a new level. I have data all the way back to kids who are seven and eight years old, so I know Bryce Harper’s stats when he was eight and I —
Tim Ferriss: Wow.
John List: — scraped a bunch of data and built a model for the White Sox. And I bring that up because economics is life, and life is economics. And with a little bit of data, and just an economics 101 understanding, you can really go a long way to change the world and make it a better place — from the White Sox to the White House, to Uber and Lyft, to the academy, the type of thinking, not only around scaling, how to scale ideas, but also just very simple economic principles will absolutely make your life and your family’s life a lot better.
Tim Ferriss: Mm-hmm (affirmative). Well, I am very excited to not just dig into the new book but dig into just about everything that you’ve done. It’s been so fun to prepare for this conversation, and I really appreciate you taking so much time. And again, probably at some point, we’ll see if it happens, but do a round two just to talk about how in the hell you get all of this done. I’m looking at the list of your projects. It goes on for pages and pages and pages. We didn’t even get to talk about Vernon Smith and Gary Becker. Another time. Prospect theory. Although I think we touched on a good portion of that, there is so much here.
So I look forward to the book and learning more from you virtually, maybe directly, and I really appreciate you sharing your knowledge and learnings and successes and failures with us in this conversation, so thank you very, very much. And for everybody listening, we will have links to everything we’ve discussed in the show notes as pretty usual at tim.blog/podcast, and until next time. Thank you so much for tuning in.
The Tim Ferriss Show is one of the most popular podcasts in the world with more than 900 million downloads. It has been selected for "Best of Apple Podcasts" three times, it is often the #1 interview podcast across all of Apple Podcasts, and it's been ranked #1 out of 400,000+ podcasts on many occasions. To listen to any of the past episodes for free, check out this page.
So game theory was not invented by Nash. Modern game theory was “invented” by John von Neumann in 1928.