Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor, startup advisor)

Lenny's Podcast: Product | Growth | Career

Lenny Rachitsky

Technology, Business, Entrepreneurship

5 • 1.5K Ratings

🗓️ 9 November 2023

⏱️ 84 minutes

🔗️ Recording | iTunes | RSS

🧾️ Download transcript

Summary

Ramesh Johari is a professor at Stanford University focusing on data science methods and practice, as well as the design and operation of online markets and platforms. Beyond academia, Ramesh has advised some incredible startups, including Airbnb, Uber, Bumble, and Stitch Fix. Today we discuss:

• What exactly a marketplace is, if you boil it down

• What you need to get right to build a successful marketplace

• How to optimize any marketplace

• An easy litmus test to see if there’s an opportunity to build a marketplace in the space

• The role of data science in successful marketplaces

• Ramesh’s philosophy on experimentation and AI

• Advice on implementing rating systems

• Why learning isn’t free

—

Brought to you by Sanity—The most customizable content layer to power your growth engine | Hex—Helping teams ask and answer data questions by working together | Eppo—Run reliable, impactful experiments

—

Find the full transcript at: https://www.lennyspodcast.com/marketplace-lessons-from-uber-airbnb-bumble-and-more-ramesh-johari-stanford-professor-startup/

—

Where to find Ramesh Johari:

• LinkedIn: https://www.linkedin.com/in/rameshjohari/

• Website: https://web.stanford.edu/~rjohari/

• X: https://twitter.com/rameshjohari

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Ramesh’s background

(04:31) A brief overview of what a marketplace is

(08:10) The role of data science in marketplaces

(11:21) Common flaws of marketplaces

(16:43) Why every founder is a marketplace founder

(20:26) How Substack increased value to creators by driving demand

(20:58) An example of overcommitting at eBay

(22:24) An easy litmus test for marketplaces

(25:52) Thoughts on employees vs. contractors

(28:02) How to leverage data scientists to improve your marketplace

(34:10) Correlation vs. causation

(35:27) Decisions that should be made using data

(39:29) Ramesh’s philosophy on experimentation

(41:06) How to find a balance between running experiments and finding new opportunities

(44:11) Badging in marketplaces

(46:04) The “superhost” badge at Airbnb

(49:59) How marketplaces are like a game of Whac-A-Mole

(52:41) How to shift an organization’s focus from impact to learning

(55:43) Frequentist vs. Bayesian A/B testing

(57:50) The idea that learning is costly

(1:01:55) The basics of rating systems

(1:04:41) The problem with averaging

(1:07:14) Double-blind reviews at Airbnb

(1:08:55) How large language models are affecting data science

(1:11:27) Lightning round

—

Referenced:

• Riley Newman on LinkedIn: https://www.linkedin.com/in/rileynewman/

• Upwork (formerly Odesk): https://www.upwork.com/

• Ancient Agora: https://en.wikipedia.org/wiki/Ancient_Agora_of_Athens

• Trajan’s Market: https://en.wikipedia.org/wiki/Trajan%27s_Market

• Kayak: https://www.kayak.com/

• UrbanSitter: https://www.urbansitter.com/

• Thumbtack: https://www.thumbtack.com/

• Substack: https://substack.com/

• Ebay: https://www.ebay.com/

• Coase: “The Nature of the Firm”: https://en.wikipedia.org/wiki/The_Nature_of_the_Firm

• Stitch Fix: https://www.stitchfix.com/

• A/B Testing with Fat Tails: https://www.journals.uchicago.edu/doi/abs/10.1086/710607

• The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon): https://www.lennyspodcast.com/the-ultimate-guide-to-ab-testing-ronny-kohavi-airbnb-microsoft-amazon/

• Servaes Tholen on LinkedIn: https://www.linkedin.com/in/servaestholen/

• Bayesian A/B Testing: A More Calculated Approach to an A/B Test: https://blog.hubspot.com/marketing/bayesian-ab-testing

• Designing Informative Rating Systems: Evidence from an Online Labor Market: https://arxiv.org/abs/1810.13028

• Reputation and Feedback Systems in Online Platform Markets: https://faculty.haas.berkeley.edu/stadelis/Annual_Review_Tadelis.pdf

• How to Lie with Statistics: https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728

• David Freedman’s books on Amazon: https://www.amazon.com/stores/David-Freedman/author/B001IGLSGA

• Four Thousand Weeks: Time Management for Mortals: https://www.amazon.com/Four-Thousand-Weeks-Management-Mortals/dp/0374159122

• The Alpinist on Prime Video: https://www.amazon.com/Alpinist-Peter-Mortimer/dp/B09KYDWVVC

• Only Murders in the Building on Hulu: https://www.hulu.com/series/only-murders-in-the-building-ef31c7e1-cd0f-4e07-848d-1cbfedb50ddf

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

—

Lenny may be an investor in the companies discussed.

Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

Transcript

Click on a timestamp to play from that location

0:00.0	Marketplace is a little bit like a game of Wacamole.
0:03.0	Like one example that I came across with one of the companies I worked with that I love is our new supply
0:09.6	side was having a pretty bad experience.
0:12.1	So what we decided to do is build some custom bespoke
0:14.4	features that were really going to direct them to more experienced folks on the
0:18.0	other side of the market. Good. And then yeah, lo and behold, you know, pretty soon
0:22.0	those metrics start to look better, but then we're looking at it.
0:24.0	We're like, wait a second, now, you know, the existing folks on the other side are having a worse experience.
0:29.2	So you kind of whiplash around, you're like, oh, wait a second, we better do something about that.
0:32.4	So we take
0:32.8	them we try to match them up with the more experienced folks and now suddenly a month
0:36.6	after that you're like you know wait a second and your metrics just keep moving
0:40.0	around and that's because the Wacamole game here is ultimately a lot of marketplace
0:44.0	management is moving attention and inventory around. Many of the changes that are
0:48.1	most consequential create winners and losers and rolling with those changes is
0:51.7	about recognizing whether the winners you've created are more important to your business than the losers you've created in the process.
1:00.0	Today my guest is Ramesh Joe Hari. Ramesh is a professor at Stanford University where he does research on and teaches data science methods and practices with a specific focus on the design and operation of online marketplaces.
1:14.7	He's advised and worked with some of the biggest marketplaces in the world, including
1:18.4	Airbnb, Uber, Stripe, Bumble, Stitch Fix, Upwork, and many others, and in our conversation we get super nerdy on how to build a thriving marketplace,
1:29.0	including where to focus your resources to fuel the marketplace flywheel of growth, why data and data
1:34.6	science is so central to building a successful marketplace, how to design a better review
1:40.1	system, why as a founder you shouldn't think of yourself as a marketplace
	...

Please login to see the full transcript.

Previous episode | Next episode

Disclaimer: The podcast and artwork embedded on this page are from Lenny Rachitsky, and are the property of its owner and not affiliated with or endorsed by Tapesearch.

Generated transcripts are the property of Lenny Rachitsky and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.