The UX research reckoning is here | Judd Antin (Airbnb, Meta)

Lenny's Podcast: Product | Growth | Career

Lenny Rachitsky

Technology, Business, Entrepreneurship

5 • 1.5K Ratings

🗓️ 4 January 2024

⏱️ 75 minutes

🔗️ Recording | iTunes | RSS

🧾️ Download transcript

Summary

Judd Antin has spent 15 years leading research and design teams at companies like Yahoo, Meta, and Airbnb. His direct reports have gone on to lead user research at Figma, Notion, Slack, Robinhood, Duolingo, AllTrails, and more. In our conversation, we unpack the transformation that the user-research field is experiencing. Specifically:

• Where user research went wrong over the past decade

• The three types of research—macro, middle-range, and micro—and the purpose of each

• How to effectively integrate researchers into the product development process

• The “user-centered performance” phenomenon and why it’s a waste of time

• Common tropes about PMs, from researchers

• The ideal ratio of researchers in a company

• Why Judd says NPS is useless, and what to use instead

—

Brought to you by Teal—Your personal career growth platform | Vanta—Automate compliance. Simplify security | Ahrefs—Improve your website’s SEO for free

—

Find the full transcript at: https://www.lennyspodcast.com/the-ux-research-reckoning-is-here-judd-antin-airbnb-meta/

—

Where to find Judd Antin:

• LinkedIn: https://www.linkedin.com/in/juddantin/

• Website: https://juddantin.com/

• Blog: https://medium.com/onebigthought

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Judd’s background

(04:16) Critiques and responses to Judd’s post “The UX Research Reckoning Is Here”

(07:33) The state of user research

(08:53) Macro, middle-range, and micro research

(14:05) What teams get wrong when it comes to research

(15:46) The importance of integrating research from the beginning

(17:30) Traits of great researchers

(19:53) Advice for evaluating user researchers

(21:10) Balancing business and product focus

(23:55) User-centered performance

(26:42) The role of intuition in product development

(30:15) Checking your gut instincts

(32:54) Common tropes about PMs, from researchers

(41:02) A/B testing vs. user research

(43:15) Hindsight bias and narrative fallacy

(44:55) Making recommendations based on research

(47:26) Advice for teams on how to leverage researchers

(51:18) How product managers can be better partners to user researchers

(56:53) The ideal ratio of researchers in a company

(59:43) Empowering user researchers to drive impact

(01:03:39) The limitations of NPS as a metric

(01:06:48) The risks of dogfooding

(01:08:51) Lightning round

—

Referenced:

• Matt Gallivan on LinkedIn: https://www.linkedin.com/in/mattgallivan/

• Janna Bray on LinkedIn: https://www.linkedin.com/in/janna-bray-a4046a25/

• Celeste Ridlen on LinkedIn: https://www.linkedin.com/in/celesteridlen/

• Rebecca Gray on LinkedIn: https://www.linkedin.com/in/rebeccagray2/

• Hannah Pileggi on LinkedIn: https://www.linkedin.com/in/hannah-pileggi-43169314/

• Louise Beryl on LinkedIn: https://www.linkedin.com/in/louise-beryl-13225833/

• The UX Research Reckoning Is Here: https://medium.com/onebigthought/the-ux-research-reckoning-is-here-c63710ea4084

• The end of the “free money” era: https://www.theguardian.com/technology/2023/apr/11/techscape-zirp-tech-boom

• Cognitive biases: https://en.wikipedia.org/wiki/List_of_cognitive_biases

• IDEO design thinking: https://designthinking.ideo.com/

• Everything Is Obvious: How Common Sense Fails Us: https://www.amazon.com/Everything-Obvious-Common-Sense-Fails/dp/0307951790

• Patrick Collison’s tweet: https://twitter.com/patrickc/status/1443215022029619200?lang=en

• Brian Chesky on LinkedIn: https://www.linkedin.com/in/brianchesky/

• Brian Chesky on Lenny’s Podcast: https://www.lennyspodcast.com/brian-cheskys-new-playbook/

• NPS: https://en.wikipedia.org/wiki/Net_promoter_score

• What is CSAT and how do you measure it?: https://www.qualtrics.com/experience-management/customer/what-is-csat/

• Michael Murakami on LinkedIn: https://www.linkedin.com/in/michaelhmurakami/

• Bad Leadership: What It Is, How It Happens, Why It Matters: https://www.amazon.com/Bad-Leadership-Happens-Matters-Common/dp/1591391660

• Demon Copperhead: https://www.amazon.com/Demon-Copperhead-Novel-Barbara-Kingsolver/dp/0063251922

• All Systems Red: The Murderbot Diaries: https://www.amazon.com/All-Systems-Red-Murderbot-Diaries/dp/0765397536

• The Last of Us on HBO: https://www.hbo.com/the-last-of-us

• Belay glasses: https://www.amazon.com/Belay-Glasses-Climbing-Comfortable-Sturdy/dp/B08GSBYDKQ/

• Epictetus: Control What You Can—Especially Yourself: https://www.shortform.com/blog/epictetus-control/

• The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change: https://www.amazon.com/Habits-Highly-Effective-People-Powerful/dp/0743269519

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

—

Lenny may be an investor in the companies discussed.

Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

Transcript

Click on a timestamp to play from that location

0:00.0	user-centered performance refers to customer obsession or user-centered practice that is symbolic
0:06.8	rather than focused on learning. It's hugely common, I would argue. It's work we do to signal to each other how customer obsessed we are not because we want to make a different decision.
0:17.0	If you're like listeners are like I don't do that I'm kind of like think about it for a second. This is extremely common.
0:24.1	Every time a PM comes to a researcher at the end of a product process and says,
0:28.8	can you just run a quick user study? You know, just to validate our assumptions.
0:32.4	That's user-centered performance. It's too late to matter. We got to ship it, right? What they want is to check the box. One of my big kind of mantras was, we don't validate, we falsifyify we are looking to be wrong many PMs many designers are not in that place
0:47.6	They do not want to be wrong. They're looking to validate and that's user-centered performance
0:57.0	Today and that's user-centered performance. Today my guest is Judd Anton. Judd helped build the user research practice at Facebook.
1:00.0	He was a long time head of research at Airbnb, and his direct reports have gone on to lead research teams at Figma, Notion, Slack, Robin Hood, Duolingo, Fair, and other amazing companies.
1:11.6	These days, Judd spends this time consulting, helping companies
1:15.0	with organizational challenges, product strategy, design, research, hiring, onboarding, and crisis
1:20.8	management. In our conversation, we unpack a conclusion that Judd has come to recently
1:26.7	about how the user research field is going through a reckoning and what needs to change,
1:32.3	both within the user research field and how companies leverage
1:35.6	user research going forward. Judge shares what the user research field has gotten wrong
1:40.3	over the last decade, how PMs and designers rely on user research too often and to answer
1:46.3	the wrong questions.
1:48.0	Where user research will continue to provide significant value and how to best leverage
1:51.8	your researchers, why it's important for
1:53.8	researchers to think about the business goals more versus just what the users need,
1:57.4	what to look for when you're hiring you user researcher, how PMs can be better
2:01.7	partners to researchers, and also a phenomenon that I love
	...

Please login to see the full transcript.

Previous episode | Next episode

Disclaimer: The podcast and artwork embedded on this page are from Lenny Rachitsky, and are the property of its owner and not affiliated with or endorsed by Tapesearch.

Generated transcripts are the property of Lenny Rachitsky and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.