Total Survey Design
Total Survey Design is a podcast for explaining the complexities of survey design. This podcast serves a diverse audience, including academics, small business owners, nonprofits, industry professionals, and students. Each season features episodes covering topics from survey utility to sample sizes, and question design to total survey error. Episode content includes insightful discussions, expert interviews, and special event coverage to enhance your survey skills and understanding.
Total Survey Design
How the Dunning-Kruger Effect Skews Self-Reported Skill Ratings
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
In this episode, Dr. Azdren Coma explores how the Dunning-Kruger effect distorts self-reported skill ratings in hiring and surveys. A seemingly straightforward question like “On a scale of 0 to 10, how skilled are you in Microsoft Excel?” can produce misleading data because undefined scale points allow respondents to interpret them differently. Beginners often overestimate their abilities due to limited awareness of complexity, while more skilled individuals rate themselves more conservatively as they recognize what they don’t know—potentially reversing the ranking of candidates.
This episode breaks down the psychological mechanism behind the bias, illustrates it with real-world examples, and offers practical fixes for better data quality, including fully labeling scales, using clearer anchors, and shifting to task-specific questions. Ideal for HR professionals, hiring managers, and anyone relying on self-assessments in talent acquisition or people operations.
Contact us at: totalsurveydesign@gmail.com
Find us online at: instagram.com/totalsurveydesign/
Podcasts we love
Check out these other fine podcasts recommended by us, not an algorithm.
POP: Public Opinion Podcast
American Association for Public Opinion Research
NN/G UX Podcast
Nielsen Norman Group
Scope Conditions Podcast
Alan Jacobs and Yang-Yang Zhou
Social Research Methods
Armin Trost