The Utility of Surveys

In this episode of the Total Survey Design Podcast, Seon Yup and Azdren dive into the utility of surveys and their vital role in understanding public opinion and decision-making. They share examples, like A&W's marketing mishap with their third-pound burger, and the importance of surveys in evaluating high-cost programs. From business insights to educational improvements, surveys offer invaluable data that can guide better choices and avoid costly mistakes. Tune in to learn why surveys matter and how they can transform various aspects of life and work.

"Kalimba Relaxation Music" Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 4.0 License
http://creativecommons.org/licenses/by/4.0/

A special thank you to Don Coma.

Find us online at: instagram.com/totalsurveydesign/
https://taplink.cc/totalsurveydesign
Contact us at: totalsurveydesign@gmail.com

Transcript

SYL: Hello everyone. Welcome back to the Total Survey Design Podcast.

My name is Seon Yup,

In this episode, we talk about the utility of surveys and why surveys matter.

AC: And I am Azdren.

I like to describe Surveys as a tool that is essentially the closest thing that we have to reading people’s minds.

AC: One of the main uses of surveys is to better understand public opinion on various topics, such as political attitudes and preferences, product preferences, fashion tastes, happiness levels, stress levels, ideological outlooks, and other things that cannot be physically observed or measured. 

I am convinced that every business, every organization, every service, and every individual can find a use for surveys that would benefit their lives. Everybody from business owners or entrepreneurs, managers at nonprofit organizations, educators at every level, or even event planners, lawyers, and everyone in between can find a use for surveys that will have a meaningful impact on their businesses and their careers.

SYL: If we think about how businesses can use surveys, the survey can take out a lot of the guess work of what customers want and need. 

Let’s take an interesting case involving the burger chain A&W. 

SYL: The third pound burger … This was a real product launched by A&W in 1985. And it flopped big time. Even though it was cheaper than the McDonalds quarter pounder, it had more meat, and in taste tests it beat the quarter pounder. But it failed big time. Why could it be?

SYL: The problem was that A&W did not anticipate that most people do not intuitively understand fractions, and that most people actually thought that one third was less than a quarter, just because a quarter had a larger denominator in it. The former owner of A&W blamed Americans’ for what he called their widespread innumeracy, but in our opinion, this was the failure of a company launching a new product with decisions made based on intuition, without actually collecting the data through a simple survey. 

This is where surveys come in as one of the most powerful tools. It would have been possible to discover this issue for a fraction of the cost of what was lost by the failed product launch.

AC: Some of the projects that I have worked on involved a lot more complicated surveys, that often cost tens of thousands of dollars to conduct.

In my current role as a program evaluator for the Social and Economic Sciences Research Center, one of our tasks that we do is that as a team we design, administer, and analyze data from surveys for programs that have received National Science Foundation funding. 

The program evaluations typically collect survey responses from the Principal investigators who are running the projects, from students participating in the programs, and sometimes even from the parents of the students.

The goal with these surveys is to gauge the impact of the programs on various student outcomes, such as their desire to pursue careers in STEM as a result of having participated in the program. 

We then analyze the data collected through the surveys and create reports with graphs and charts that illustrate the impact that a program has. 

But the cost of a few tens of thousands of dollars is justified by the sincere desire to learn whether a project that costs hundreds of thousands or millions of dollars is having its intended impact. 

Instead of just guessing if a special program at a university is achieving the intended goal, through surveys, we can now get a better sense if the goals are achieved, and if not, how future years could be improved so that the same mistakes are not repeated and so that outcomes improve over time. This is the utility of surveys…

SYL: Another example of the practical uses of surveys. As an instructor, I have used surveys to gauge simple desires of my students, such as what time of the day students prefer to receive their weekly announcements or what kind of content they would like to see more of in class. I have also used anonymous surveys in the classroom and after guest lectures to assess my own performance and figure out what I can do better in future lectures.

While many surveys can be conducted practically for free, the most expensive survey to ever be conducted is probably the 2020 Decennial U.S. Census. It is one of the largest and most comprehensive surveys, aiming to capture demographic data on every single person living in the United States of America. The 2020 Census cost an estimated $15.6 billion, which is the cost of planning, data collection, and data analysis of the Census. The U.S. census data is a critical tool for many important uses, such as the re-drawing of Congressional seats, or the redistribution of Federal funds.

AC: I actually worked as a census field supervisor during the 2020 census, covering several counties in Eastern Washington. It was fun. I worked with many enumerators, who conducted the survey on the ground. I got to officially swear in and train workers. And I got to deal with some difficult antigovernmental people who would refuse to answer any questions. It was fascinating.

SYL: Luckily, most surveys do not cost billions of dollars. Many surveys can be done for little to no money, depending on what the goal of the survey is. Most academic surveys require a few thousand dollars to get good data for their research. Business surveys might require some kind of an incentive, like a product discount to get people to respond… We will spend more time covering the costs of surveys in a future episode. 

AC: One of the most important takeaways we hope you have from this discussion is that surveys are a useful tool that should be used by everyone. Having a better understanding of how surveys work can only be beneficial, which is why we are here. But even more importantly, not using surveys is a huge missed opportunity in any context. 

I want to end this episode by mentioning how I believe that surveys are only going to be used more and more as time goes on. Even if we get to the point that a technology exists that can read minds, like some powerful futuristic FMRI machine, it will still be a long way until the price drops enough to where it is accessible to the average person and would be a viable alternative to surveys.

AC: Before initiating any data collection efforts, it is crucial to appreciate the significance of surveys in research. Surveys are pivotal tools, often underutilized. For example, it is estimated that the effectiveness of 99% of government programs remains unknown if they are successful. Surveys offer a powerful method to gauge program outcomes, potentially leading to significant reductions in inefficiency and improvements in productivity. Surveys not only serve to gather data on various subjects, including public opinion and program efficacy, but they also play a crucial role in informing decision-making processes. Understanding the utility of surveys can transform how we learn about and improve upon various initiatives.