Total Survey Design
Total Survey Design is a podcast for explaining the complexities of survey design. This podcast serves a diverse audience, including small business owners, nonprofits, industry professionals, academics, and students. Each season features 10 to 20 concise episodes, each lasting 5 to 20 minutes, covering topics from survey utility to sample sizes, and question design to total survey error. Episode content includes insightful discussions, expert interviews, and special event coverage to enhance your survey skills and understanding.
Total Survey Design
The Total Survey Design Method - Reducing Costs, Increasing Benefits, and How to Get the Best Data!
Find us online at: instagram.com/totalsurveydesign/
https://taplink.cc/totalsurveydesign
Contact us at: totalsurveydesign@gmail.com
AC: In this episode, we talk about the total survey design method.
Or how to make a survey that collects the highest quality data possible.
SYL: The goal of Good survey design will always be to reduce the sources of error as much as possible. The total survey design approach is meant to achieve this goal.
These ideas within the total survey design method are rooted in social exchange theory.
SYL: A passage in the 2014 edition of the Tailored Design method book says, “People are more likely to comply with a request from someone else if they believe and trust that the rewards for complying with that request will eventually exceed the costs of complying.”
AC: This means that when we design a survey, we need to consider the respondents’ perspectives while considering three factors: cost, benefit, and trust.
In a nutshell, social exchange theory views social interactions as a balance between weighing the costs and benefits of participating in a survey, while also factoring in the trust that individual has for the surveyor. Essentially, the goal of a survey should be to reduce the costs for a respondent, increase the benefits, and increase trust.
AC: Let’s talk more in detail about social exchange theory, beginning with reducing the costs for respondents.
SYL: Here, when we say cost, we’re not referring to costs only in its literal monetary sense, But also from a social psychological perspective. So, costs could mean things such as the cost of inconvenience, whether the survey takes a lot of effort for the respondent, perhaps the questions are too difficultto answer. Or perhaps the survey is too time-consuming?
Let’s discuss how we could reduce costs for the respondents.
SYL: One way to reduce costs is by reducing the length of the survey. The longer the survey, the more time and effort it take for the respondents to complete the survey.
AC: Often, there is an urge for people creating a questionnaire to ask about more things than they need to. It is tempting. You think, I am only getting one chance to ask everything that I want to, might as well cover my basic.
Yet, if the data is not going to be used, the questions should not be in the survey. It takes a lot of time to figure out what exactly is absolutely essential and needed, but this is something that we urge more survey creators to consider when designing a survey. For business surveys, it might help to ask “what is the ultimate goal of this survey?” For academics, it might be “what is my main dependent and independent variable or variables?” Answering these questions might help focus your survey.
Taking the time to think about what questions can be removed will result in better data by reducing the costs to the respondents.
SYL: We can also reduce the cost to respondents by making the survey less complex. When questions are worded with overly complex wordings or technical jargons, it takes more effort to understand the meaning of the question.
Costs can also be reduced through good visual design. For example, by using contrasting colors to emphasize a text.
AC: Next, let’s move on increasing the benefits to respondents.
Again, here we’re not necessarily talking about just the financial benefits, although financial incentives are often the most important element of increasing the response rate. But many social scientists would agree that people are not only motivated by monetary benefits, but also social-psychological factors – like prestige or the idea that they’re being helpful.
Here are some of the ways to emphasize the benefits to respondents.
AC: First of all, money can be used as an incentive in various ways: such as in pre-incentives, before starting the survey, or incentives that are given upon completing a survey, such as a direct payment, a discount coupon for a business, or a raffle.
Pre-incentives are an interesting factor that are highly relevant especially considering social exchange theory. A great example of a pre-incentive would be in the form of a dollar bill included in mail surveys. When the respondent opens up the envelope, the first thing they see is this one-dolllar bill. Keep in mind, the one dollar incentive is not meant as a payment, but the researcher emphasizes that the dollar is an incentive and a show of appreciation for the participation in the survey. The surveyor is taking the first step in trusting that the respondent will complete the survey, and based on social exchange theory, we can predict that many people would also want to follow through and return the favor.
SYL: Other benefits are based on emphasizing values, norms and, ideas that most people see as positive. For example, Respondents could be informed that their participation are helping the scientific research. Or, if it is a subject that the respondent cares about, something like a particular political issue, the respondent could be invited to share their opinion on that particular political issue.
Another example would be highlighting the potential ways in which how the collected data would be used to making the community a better place. If the survey is about increasing bus access in a community, by participating in the survey, a respondent is ensuring that their own access to city buses might be improved.
Another solution would be offering the respondents different options of completing survey. Such as a different time. I could speak about this from a first-hand experience. While I was collecting data for survey projects through telephone, a lot of people would pick up but say they do not have the time at the moment. One of the protocols were to ask them a different time that we could call them back. This offers people an option to take it at a different time, therefore letting the potential respondents be more convenient when to take the survey.
Another solution would be offering the respondents more than one mode of survey. Again while interviewing potential respondents over telephone, a lot of people were reluctant to respond right away. When I suggested them there is an online version of the survey that I could send to their email, people said “Yes – that would work better for me!” In other words, giving people an option to choose a medium could be beneficial for the respondents.
Those are the examples of how we could increase benefits and decrease costs for the respondents.
Lastly, let’s talk about how to increase Trust andauthority
AC: I would define trust as the expectation that your actions will not be taken advantage of, and will be reciprocated. More trust means an increase in the stability of expectations. It means you are more likely to invest time and energy in one another with less worry that the efforts will be reciprocated. Similarly, if you trust an institution or organization, you may be more willing to do something for them. From this perspective, people are more likely to respond to a survey that seems to come from an entity that is more trustworthy, and less likely to respond if the survey seems like it is coming from an untrustworthy source.
So, when we say a surveyor is trying to establish trust, or at least emphasize trustworthiness, we can do it by highlighting elements of trustworthiness to the respondents.
SYL: What are some ways to establish this trust and communicate trust with the potential respondents?
AC: Trust can be communicated by symbols or words that convey legitimacy. It could be conveying that you come from a legitimate institution by prominently displaying that institution's logo or clearly stating whether the collected data is confidential and protected, ensuring your respondents that the collected data will be used only for research purposes and not misused. If you are a researcher who is affiliated with a university, I highly recommend that you include relevant logos or titles on the survey or that you include the dot EDU email in the invitation letter. If you are doing research for a company, similarly, you can include the company logo, emblem, or relevant titles in the survey.
SYL: One way to increase trust is to include a consent form if necessary. This not only signals that the survey is from a form institution, but also underlies the idea that there are some protocols that will be followed for this survey.
Another way is to include a summary of the survey information at the beginning. For instance, the surveyor can be upfront about the objectives of the survey, the overview of the questions that will be asked in the survey, or the expected duration of the survey.
Providing contact information and an easy way for respondents to reach you with questions or concerns can increase trust.
Lastly, emphasizing the idea of community or locality. For example, while I was collecting data for a project that studies a sample of Washington state residents, mentioning that I am affiliated with Washington State University seemed to be a plus, some people even said “Oh WSU, Go Cougs!”.
