Survey research - Ways to clean survey data before analysis

 

A survey is a research method used for collecting data from a predefined group of respondents to gain information and insights into various topics of interest. The process involves asking people for information through a questionnaire, which can be either online or offline. According to Interaction Design Foundation “Surveys and Questionnaires – When you ask for many users’ opinions, you will gain massive amounts of information. Keep in mind that you’ll have data about what users say they do, as opposed to insights into what they do. You can get more reliable results if you incentivize your participants well and use the right format.”

Surveys are an important user research method, in which userinformation at bigger counts gets considered. Before stating with data analysis it is important to validate the data quality, else it may deteriorate the inferences and can lead to wrong insights. The data quality can be improved by doing detail data cleaning. 

DISCLOSURE: This post may contain affiliate links, meaning when you click the links and make a purchase, we receive a commission.

Survey data cleaning involves identifying and removing responses from individuals who either don’t match your target audience criteria or didn’t answer your questions thoughtfully. There are several ways your survey data can have false data points which needs to be removed before beginning the analysis. Below are some of the ways you can clear false data from gathered survey data and improve the quality of your data. 

How to conduct data cleaning in survey

Remove partially completed responses - 

You might have noticed that sometimes few participants did not answer all the survey questions due to any reason (ex. Technical issue, fatigue, non-interest etc.) This partial filled data can create some noise in pure data analysis. It is advised to remove the partial completed responses from the overall data before starting analysis.   

Remove straightliners - 

Researchers needs to be cautious about straightline respondents and responses must be removed before analysis. Straightlining is when respondents choose similar answer option frequently (such as first/last option etc.). There might be higher possibility that the respondent has not responded the answers honestly. 

Remove Speed responses - 

Imagine the average time to complete your survey is 4:30 minutes and you have found the quickest response by one of the participant was 20 seconds. There is higher chances that quickest respondent might have completed survey just to complete. This kind of responses are called speed responses. There is no certain rule to identify such speed responses but some statistics calculation can help you such as: take average completion of time, check min and max range of time, set your own rule of remove responses which are x% less than the average. 

Remove outliers - 

Sometimes you have encountered responses falling under unrealistic range for example “In which sport you see yourself as pro” and one respondent selected all the sports from option. This kind of responses are called outliers. Outliers can not be said as fake response but also should be removed before starting analysis because it may impact some calculation ex. Range, min-max values, st. deviation, average etc. 

Remove fake or manipulated answers - 

This becomes tricky sometimes but researchers should always be beware of fake or manipulated responses from participants. There are many ways to check fake responses such as, using open ended questions - check is any response contain unreadable or meaningless response like ‘fgfgfh’ type text etc. Another way is to check fake response by having multiple questions to validate response like one question “Do you play outdoor games? (With responses I don’t like playing outdoor game)”, after few questions another question can be added in survey like “which outdoor question do you like the most”. If some participants responding “I don’t like playing outdoor games” and later the same participants responding like they like football. These participants might have a high probability in faking the responses and can be eliminated from master data. 

Read more:

https://www.interaction-design.org/literature/article/useful-survey-questions-for-user-feedback-surveys?ep=uxness

https://www.uxness.in/2016/06/10-survey-tools-for-ux-designers.html 

 

What is triangulation in User Research (UX research)?

What is Triangulation 

One of the biggest challenge or doubt for any research project is to bring confidence levels in research findings & insights. To increase confidence in to insights researchers often use a technique called ‘Triangulation’, which is nothing but to gather the insights from multiple methods or ways for the same research question(s). According to Wikipedia - triangulation refers to the application and combination of several research methods in the study of the same phenomenon. By combining multiple observers, theories, methods, and empirical materials, researchers hope to overcome the weakness or intrinsic biases and the problems that come from single method, single-observer, and single-theory studies. 

The term ‘triangulation’ originates in the field of navigation where a location is determined by using the angles from two known points. Triangulation can be used in both qualitative studies & quantitative studies and popularily used in Sociology. The purpose of triangulation in qualitative research is to increase the credibility and validity of the results.

Triangulation is an analysis technique used in multi-method research designs. Many research projects utilize more than one data collection method, leading to the development of different datasets. Datasets might be those collected from a quantitative survey or participant observation, for example. The results from the datasets are analyzed independently, but they also need to be compared to each other in some way. How they are compared depends on the methodological framework used. Triangulation is one technique to combine datasets, and three different kinds of triangulation can be distinguished: convergence, complementarity, and divergence or dissonance.  (International Encyclopedia of Human Geography, 2009)

DISCLOSURE: This post may contain affiliate links, meaning when you click the links and make a purchase, we receive a commission.

Types of triangulation 

1. Data triangulation: involves time, space, and persons. 

2. Investigator triangulation: involves multiple researchers in an investigation. 

3. Theory triangulation: involves using more than one theoretical scheme in the interpretation of the phenomenon. 

4. Methodological triangulation: involves using more than one method to gather data, such as interviews, observations, questionnaires, and documents.

Triangulation in UX research (user research) 

Triangulation involves using several data collection techniques in order to validate findings. For example, instead of performing solely qualitative interviews for data collection, add focus group sessions, collection of documents, and/or collection of personal journal entries from participants. These additions bolster the quality of your findings. (Cheryl Patton, PhD). According to NNgroup’s article on Triangulation in UX research - Triangulation means looking at a question from a different point of view, which lets you see part of the answer that wasn’t previously apparent. You’ve probably done this many times in your everyday life — when you asked someone else’s opinion about a situation, hoping that a different point of view will reveal something that wasn’t obvious to you. In UX research, to triangulate the findings one can combine the research methods and support the findings by doing comparison analysis. Also one can relook at the historic (previous research) and triangulate with present findings using comparison. Having said that for referencing the previous or historic research, consider the insights validity in to account.

References

https://en.wikipedia.org/wiki/Triangulation_(social_science) 

https://ebn.bmj.com/content/16/4/98 

https://www.sciencedirect.com/topics/social-sciences/triangulation 

https://www.nngroup.com/articles/triangulation-better-research-results-using-multiple-ux-methods/