SurveyMonkey™ is a great service. It's cheap, it's easy to use, it has the right collection of features and most importantly, it serves a purpose. But what about the analysis of data collected with it? In many cases for real analysis you cannot use the tool itself and have to use other tools. In the next two blog posts, I’ll try to examine this issue.
About Looking Cool
Whenever I need to collect my friends' feedback about a certain issue and reach a consensus, instead of just sending an email and receiving a dozen replies, I quickly setup a short survey and publish it among them.
It's easy, it's cheep and let’s face it…it makes me look cooler.
But there is a but ….
When it comes to viewing and understanding the feedback gathered, things get less cheerful and more complex.
Yes, I can see all the questions and answers in a table layout.
And Yes, I can create charts.
And yes again, I can filter and crosstab the results.
So if everything is so great, why do I have problems understanding what the data is really saying?
Wandering around SurveyMonkey™ 'Analyze Results' section made me realize that there are two main weaknesses in their offering having to do with Data Visualization and The Big Picture.
About Data Visualization
Data Visualization is a science. The ability to summarize hundreds or thousands of pieces of information into a single meaningful picture is far from trivial, and should be carefully thought out.
In SurveyMonkey™, charts do not serve as means for understanding the data; they are used merely for decoration, and are referred to as such. You can easily create charts for a question to decorate your report, but you get to see them only in a modal window, one chart at a time, outside of the survey's context. And if you want to download them you'll get only a still image, which is a dead-end for further analysis. Visualizing data outside of the survey context in not just worthless it can also be misleading.
Let’s look at an example of what happens when visualizing data in a wrong way.
Take the simple, commonly used pie chart.
The circle, or the pie, represents a whole, therefore the slices comprising it should sum up into a significant whole.
Here is an example:
*Q. How many times do you watch TV per week?
It actually makes no sense to represent four TV watching habits as slices of a pie chart, obviously there are many more than the four mentioned in the question. The 'whole' in this case, is meaningless. Now, look at the green and yellow slices? Can you easily tell which one is larger? A bar chart would be a more appropriate display to compare performances of the selected TV watching habits.
What Is The Big Picture?
The other weakness I’ve noticed in SurveyMonkey™ is its lack of ability to 'see' the Big Picture.
Survey data has a huge advantage over other types of data – it is structured.
By 'structured', I mean that we not only know how many people chose a specific answer. The data can also tell how many did not. Furthermore, there are relations between the different questions. For example, the data can tell how many males in their 30s prefer chocolate ice-cream and what they prefer to do in their free time.
There is a lot of uncovered information under the bars and numbers reflecting individual answer choices. To uncover this information you need strong filtering and analysis capabilities. With SurveyMonkey™, we cannot access this hidden gem.
In my next blog post I'll discuss how using online analytical tools will fill in those gaps and complement SurveyMonkey’s great data collection capabilities. I'll demonstrate how working with SurveyMonkey™ data on groketeer can be effortless and streamlined and how to fully understand what the numbers are saying.