Why?
We want to support decisions with data and insights to continually improve user experience, which will increase revenue and save money by preventing costly development errors. Here is a list of other benefits a data-driven culture supports:
- Organized collaboration & coordination
- Consistency through data democratization
- Progress tracking and transparency
- Identify potential opportunities
- Identify issues & solutions
- Solving for both the unknown & “known unknown”
Heatmaps & Recordings 
Annotations
How?
- Data Analytics & Heat Mapping
- User Survey
- User Testing
- A/B Testing
What?
Data Analytics & Heat Mapping
Analytics is good at collecting large samples of data to help identify issues and help form hypotheses to test and solve. Heat maps further improve data analytics with a visual representation of how they are interacting with web pages. Analytics helps with quantitative data.
User Survey
Data analytics & heat mapping are extremely powerful tools, but they do not fill in all the knowledge gaps. Since users are still human and not yet androids, we use User Testing to solve for a more complete understanding of the user experience through live testing and psychological approaches. User Testing helps with qualitative insights, something that analytics does not provide. Analytics can provide “how” & “what” users are doing. User Testing can provide the “why” from the “how” & “what”. Experiencing the user-experience also helps with the following:
Advantages:
- Easy to implement
- Cost-effective
- A variety of ways to administer (online, email, etc.)
- Can cover a large sample
- Can be standardized
Disadvantages:
- This type of data can run the risk of containing certain types of errors
- Surveyors may not feel empowered to provide answers with the best insight.
- Surveyors may not fully understand the question, or the question is unclear.
- Questions on the survey may give non-responses. The number of those who do choose to respond may be different than those who don’t, which can create a bias.
User Testing
Data analytics & heat mapping are extremely powerful tools, but they do not fill in all the knowledge gaps. Since users are still human and not yet androids, we use User Testing to solve for a more complete understanding of the user experience through live testing and psychological approaches. User Testing helps with qualitative insights, where analytics and A/B Testing does not provide. Analytics can provide “how” & “what” users are doing. User Testing can provide the “why” from the “how” & “what”. Experiencing the user-experience also helps with the following:
- Identify issues with complex flows that analytic analysis fails at
- Find out the “why” from quantitative data.
- Complement and illuminate other data points
- Ensure that the product meets expectations
- Validate the prototype
- Develop empathy and break down the “tunnel vision” effect
A/B Testing
A/B Testing is a powerful tool to optimize the user-experience. Using a hypothesis we use a controlled test (test A) and an altered test (test B). Test B is the implementation of a hypothesis that will test the impact of that change to Test A. A/B Testing on its own is quantitative in nature, but when paired with User Testing you also get qualitative insights that help give the “why” to the results shown through A/B Testing.
Leave a Reply