Data analytics explained | How to effectively analyze and segment post-experiment data
Insights BI Software
You might wonder what is the purpose of running experiments, not to mention spending so much of your time on analyzing and segmenting post-experiment data. You’re probably always in a hurry to reach those grand KPIs, and this would require additional resources. Yes, we know, we heard it so many times.
But, did you know that running tests can help you increase conversion rates more than 100%?
Running tests and experiments will allow you to gather actionable and comparable data that will further allow you to make data-driven decisions that will grow your business at an exponential rate.
But, as always, there is a catch.
If you do not segment your data and compare various reports on each variation – you are not going to get far. Therefore, we decided to show you a real-life example of how two variations (plus the controlled one) performed and how we analyzed it.
Post-Experiment Data Report Segmented by Device Type
In this chart, we can see only one segmentation.
We broke down the conversion rates by device type. And we can see right away that the conversion rates between mobile, desktop and tablet were different between variations:
Mobile: the winner was the controlled variation.
Desktop: the winner was variant number two.
Tablet: the winner was variant number one.
These post-experiment data analysis show us that the experience we shared with users on the mobile device wasn’t optimal. There was something about the mobile experience in the controlled variation that was better for user experience and helped conversion rates.
However, our variation helped with the user experience on the desktop. That insight allows us to make the best of the second variation and explore how can we take it even further.
On the other hand, it tells us that there is something very wrong with variant number two on the mobile device. So, we should keep digging until we find out what did we do that hurt the conversion rates on the mobile device. Also, we should find out what did the controlled variation do better than us?
The findings will allow us to learn and optimize accordingly and explore new possibilities for running experiments.
Is Segmenting Post-Experiment Data Really That Important?
Not only is it important, our dear reader, but it is also necessary.
Data segmentation will save you from falling in the trap of averages. Click To Tweet
If you read the averages of experiment variants, you would get a completely different picture! You could even think that variants didn’t significantly improve the conversion rates. And this couldn’t be farther from the truth!
Look at the desktop. The conversion rates were increased by more than 100%, and the same goes for tablets. And the reason why conversion rates weren’t significantly increased on mobile devices is that it attracted a lot of traffic, and that’s what brought the conversion rates down.
You see, the averages will never show you the full picture.
Here’s what we would do in this example. We would:
Keep the variant number two for desktop,
Variant number one for tablet, and
We would apply different variations for different device types.
Additionally, we could segment these variations by different categories, overlay, breakdown, and identify what user groups and user segments do and don’t resonate and come up with ideas to explore and push things further.
It is incredibly important that when you do a test, experiment, or try something new, don’t just observe the data and accept it without asking questions. You need to create a data extraction plan, dig into it and explore what does it mean for different segments of your audience. Trust us when we tell you that you can find all of the answers in post-experiment data. You just need to know where to look.
If you need some help finding answers, we’ve got your back.
CONTACT US AND WE’LL HELP YOU FIND ANSWERS
Head of Optimization & Business Intelligence @ ConversionAdvocates
Data Analytics Data Segmentation
How To Grow eCommerce Revenue With Data
Market Basket Analysis 101: A Tutorial on Skyrocketing Revenue
ConversionAdvocates Review | From $0 to $100K per Month | The Paint People
Accelerate growth with clear and actionable insights into visitor behaviour
Supercharge your conversion optimization program with new data-backed test ideas
Increase your revenue month-over-month with data-driven experiments
Stop wasting money and identify website conversion leaks you don’t know exist
Book a Discovery Call & Get a FREE Analysis
What is your primary conversion objective?*
Online Sales (eCommerce)
How much revenue does your business generate from conversions on average per MONTH?*
© Conversion Advocates Inc.
We’ll Help You Find New Revenue Opportunities Hidden In Your Analytics
See how we can increase your revenue and reduce costs with data-driven experiments and our IIEA growth framework.
"47% increase in revenue per month"
Jeremy Jaynes Car Computer Exchange
Free Analysis Popup
What are you looking to achieve by joining the ConversionAccelerator Program?*
I want to learn how to do CRO myself
I want to train my team on CRO