Goal
We want to increase the number of contributions on our platform whether it’s reviews, photos, or ratings.
Hypothesis
It might be hard for some people to judge restaurants and places on a scale of 1-5. Simply asking whether you recommend the place or not might be easier to use. A more lightweight way to start contributing.
Methodology
We did A/B testing on the restaurant info screen between A and B versions. We keep it as simple as possible only testing the conversion of the entrance to the review flow.
We keep the popup after the users click on the main interaction the same to launch the experiment faster. This should have a small effect on the experiment (only affecting repeating contributors). But we mainly focus on testing whether designs A or B convince users to start a review better.
Metrics
Here are the main metrics we used to measure this experiment.
- Trigger conversions : Compare the conversion of the main engagement between A and B (between rating 1-5 and answering whether the user recommends the place). This tells us which UI attracts the user to engage better.
- Flow completion conversion : In contrast with trigger conversion this is the conversion rate of the whole flow.
- Completed reviews : The simpler – better experience should have more completed reviews.
- Repeating contributors : if users have a good experience they are more likely to return and contributes mroe
Result & summary
We hypothesized for a 10x increase in conversion because the interaction seems simpler. But we saw no real increase in conversion. The numbers are comparable to the previous version.
We decided to revert back to the previous version with a 1-5 scoring scale because the number of data points that we receive is the same but having a 1-5 scale tells so much more. It gives us a better idea of how much a user likes or hates a place.