Mobile A/B Testing Blog

How Appiterate’s mobile app A/B testing helped Reviews42 improve monetization by 28%

31 Oct 2013

Appiterate is helping many gaming, e-commerce and media app companies to A/B test and iterate. It helped a leading VC backed company, Reviews42 to achieve a 28% increase in conversions just by a few simple modifications, which took less than 10 mins to create.

 

The brief:

Reviews42 is a leading Indian price comparison and shopping assistance app. It lets users compare prices of multiple products across categories from online stores.
 
Something like this:
 

Price comparison

Photo credit | New Yorker

Reviews42 wanted to optimize their mobile app on two fronts – increase referrals and increase user engagement. So, we tested a couple of CTAs (Call to action) on their app.

Now, referrals to shopping sites form a significant chunk of revenue for price comparison apps. This revenue is directly related to the number of people that are visiting and buying on retailers’ sites through their app. Whether a user would tap on specific CTA buttons, and eventually visit retailers, is considerably dependent on the shape/size/formatting/location/text of the button.

 
 
 

Goals:

Reviews42 intended to increase conversion rates on the following:

  1. Review button of a product page – increases their user engagement metric.
  2. Price link on a product page that takes a user to a retailer’s site – increases revenue in the form of referral fees.
Hypotheses:

We tested a couple of parameters:

  1. Whether having a short or a long text on a ‘review’ button drives better engagement metrics
  2. Whether having a small visual element beside the price has an increase in referrals. A small arrow was the visual element in this case.

Appiterate created a test for Reviews42 that was deployed on all product pages.

 

The test:
Control Version Experiment version
A review button with longer text on the product page (‘Add Review’) A review button with shorter text (‘Rate It!’)
No visual element beside the price on the product page A small arrow beside the price to goad users to tap on it and visit the retailer’s site
 Reviews42 test appiterate
50% distribution 50% distribution

 

Appiterate got progressively deployed on 25,000+ devices within 60 days, as users downloaded new versions of the Reviews42 app, with Appiterate’s SDK in it.

 

Impact:

So, the new version of the price link (with a visual element) resulted in an increase of 27.9% in a key metric which has a direct revenue impact!
 
The control version performed better in case of adding reviews. The longer, but more explanatory text (‘Add Review’) on the review button is better at driving engagement by 7.7%.
 
So, the following performed significantly better over its variation:

 

  • 1. The review button with longer text (‘Add Review’)

Add review appiterate reviews42

  • 2. A visual element beside the price

Arrow appiterate reviews42

 

 

 

 

 

Rather than acting on opinions and conjectures, Reviews42 assessed the variations on actual users. It got statistically significant differences between variations and thus rolled out the winning variations to all users, potentially multiplying their revenues in the long-term.

Tanuj Mendiratta

Co-founder & CEO
With rich experience of his previous start-ups, Tanuj is leading the growth at Appiterate. He keeps a keen watch on the latest trends and makes sure that we do not miss out on having fun between all the hard work.