What does the "Monthly Revenue Impact" figure mean?

The Monthly Revenue Difference figure is the predicted monthly revenue difference between variants for each of your products. This is found empirically using your test results. The Monthly Revenue Difference helps you gauge your ROI and to see how much Splitly is improving your business. It is calculated by:

  1. Finding the revenue difference between the best and worst variant for each of your tests
  2. Tests that have less than 80% average daily revenue statistical significance are skipped (this is not shown in test results)
  3. Tests that have at least one variant with less than 5 rotations are skipped
  4. Only one test per product is counted, which is the test with the biggest revenue difference

I have run a few tests, but my Monthly Revenue Difference is still showing $0

This happens when you either have insignificant results or you've aborted the tests too soon. There's a few things you should try:

  • Create tests with bigger variant differences, such as a big price change or images that are completely different.
  • Allow tests to run the full course, we'll end tests automatically when we find a statistically significant winner.

What if the original wins? Are they still included in the Monthly Revenue Difference figure?

If your original wins, the revenue difference is still included in the Monthly Revenue Difference figure for a few reasons:

  1. Users have the ability to edit the original variant before running a test, in which case it is no longer the "original" but is a standalone variant, just like the others.
  2. Finding out the original is the best performing variant is still beneficial to your business. You will know not to make any more changes to your variant, because doing so could cause harm to your revenue. In this sense, it is cost saving.

Is this figure the additional amount of revenue I should expect to see in my seller account?

The revenue difference is just that - the difference in revenue between variants for each of your products/tests. If each one of your tests lead to a more optimised listing then you should expect to see extra revenue by this margin in your seller account. This number should be used as a guide to see how well Splitly is benefiting your business.

Still need help? Contact Us Contact Us