Why is there missing data in my test results?
When we pull your results data from Amazon to Splitly using our Chrome Extension, we often find that data isn't yet finalised until a few days has passed. This means we cannot trust data until a few days has gone by. In the past we've seen customers have >100 conversions with 0 sessions, which is impossible!
For this reason, we do not include test data from the past 2-3 days in results. We don't even bother collecting and displaying it, because this will likely cause confusion among users who see numbers jumping around everywhere. In the rotation log you will see some missing data, you will see something like this:
During the first few days of a test it is very important to be patient and wait for the results to come in. We understand you might be eager to see some results as quickly as possible, but this just isn't possible when A/B testing on Amazon. We suggest you wait about a week or so before you look back and check results.
For more information on how long an A/B test lasts for, see here.