Pricing Optimization using A/B Testing Part III: Running the A/B Test

1. What am I Testing?

The overall purpose of this project is to test whether a price to $69 will result in a greater revenue than the original price of $49. Keeping this in mind, we need to write our Null (default) and Alternative Hypothesis (what we are testing), based on this information.

2. Creating Test Group and Control Group Columns

Now that we have our revenue column with price shown for returning customers and 0 for one-time, we can create our two columns. The columns should look something like this:

3. What are the Column Variances?

Now that we have our columns ready to go, we can run the variances of each. This will let us know whether we need to run our A/B Test assuming equal or unequal variances. Generally speaking, if the variances are off by more than one then we can categorize them as unequal.

4. One or Two-Tailed Test?

Last, but certainly not least, we need to decide whether our test is one or two-tailed. Looking back at our alternative hypothesis, we can see that we are checking whether the revenue is greater than (inequality). This lets use know we will be running a one-tail test. Had I been looking to find out if there was a significant statistical difference between the two prices, then I would be looking at a two-tail test.

5. Running the A/B Test

We now have all the information we need to run the test. Because we do not know the population standard deviation (dataset is a sample), I know the variances are unequal, and I know it is a On-Tail, we know the kind of test to run.

Running the test with a 95% significance level, we need to look at the p-value and see if it is less than .05; if it is and the T-Stat is positive, then I will leave the p-value as is. This results in my rejecting the null hypothesis, leaving me with our alternative hypothesis as our conclusion.

CONCLUSION: After running the A/B Test, I have concluded that raising the price to $69 will result in more revenue for the company than if the price remained at $49.