Turn to Statistics to Unveil Value
The wise CPA expands beyond compliance and adds value by using tools such as net present value, marginal analysis, and cost and revenue analysis. In a data-driven environment, the CPA will grow his or her worth by updating their analytical skill set with tools from the school of probability and statistics.
by Bruce S. Weitzman, CPA, and Leslie Marlo, FCAS, MAAA Dec 3, 2018, 16:24 PM
The wise CPA expands beyond compliance and adds value by using tools such as net present value, marginal analysis, and cost and revenue analysis. In a data-driven environment, the CPA will grow his or her worth by updating their analytical skill set with tools from the school of probability and statistics.
Say you are a CPA working in industry, and the company’s marketing department wants to send out a mailing to 100,000 households. First, they tested the response rate of two different offers. They sent out 5,000 mailings of offer A and received 105 positive responses (a response rate of 2.1 percent). The cost of the mailing was $2.37 per resident. They also sent out a 3,500 mailing of offer B, and received 66 positive responses (a response rate of 1.9 percent). The cost of this mailing was $1.75 per resident.
The marketing department concludes that offer A had a better response rate, and now wants to roll out offer A to 100,000 households. It submits a funding request of $237,000.
The company president isn’t so sure, and turns to you for your analysis. How do you sort through this comparison? Was the increase in the response rate due to the difference in the offer or was it due to chance?
The Z-Test
You need to determine whether the two mailings are substantially equal (with any difference being random noise: that the mean response rate of offer A is equal to the mean of offer B – or the null hypothesis) or if there is a true statistical difference in the results.
One way to determine this is to use the common statistical technique of the z-test. A z-test is a hypothesis test. In this example, the z-statistic measures how many standard deviations above or below the true population mean the marketing data is. In this case, the test compares the calculated z-statistic to the z-value at 2.5 percent (a probability level of 95 percent since we are measuring deviation both above and below the true mean). If the calculated z-statistic is less than the z-value at 2.5 percent, then they would be substantially equivalent since it will show the result is not an outlier. But if the calculated z-statistic is greater than the z-value at 2.5 percent, then it is an outlier and the difference in results is significant.
The calculation of the z-statistic follows this formula:
z = (MeanA – MeanB) - (μA - μB)
[ (σA2/nA) + (σB2/nB) ]0.5
The mean and variance are calculated from the samples, and n is equal to the number of observations in each sample. We do not know the true μA or μB but since we are testing to see if there is no significance between the two (the null hypothesis) then μA = μB, so μA - μB equals 0.
Our calculated means and variances (σ = response rates) are 2.1 percent for offer A and 1.9 percent for offer B. (Note: here, the calculated means and variances are equal, but that is coincidental.)
Therefore, the numerator would be:
(0.021 - 0.019) = 0.002
The denominator would be:
[ (0.021/5000) + (0.019/3500) ]0.5 = 0.0031
So, the calculated z-statistic is:
0.002/0.0031 = 0.645
Now, compare this to the z-value at 2.5 percent. If you go to www.calculators.org/math/z-critical-value.php, the calculator returns a z-value of 1.96.
Since the calculated z-statistic of 0.645 is less than the z-value at 2.5 percent of 1.96, the mail campaigns are statistically similar.
Sensitivity Analysis
CPAs like to see how changes in inputs will change outputs; in a similar vein, we want to see how increasing or decreasing the sample size may impact our decision. (Spoiler alert: a large change in sample size could dramatically change the calculated conclusions). We can also vary the probability level. What if you test at 5.0 percent or 1.0 percent instead of 2.5 percent? And what would that mean?The adviser who is recommended and favored is the one who examines the numbers to reveal insight into operations. If the president can’t get that from his or her CPA, he or she will seek it elsewhere. Add the tools of statistics to your toolkit, and be that favored adviser.
Bruce S. Weitzman, CPA, is controller for Capitol Insurance Company in Lafayette Hill. He can be reached at bweitzman@capitolinsurance.com.
Leslie Marlo, FCAS, MAAA, is a consulting actuary for Madison Consulting Group in Newtown Square. She can be reached at leslie.marlo@madisoninc.com.
