The reporting metrics from many of the organizations are not descriptive in nature. We may have noticed that the most of those metrics are based on averages or sometimes median value. The average may be misleading because of uneven spread in the results or uncertainty about whether patients had an important improvement.
Some of the problems with averages are as below:
1. The mean does not show the spread of data
For example, if we want to look at the height of the students in a class, the average height is the same in classes A, B and C below but the individual students height are different.
2. The spread of data may not be even
In the above example, the data are spread evenly above and below the average. However, this is not always the case. One person’s data can have a large effect on the average of the whole group.
For example, we ask patients to rate their pain on a 0-100 scale and the results are:
Group A: For 99 people, pain reduced by 10 points; for 1 person, pain reduced by 50 points.
Group B: For 99 people, pain reduced by 10 points; for 1 person, pain increased by 50 points.
The average is a 10.4 point improvement in group A and a 9.4 improvement in group B. Based on the average, the treatment in group A looks better but, in reality, for 99% of people, both treatments are equally effective.
In this case, the mean is misleading since two people with extreme results are pulling the average up or down, so it does not represent the population as a whole. This is what will happen when there is a ‘skewed’ distribution because of extreme outliers.
In a similar way, two groups can have the same mean result, even though most participants do better on treatment A but there is just one patient who does really well on treatment B, as in the example below.
3. The mean does not show how many people had a significant improvement
Another way in which the mean can be misleading is that it does not give us information about how a patient’s improvement relates to whether they had an significant change. Sometimes this change is referred to as the minimum significant difference.
From the example below, the average of group C seems better than group D since they improved by 5 points on average compared to 4 points in group D.
However, if a minimum 7 point change is needed to be significant in the patient, no patient in group C experienced a significant change but 2 people in group D did. In terms of the proportion of patients who have a significant improvement, group D is better than group C.
The average in this example is misleading as it does not show which patients have an significant change. When the average is higher in one group, it does not mean that the treatment was better for the other group, since most patients do not experience the ‘average effect’.
Averages are meant to be a measurement across a diverse group of samples. The main purpose of averages is to measure changes over time in the same sample group.
Three common errors that can happen by following only the averages :
1) Any data set will have outliers. These outliers tend to skew the average of the dataset to “pull” it in their direction.
2) Many people tend to think of “average” as “typical”, the problem with that assumption is that, there are many, exceptions to “typical.” For example, the average obesity rate in the United States, which is often cited as “about 50%.” If one researches the question, they will discover average obesity rates vary widely based on factors such as age, level of educational, profession, genetics, etc. As such, everyone is categorized differently by these factors resulting in many different average obesity rates depending on which factors describe those being measured.
3) Averages of often wrongly used is in characterizing individual scenarios. It is statistical err to apply the average of a group of data points to a single point and expect it to be true. Even assuming data as normally distributed, the probability that any one data point will be the same as the average is 50% — the same as a random guess.
Solution:
The problem is “Customers do not feel the average — they feel the variation”. The in-depth view of the business is based on average based measures of our performance. Customers don’t weigh us on averages, they feel the variance in each performance, service, each product, each interaction we have on the phone, each correspondence we have through email or a letter, and every other process that touches the customer in one way or other — online or offline.
Customers expect a consistent, predictable business processes that deliver world-class levels of quality. They feel the difference, not the average.
A few examples
For qualitative measures, the feel, Visual, taste, smell et., of a product, we do not think in terms of an “average”. Instead, the customers weigh them by relative measures and the difference between them.
For Example ,
1) “the coffee from Cloudbucks tasted better than that from Coffeenight, (or) the perfume Wildrock smells good compared to the Axel perfume.” The customers feel the difference, not the average. That’s the reason we define Quality based on Customer’s perception.
For quantitative measures, variation and not the average, that works well. The customers tend to perceive the variation from the previous event.
For Example,
1) ”The last time I ordered from EKart, I received that package in 3 business days; but, on other times I receive my order usually after 6 days on standard shipping”
Ekart may calculate and report their average lead time as 3 days which may not account the instances when the lead time were more than 3 days (dissatisfied customer) and those events when it was less than 3 days (happy customer).
2) ”Yesterday when I went to PizzaHouse during lunch hour, I was in and out of there within 30 minutes, but on other days I’m there for my whole lunch hour”
PizzaHouse may advertise as their average service time as 30min, which may not account those events when it went >30 min (dissatisfied customer) & those events when it was <30 min (happy customer).
In general, the organizations that report averages, are not concerned about both happy customer and the Dissatisfied customer. They only race towards the average customer satisfaction.
“On average, our customers are 90% satisfied”- a popular tagline by some businesses and organizations.
If I have a bad experience with a vendor, I don’t cognitively step back and think that I have been a customer for this vendor during last 10 instances, and I had only one bad experience, so far, so on average I’m 90% satisfied.
Customers experience good or bad, as it is and their future behavior, purchases and recommendations are shaped by these experiences. To further add to it, cognitive biases such as negative and recency bias often interject and give disproportionate weight to these positive or negative experiences, reinforcing them and the resulting behavior that follows it.
In probability theory and statistics, the coefficient of variation (CV), also known as relative standard deviation (RSD),[citation needed] is a standardized measure of dispersion of a probability distribution or frequency distribution. It is often expressed as a percentage, and is defined as the ratio of the standard deviation to the mean (or its absolute value. The CV or RSD is widely used in analytical chemistry to express the precision and repeatability of an assay. It is also commonly used in fields such as engineering or physics when doing quality assurance studies and ANOVA gauge R&R. In addition, CV is utilized by economists and investors in economic models.
It shows the extent of variability in relation to the mean of the population. The coefficient of variation should be computed only for data measured on scales that have a meaningful zero (ratio scale) and hence allow relative comparison of two measurements (i.e., division of one measurement by the other). The coefficient of variation may not have any meaning for data on an interval scale.
Hence Standard Deviation, Variance and Co-efficient of Variation are some of the alternatives to the averages, while presenting the performance to the Customers.