Q 474. A famous quote says - "Customers don't feel Averages". Still a lot of our metrics focus on average performance - Average Handling time, Average Defects etc. If not averages, then what does the customer actually feel? How can we improve the performance metrics to reflect the actual feelings of the customers? Provide examples to support your answer.  

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time. Questions launched on Tuesdays are open till Friday and questions launched on Friday are open till Tuesday. 
• When you respond to this question, your answer will not be visible till it is reviewed. Only non-plagiarised (plagiarism below 5-10%) responses will be approved. If you have doubts about plagiarism, please check your answer with a plagiarism checker tool like https://smallseotools.com/plagiarism-checker/ before submitting. 
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term

Yes, I do agree about the averages which are not always true still most of time we used the averages which tell us about central tendency of data.

Average value is very sensitive for the outliers and will not represent actual picture of data set.

Other Metrics are as follows:

Metric	When to use	Example
Mean	No Outlier	Average Temp in Winter
Median	Outlier Present	Median Salary of IIM
Mode	Preferred Categorical Data	Type of Pizza Selling
Percentile	Comparison & Outlier Present	TAT for issuing of Insurance Policy
Standard Deviation	Minimize Variability, Measure of Spread	Selection of Stock in Market
Range	Measure of Spread	Heat Exchanger Temp range
DPMO	Checking Capability of Process/System for Discrete Data	No of Defect in the HRC
Sigma Level	Checking Capability of Process/System for Discrete/Continuous Data	Specification of Shaft 10+-.02 No of error in 1000 Documents
Cp/Cpk, Pp/Ppk	Capability performance with Specification Limit	Customer Complaints

 

How can we improve the performance metrics to reflect the actual feelings of the customers?

What customer wants, can be reached to us through Specification Limit or through Voice of Customer (VOC). Our primary role is to convert VOC into CTQ through using of different metrics used in the above table. Also performing time to time customer survey, feedback and market understanding to get closer what actually customer wants and how to meet or exceed their expectations.

We generally use Average or Median in our KPIs. The customer matches their experience with taste and most importantly how we made them feel. So if its a complaint then there is variation from the standard experience customer always received or perceived it as a standard experience. 

For example if someone in todays age says, "we dont capture complaint as an organization but a feedback." Then one immediately knows there is something fishy. Because customer perceived that for any issue there is always a customer complaint and turnaround time provided. 

This disconnect is variation in what is mentioned in SOP and what is experienced by the customer. 

Benchmark Six Sigma Expert View by Venugopal R


'Customers do not feel averages'....

 

In the case of B2B, customers would be organizations. Examples of expectations from such customers would be Product availability, Timely delivery, Zero DOA, Low response time, Higher customer preference etc. 

 

In the case of B2C, where customers would be and end-consumers, and the expectations could be different. Many end consumers may purchase a product or avail a service only once in a while. For them, a failure of the product or service is perceived as a 100% failure. 'Time to First Repair' denotes the period for which the consumer expects a failure free performance. Other expectations would include Quicker response time, User friendliness, After sales support and so on.

 

If we look at the various customer expectations narrated above, and we convert them as metric, most of them would need a one-sided specification. (For example, Delivery time 2 days Maximum). Averages may not be considered. We may define the 'defect' for each expectation as an instance when the expectation is not met. Thus most expectations can be measured as DPMO (Defects per million opportunities), Defective %.

 

Quite often, the degree of consumer expectations being met is assessed as part of the Pre-delivery audits by organizations.  This would address the Product Quality and performance expectations. For example, a consumer durable manufacturer will do a Finished Product Audit on a random sampling basis and report a score based on the findings. The score will be weighted based on the criticality and frequency of findings during the audit.

 

A service organization would measure a CSAT score based on customer feedback. Net Promoter Score (NPS) is one of the popular methods by which we obtain an estimate about the likelihood of customers recommending the company to others based on their experience.

 

If averages are not felt by customers, why do we have averages measured as part of various metrics in an organization?

 

In production processes, where we would like to monitor performance metrics based on samples, tracking sample averages would help to apply SPC tools such as 'control charts'. To express statistically, the principles of Normal distribution work better on sample averages.

 

Averages do not mean much unless we examine the associated variability as well. Variability is derived from sample results and expressed as the 'control limits'. Such tools help us to monitor the stability (consistency of performance) of a process.

 

Assuring stability is a pre-requisite to assess the capability of a process. Averages would not be the final way of expressing performance. Capabilities are expressed as Sigma levels, Capability stats or in terms of Parts Per Million. 

                   The reporting metrics from many of the organizations are not descriptive in nature. We may have noticed that the most of those metrics are based on averages or sometimes median value. The average may be misleading because of uneven spread in the results or uncertainty about whether patients had an important improvement.

Some of the problems with averages are as below:

1. The mean does not show the spread of data

For example, if we want to look at the height of the students in a class, the average height is the same in classes A, B and C below but the individual students height are different.

2. The spread of data may not be even

In the above example, the data are spread evenly above and below the average. However, this is not always the case. One person’s data can have a large effect on the average of the whole group.

 

For example, we ask patients to rate their pain on a 0-100 scale and the results are:

Group A: For 99 people, pain reduced by 10 points; for 1 person, pain reduced by 50 points.

Group B: For 99 people, pain reduced by 10 points; for 1 person, pain increased by 50 points.

 

The average is a 10.4 point improvement in group A and a 9.4 improvement in group B. Based on the average, the treatment in group A looks better but, in reality, for 99% of people, both treatments are equally effective.

 

In this case, the mean is misleading since two people with extreme results are pulling the average up or down, so it does not represent the population as a whole. This is what will happen when there is a ‘skewed’ distribution because of extreme outliers.

 

In a similar way, two groups can have the same mean result, even though most participants do better on treatment A but there is just one patient who does really well on treatment B, as in the example below. 

 

 3. The mean does not show how many people had a significant improvement

Another way in which the mean can be misleading is that it does not give us information about how a patient’s improvement relates to whether they had an significant change. Sometimes this change is referred to as the minimum significant difference.

From the example below, the average of group C seems better than group D since they improved by 5 points on average compared to 4 points in group D.

However, if a minimum 7 point change is needed to be significant in the patient, no patient in group C experienced a significant change but 2 people in group D did. In terms of the proportion of patients who have a significant improvement, group D is better than group C.

 

The average in this example is misleading as it does not show which patients have an significant change. When the average is higher in one group, it does not mean that the treatment was better for the other group, since most patients do not experience the ‘average effect’.

 

Averages are meant to be a measurement across a diverse group of samples. The main purpose of averages is to measure changes over time in the same sample group.

 

Three common errors that can happen by following only the averages :

 

1)     Any data set will have outliers. These outliers tend to skew the average of the dataset to “pull” it in their direction.

2)     Many people tend to think of “average” as “typical”,  the problem with that assumption is that, there are many, exceptions to “typical.” For example, the average obesity rate in the United States, which is often cited as “about 50%.” If one researches the question, they will discover average obesity rates vary widely based on factors such as age, level of educational, profession, genetics, etc. As such, everyone is categorized differently by these factors resulting in many different average obesity rates depending on which factors describe those being measured.

3)     Averages of often wrongly used is in characterizing individual scenarios. It is statistical err to apply the average of a group of data points to a single point and expect it to be true. Even assuming data as normally distributed, the probability that any one data point will be the same as the average is 50% — the same as a random guess.

 

Solution:

 

The problem is “Customers do not feel the average — they feel the variation”. The in-depth view of the business is based on average based measures of our performance. Customers don’t weigh us on averages, they feel the variance in each performance, service, each product, each interaction we have on the phone, each correspondence we have through email or a letter, and every other process that touches the customer in one way or other — online or offline.

Customers expect a consistent, predictable business processes that deliver world-class levels of quality. They feel the difference, not the average.

 

A few examples

For qualitative measures, the feel, Visual, taste, smell et., of a product, we do not think in terms of an “average”. Instead, the customers weigh them by relative measures and the difference between them.

For Example ,

1)        “the coffee from Cloudbucks tasted better than that from Coffeenight, (or) the perfume Wildrock smells good compared to the Axel perfume.” The customers feel the difference, not the average. That’s the reason we define Quality based on Customer’s perception.

 

For quantitative measures, variation and not the average, that works well. The customers tend to perceive the variation from the previous event.

For Example,

1)        ”The last time I ordered from EKart, I received that package in 3 business days; but, on other times I receive my order usually after 6 days on standard shipping”

Ekart may calculate and report their average lead time as 3 days which may not account the instances when the lead time were more than 3 days (dissatisfied customer) and those events when it was less than 3 days (happy customer).

 

2)        ”Yesterday when I went to PizzaHouse during lunch hour, I was in and out of there within 30 minutes, but on other days I’m there for my whole lunch hour”

PizzaHouse may advertise as their average service time as 30min, which may not account those events when it went >30 min (dissatisfied customer) & those events when it was <30 min (happy customer).

 

In general, the organizations that report averages, are not concerned about both happy customer and the Dissatisfied customer. They only race towards the average customer satisfaction.

 

“On average, our customers are 90% satisfied”- a popular tagline by some businesses and organizations.

 

If I have a bad experience with a vendor, I don’t cognitively step back and think that I have been a customer for this vendor during last 10 instances, and I had only one bad experience, so far, so on average I’m 90% satisfied.

 

Customers experience good or bad, as it is and their future behavior, purchases and recommendations are shaped by these experiences. To further add to it, cognitive biases such as negative and recency bias often interject and give disproportionate weight to these positive or negative experiences, reinforcing them and the resulting behavior that follows it.

 

In probability theory and statistics, the coefficient of variation (CV), also known as relative standard deviation (RSD),[citation needed] is a standardized measure of dispersion of a probability distribution or frequency distribution. It is often expressed as a percentage, and is defined as the ratio of the standard deviation to the mean (or its absolute value. The CV or RSD is widely used in analytical chemistry to express the precision and repeatability of an assay. It is also commonly used in fields such as engineering or physics when doing quality assurance studies and ANOVA gauge R&R. In addition, CV is utilized by economists and investors in economic models.

 

It shows the extent of variability in relation to the mean of the population. The coefficient of variation should be computed only for data measured on scales that have a meaningful zero (ratio scale) and hence allow relative comparison of two measurements (i.e., division of one measurement by the other). The coefficient of variation may not have any meaning for data on an interval scale.

 

Hence Standard Deviation, Variance and Co-efficient of Variation are some of the alternatives to the averages, while presenting the performance to the Customers.

The intent of the question was to highlight the fact that customers do not feel the average or the mean. They always feel the variation or difference in the service.

 

While there were many responses to the question, few of them could not be approved as they were not relevant to the context.

 

The best answer has been provided by P Balakumaaran.

 

Also review the answer provided by Mr Venugopal R, Benchmark Six Sigma's in-house expert.