Q 601. Quality is one of the three parameters in the calculation of OEE. How is it sometimes mis-calculated to artificially inflate the OEE numbers? How can this be avoided and what is the right method to calculate Quality? Provide relevant examples.

 

Note for website visitors -

• This platform hosts two weekly questions, one on Tuesday and the other on Friday. 
• All previous questions can be found here: https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/. 
• To participate in the current question, please visit the forum homepage at https://www.benchmarksixsigma.com/forum/.
• The question will be open until Tuesday or Friday at 5 PM Indian Standard Time, depending on the launch day. 
• Responses will not be visible until they are reviewed, and only non-plagiarised answers with less than 5-10% plagiarism will be approved. 
• If you are unsure about plagiarism, please check your answer using a plagiarism checker tool such as https://smallseotools.com/plagiarism-checker/ before submitting. 
• All correct answers shall be published, and the top-rated answer will be displayed first. The author will receive an honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

• Some people seem to be using AI platforms to find forum answers. This is a risky approach as AI responses are error prone as our questions are application-oriented (they are never straightforward). Have a look at this funny example - https://www.benchmarksixsigma.com/forum/topic/39458-using-ai-to-respond-to-forum-questions/

• We also use an AI content detector at https://crossplag.com/ai-content-detector/. Only answers with less than 15-20% AI-generated content will be approved.

Overall Equipment Effectiveness (OEE) is a critical metric used in manufacturing to evaluate the effectiveness and efficiency  of manufacturing processes. It is calculated as the product of three main parameters: availability, performance and quality. In particular, quality is an integral part of OEE and  measures the ratio of good parts produced to the total number of parts produced. 
 Miscalculation of  OEE quality can occur in a number of ways, resulting in artificially inflated OEE numbers. Here are some common pitfalls and ways to avoid them, along with relevant examples. 
 
 Inadequate waste data: One common mistake is not accurately counting all defective or discarded products. If only a few defects are recorded, the quality indicator is imprecisely high.  
 Example: Imagine a production line that produces 1000 devices per day. During quality control, 100 devices are discovered to be defective and are removed. However, only 50 of these failures are recorded. If you use the errors recorded  in your quality calculation, you would mistakenly assume  95% quality (950 good widgets out of 1000), resulting in an artificially high OEE value. Avoidance: Ensure that all defects and waste are accurately recorded and included in the quality calculation. 
 Exclusion of reproduced parts: Some organizations may not consider reproduced parts in their quality calculations. If reworked parts are not counted as defects, this can increase the quality gauge. 
 Example In the production process, 50 devices out of 1000 are initially found to be defective and  sent for repair. After being rewritten, 40 of them are kept and are considered "good". If these reworked parts are not considered as defects, the quality calculation would erroneously give a  high quality percentage.  Avoidance: Include repaired parts in  total defects when calculating quality. 
 Overly generous tolerance thresholds: Setting too high tolerance thresholds for what is considered a "good" product can artificially inflate quality. If too many truly defective products  are considered accepted, this leads to a false indication of quality. 
 Example: If a manufacturing process allows for a 20 percent deviation from  ideal product specifications, many subproducts can still be classified as "good," resulting in a high quality score.  
 Avoidance: Define clear and realistic tolerance limits that closely match product quality standards. Accounting for process variability: Sometimes manufacturing process variances are not considered in the quality calculation. Failure to account for changes can lead to  overestimation of quality. 
 Example: In a continuous production process, if some products are produced above the standard quality and others below, but the quality indicator is calculated  only on average, this can artificially increase the quality percentage.  
 Avoidance: Consider  variability in product quality by calculating quality from individual product quality data or within control limits. 
 Calculate the quality exactly by following the following formula: 
 
 Quality = (total number of parts produced / total number of parts produced) × 100 
 
 Accurate data collection and a clear understanding of the defective part are critical to avoid OEE inflation  due to incorrect quality calculations. It is important to continuously monitor and improve both product quality and the accuracy of data used in OEE calculations to achieve significant improvements in production operations.

Quality Rate is the ratio of good product output out to total product output and it is one of the three parameters used in calculation of OEE along with Availability and Performance.

 

Quality losses includes Rework, Yield, Scrap. It not only involves loss of capacity but also materials, energy, and schedules. Quality loss must include both mistakes as well as variation.

 

There are two type of defects in Quality loss.

1.     Startup defects

2.     Production defects

 

Startup Defects:

Startup Defects can be defined as the defective parts produced from startup until normal or steady production is achieved. It can be after any equipment startup. However, mostly monitored after changeovers.

E.g. In Cold forging after job changeovers.

 

Production Defects:

 Production Defects can be defined as defective parts produced during running of stable (steady state) production. This includes parts that can be reworked, since OEE measures quality from a First Pass Yield perspective.

E.g. In Cold forging, Variation in Length of the Bolt due to tool damage.

 

Ideal condition of OEE based on Experience is

Availability: > 90%

Performance Efficiency: > 95%

Quality rate: >99%

 

Therefore, Ideal OEE = 0.90 x 0.95 x 0.99 = 0.85

 

 

How Quality rate is mis-calculated to inflate the OEE Numbers:

 

1.     Rework also considered as First time right and included in the calculation of Quality rate as shown below,

1.     Not including all the Startup rejects or showing less startup rejection inflates high Quality rate.

1.     Measuring OEE Plant wide, which average out the readings and inflates higher OEE. Target only critical machines by using pareto analysis.

E.g., The scenario shown below,

Machine-1: 98.32%

Machine-2: 99.73%

Machine-3: 87.92%

Machine-4: 99.64%

Machine-5: 99.72%

 

Average Quality rate of all the 5 Machines: 97.07%

 

If the plant considers the Average quality rate, then actual under lying issue on Machine 3 and Machine 1 will get unnoticed and focus will be missed for taking further corrective actions.

How can this be avoided and what is the right method to calculate Quality:

 

1.     Every category of machines needs to have a maintenance standard together with the frequency and the reason for the standard.

2.     Monitor the OEE along with SPC (Statistical process) guidelines and principles.

3.     Machine process capability to be studied and ensure it is under control.

4.     Motivating by involving the operators, team members to give them the opportunity, responsibility, Incentives, and recognition. Use small wins with feedback.

5.     The OEE Calculation gives equal importance to Quality and Availability. But usually Quality is far more important because rework results in greater load and more instability.

6.     A serious drawback is that no measure of variation is included in the standard OEE calculation.

Hence, Machine process capability to be studied and ensure it is under control.

 

For e.g. Two machines may have similar OEE’s of 80%, but very different variation.

Machine 1: Over all OEE is 80%, but variation 20-100%

Machine 2: Over all OEE is 80%, but variation 78-82%

Overall Equipment Effectiveness is an industry best practice metric which calculates the percentage of planned production time that is actually productive. An OEE score of 100% indicates Perfect Production, that is, producing only OK units, as quickly as possible, with zero downtime.

OEE = Availability * Performance * Quality

Where Availability considers all the reasons that disrupt planned production for a considerable period of time ( at least for a few minutes ) such that it needs to be tracked. Performance considers every reason as a result of which a process has less than maximum Throughput. Quality takes into considerations any reason as a result of which the product does not meets its expected Quality Standards including Rework. OEE’s paradigm resembles First Time Yield such that it defines a Good Part as one that successfully passes through the production the 1^st time without any rework. Quality is calculated as

Quality = Good Count / Total Count

 

OEE is measured using actual data from the ground operations and is capable of helping middle and top management take business decisions provided the data is correct and hence, calculations reflect the true picture. Individuals using OEE must always remember that there are several possible pitfalls that can lead to miscalculated OEE value and following are the ways in which Quality can contribute to mis-calculated OEE :

1.      Manual ways of data gathering may lead to insufficient data and poor quality results. Employees who are responsible to gather data may miss some readings, overlook important events during manufacturing or even go to the extent of deliberately manipulating data for various reasons. Such reasons can lead to the Quality score to look great whereas they are not so good in reality.

 

2.      If the quality of parts is checked visually by an employee then there is a very good chance of the following:

a.      He/ she may have impaired vision and hence misses to notice that some parts are Bad.

b.      He/she may be not be experienced or trained sufficiently to recognize all the Quality Failures that must be considered for calculating OEE.

c.       He/ she may manipulate the data to hide his/her or a senior’s mistakes.

d.      He/she may not have been educated about the motive behind the data gathering and hence, may take the activity of data gathering way too lightly.

 

3.      If the quality of parts are checked using a standard measuring tool, for example, a Vernier Calliper, then the following reasons may cause wrong data and hence mis-calculation of OEE:

a.      The employee fails to notice that the tool is not correctly adjusted

b.      The employee is not trained to use the tool correctly

 

4.      If the middle or top management of the organization are not coached sufficiently about the goal and benefits of gathering data for OEE calculation then there is a high probability that the he/she may decide to relax the First Time Right rule so that more parts are OK and the manager’s personal KPI look good.

 

It is the leadership’s responsibility to make sure that every step is taken to align the entire team with the purpose of tracking OEE such that the data is always correct and reflects the ground reality along with highlighting the opportunities for improvement.  

OEE, which is determined as the product of availability, performance rate, and quality rate, is a tool used to assess the real performance of a process.

Availability is the tool used to measure process performance by considering operating hours out of total available hours.

Performance Rate is the tool used to measure actual throughput against designed throughput.

Quality Rate is the tool that measures efficiency of products produced by considering good product output out of total product output.

Generally, benchmark of OEE for manufacturing sector is taken as 85% and all manufacturers try to achieve the best in class results

OEE calculations for a manufacturing process that yields finished goods at the needed throughput using the correct calculation methodology has been shown in table below. 

The following 5 scenarios show how quality is frequently measured inaccurately in order to unnaturally increase OEE for the process:
Scenario 1: Including reworked items in good products – In a few manufacturing processes, some of the rejected products are reworked to get a good product, and calculated those reworked items in good products.

Scenario 2: Finalize rejection % after audits that take a more time but consider rejection % into account beforehand.

Scenario 3: Reducing the customer specification limits to increase good product volume – Sometimes, process shift managers slightly reduce specification limit to get less rejection % compared to other shifts

Scenario 4: Considering rejection count only during stable production not during start ups – During production start-ups, rejection % is high as compared during stable production and so managers consider only rejection% while stable production

Scenario 5: Considering rejection % for the best production shift instead of monthly average or for the best operator instead of all operators

 

 

 

 

 

 

 

 

 

Very thoughtful answers from all respondents. Though all answers are correct, the best answer is from B Ravi Sankar for providing detailed examples of how quality can be miscalculated. Well done.

 

It is recommended to read the other answers as well.