Severity Assessment in Risk Analysis

[Vishwadeep Khatri 265]

Q. 188  Explain the common challenges in Severity Assessment as a part of PFMEA (Process Failure Mode and Effects Analysis). Also mention how these challenges can be addressed.

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening as per Indian Standard Time.  
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

 

[Prasanna Pokhrel 3]

Some of the common challenges in conducting severity rating in PFMEA are listed below along with some thoughts on how these could be mitigated. 

    1. Understanding the ordinal rating scale: Interpretation of ordinal rating scale may be different from the interpretation of ratio scales and there might be the risk of drawing incorrect assumption. For example if the rating scale gives 3 likely and 6 very likely , in the rating, the impact may be significantly different from that of 2 & 4 and not exactly double the rate (as is the ratio in both the cases). The range, however, may be considered as such if the rating scale is not well explained. The solution is to have detailed discussion on the assessment mechanism including the rating scale.

    2. Different rating scale for different industry: The severity rating scale may have very different implications for example, the rating scale used for healthcare industry will have very different scaling parameters and levels vs insurance industry or automobile industry. This challenge can be addressed by working with the actual team members & the respective functional leaders to design a rating scale which is relevant to the organization.

    3. Difference in interpretations: Even in case of the same rating scale being provided, there could be difference in interpretation of the severity & impact of a possible risk based upon the personal experiences of the person conducting the assessment. The solution in such a situation is to have calibration meetings to ensure that every one is on the same page. 

    4. Cognitive Biases: The challenge in using rating scales and not statistical data in arriving at severity rating is that it may be subject to cognitive biases as follows: 
        a. Only takes in to account "known -unknowns"  & does not plan and design suitable response mechanisms for black swan events & "Unknown-Unknowns)
        b. Availability : People will typically ignore statistical evidence and base their estimates on their memories, which favor the most recent, emotional and unusual events which have a significant impact on them. 
        c. Gambler’s Fallacy: People make the assumption that individual random events are influenced by previous random events which might be spurious correlations and may not have causal relationships.
        d. Optimism bias: People overestimate the probability that positive events will occur for them, in comparison with the probability that these events will occur for other people.
        e. Confirmation bias: People seek to confirm their preconceived notions while  gathering information or deriving conclusions.
        f. Majority: People may go with the assessment of majority to conform with the group at the cost of their objective opinion which may be truer representation but different from the group opinion.  
        g. Self-serving bias: People have a propensity to assign to themselves more responsibility for successes than failures.
        h. Anchoring: People tend to base their estimates on previously derived/ used quantities, even when the two quantities are not related.
        i. Overconfidence: People consistently overestimate the certainty of their forecasts.
        j. Inconsistency: When asked to evaluate the same item on separate occasions, people tend to provide different estimates, despite the fact that the information has not changed.
    Solution in this case is to screen out whether the ratings have been influenced by these biases and inform the participants in advance to consider whether the ratings may have been influenced by such biases. Other mitigation measures could be blind peer rating or benchmark comparison with industry ratings for similar processes. 

    5. Interdependence between causal factors and failure modes: FMEA assumes that each risk is an independent event, whereas there may be a high degree of interdependence between factors which could influence risk rating significantly. Understanding and articulating such interrelationship could be challenging & not considering such impact could mean that the assessment is not representative of the possible risks & the resulting impacts. The way to mitigate this is to have a detailed discussion with all the relevant stakeholders & the process expert in a well-designed structure to ensure that all the risk and their interrelationships are well understood and documented. 
    6. Challenge in considering  the effect on both the customer or the process (assembly/ manufacturing unit). As against DFMEA(Design FMEA) where we look at the effects on the customer , in process FMEA, we will need to consider the impact & hence the severity rating of the failure mode if it impacts both the process & the customers. This is because, the impact of the failure mode in this case will mean the impact on the process or the customer in both the  cases. This leads to more complications in having to consider multiple scenarios. This challenge can be mitigated by taking the higher of the severity rating of the failure modes for other the process or the customer as the severity rating for the causal factor/ failure mode.  
    7. Challenge in deconstructing the impact of Root cause vs. assessing failure mode. Though there is a perspective that in some cases Root Cause and Failure Modes can be used interchangeably, however, if we drill down further, it is evident that root cause analysis is typically conducted post facto (after the event) whereas Failure modes identification happens proactively and will take into account various other factors apart from the proximate cause. The challenge is to ensure that this understanding percolates to the team creating the FMEA document. 
    8. Challenge in ensuring the risk assessment as an ongoing process vs as a single time activity -  Risk assessment (including identification & severity assessment) has to be an ongoing process & not a single point in time activity ( as the severity and impact may show material change in cases where there have been significant changes in either internal or external drivers, process dynamics or in key the environmental factors). The challenge is ensure that the rigor of assessment is maintained & updated with any relevant changes. The solution in this case is to have a monitoring / governance mechanism which will ensure that FMEA is kept as a live document with relevant updates to ensure correct risk rating.   
    9. Challenge in considering the impact due to timescale: E.g. the impact of a risk manifesting immediately may be significantly different from  that which may manifest after some time. The solution would be to conduct time-scale analysis of such risk factors to take into consideration the impacts of recent events and see whether the severity rating could change in such cases. 
 

[Natwar Lal 38]

Severity Ranking is the value that is given to the failure mode effects. It quantifies the impact of the failure mode on the customer in an FMEA. Denoted by S, the range is 1 to 10.

 

Working on FMEA in itself is a challenging task as the team is trying to figure our the "risks". Some of the challenges while assigning the Severity ranking are as follows:

 

1. Keeping Severity independent of Occurrence and/or Detection. This is one of the most common challenge. The team usually think that if the occurrence of a failure mode is less or if it could be easily detected, the failure mode is not as severe. 

How do you address it - keep reminding the team that the three rankings in FMEA are independent of each other. E.g. presence of a smoke detector (makes detection easy) does not impact the severity of the fire

2. Considering the effect of failure mode on the external customer or internal customer. This is a classical debate topic. Do you consider the effect on the end customer or the next process step while doing PFMEA? 

How do you address it - list down all failure effects in separate row items. This way one could separate the Severity rankings for effect on internal vs external customers

3. Assigning the Severity ranking considering the effect of product failures (DFMEA) while doing the PFMEA. PFMEA is done for process failures and not product failures but sometimes with the mindset of identifying risks, one could also start listing the DFMEA failure modes and their severity rankings

How do you address it - the product failure modes are ideally covered in DFMEA. One should abstain from capturing the same in the PFMEA. Failure modes and severity rankings in PFMEA should only pertain to the failures for the process

4. Subjective nature of the Severity rankings. This comprises of multiple challenges

a. The details for Severity ranking comprises of multiple themes - level of dissatisfaction, monetary impact, amount of rework, scrap etc. If one keeps switching between these themes i.e. for one failure mode you look at level of dissatisfaction while for another you consider the amount of rework, it might lead to confusion and misunderstandings

b. Ratings of 9 and 10. Both are hazardous with 9 being with warning while 10 is without warning. Now ideally if there is warning, it could be considered as a sort of detection. But if Severity is to kept independent of any sort of detection then why have 9 and 10. I mean if it is hazardous, it simply is hazardous irrespective of whether there is any warning or not

c. Lack of quantified impact for Severity ranking. The themes for Severity ranking are mostly qualitative and lack quantification. As is true with any qualitative ranking system one could debate on assigning a rating of 6 vs 5 vs 4 etc.

How do you address it - Before starting the PFMEA, spend some time to form a common ground of understanding (pick one theme) for the different Severity rankings to avoid unnecessary confusions and debates. You might also want to quantify the Severity rankings to make the selection easier.

 

Finally, once you are done with a few steps or may be at the end of the PFMEA (though it is advised to do it after a few steps), you might want to stop and review the severity rankings that you have assigned to ensure consistency and reconfirm the understanding.

[Manjula Pujar 3]

We are going to discuss all about PFMEA process Failure Mode of Effective analysis .

Before that let’s have look on FMEA i.e Failure Mode of Effective analysis.

FMEA is tool used to prioritize identify, quantify and evaluate the risk.

 

Goals

1. Reduce the Failure Risk
2. Detection of Failure are ensured
3. Prevention of failure

 

Why FMEA :

3. • FMEA is to track Failure of potential
   • Reduce the risk counter measures to be taken

In success of FMEA it starts with capturing all requirement properly and arranging potential failure modes.

 

FMEA has 3 Points

• Failure Mode
• Failure Effect
• Failure cause

 

There are 3 Types of FMEA

Design Failure Mode of Effective Analysis : This method is to detect error in design . for ex : if any dimensions or size  of product varies with standards due to which risk generates

 

Process Failure Mode of Effective analysis: This method is to detect error in process . for ex : If any problem with Ids

 

System Failure Mode of Effective analysis: This method is to detect error in System

 

Process Failure Mode of Effective analysis

The primary objective of PFMEA is to give proof of specific cause failure. If this can not be given then next level is mistake proof ,where team has to come up with ways of catching either cause or failure of  specific failure mode of defect.

 

PFMEA approach is to

• Identify ,reveal potential of failure
• Recognize function with process which reduce opportunity of potential
• To prevent out of conformance based on current details and document the process.
• Work towards corrective and prevention

 

 RPN: Risk Priority Number is component of PFMEA,

It deals with 3 factors

Severity: Seriousness of problem. It is rated in range of 1 to 10. Depending on how seviour problem is . if it is 10 then it may be without information If problem not effect function then it may 1 or 2, for ex  if color of any product which is working fine is not so suitable.

 

Occurrence: This is opportunities to raise the issues or problem it is rated from 1 to 10. The error which occurs or repeat highest then it extreme value toward 10

 

Detection: Identifying the problem, Rated between 1 to 10. The problem detected fery easily with less effort or which reflects its existence more is 10,

 

Risk Priority number = product of Severity, Occurrence and detection

 

RPN= S  *  O * D

 

Highest value of RPN is 1000 and lowest value of RPN 1

 

How to Address the PFMEA

• A Team need to Formed with process owners
• Expectations to be set about goal, objective and duration of time line
• Team has to go through process MAP
• Implement process map in FMEA step by step
• AS team work on Severity, occurrence and detection scores.
• When team will get RPN value work on corrective measures
• Corrective measures to be tracked for consistency.

 

 

 

 

 

 

 

 

[Praveen Kumar K 1]

Attached an spreadsheet explaining on the challenges on severity ranking and solutions to overcome the same

Common Challenges in Severity ranking	How to address the challenges
No deep clarity on whether the serverity ranking is for potential failure more or potential failure effects	Detailed training covering many examples from user understandable language
Failure effects should be considered at process or end user level. If this is considered at the high level it can lead to incorrect ranking of the severity	Creating a detailed procedure on how to prepare the FMEA
If there is no involvement of an experience or subject matter expert of the process, it can lead to wrong ranking	Creating a detailed procedure on how to prepare the FMEA
Involvement of multi-departmental teams and does not have knowledge on the FMEA	Involving an experienced quality personnel in faciliting the trainign and development of FMEA

[Vinod Shanmugham 1]

What is PFMEA?

A Process Failure Mode and Effects Analysis (PFMEA) is a step-by-step approach that identifies process failure modes and the effect of each of those failures.  A calculated risk score (Risk Priority Number) is derived for each failure in the process step, based on its Severity, Occurrence and Detectability.   

Why PFMEA?

We do PFMEA primarily:

•To prioritize the list of failures based on the calculated risk score

•To target the high scoring failures for improvement

•To document the current knowledge and proposed actions to be taken to address the risk of failure

 

When do we do FMEA?

•Use FMEA whenever an assessment and prioritization of risk is needed

•When designing or redesigning a process or product

•Before developing a control plan for a modified process

•Periodically, to assess process risks and failures

 

Challenge in Severity Assessment:

One of the major challenges in Severity Assessment we face while conducting a PFMEA is that the team not agreeing on Severity Rating.  Assessing Severity is important part of FMEA.   

Once the FMEA team identifies the most serious effect of the failure mode, the team then makes the assessment of ranking for Severity.  One thing we have to keep in mind is that here we are looking at the severity of the effect of the failure mode and not the severity of the failure mode itself.  If the effect is not well defined, that’s where it becomes a challenge or difficulty to assess the severity by reviewing the severity scale criteria. 

How can this challenge be addressed:

For a Process FMEA, what the team should consider is the effect of the failure at Process level as well as at the end-user/customer level.  Once the effect is properly articulated, then the severity can easily be arrived at by looking at the severity scale.

[Rowena 0]

Severity assessment challenges

a) Ownership of process FMEA is kept with Quality department. The ownership should be given to the person responsible for processes - design FMEA  should be owned by person responsible for design requirements and systems FMEA should be handled by person responsible for it

b) Sufficient knowledge and experience is important for those filing the matrix. The cross functional team should have sufficient knowledge to determine and assess severity. It is good to leverage best practices in this process

c) Communication and need for FMEA. It is recommended to get someone with previous experience to lead the team in this regard as it will build credibility

d) . The System FMEA, Design FMEA and Process FMEAs should be integrated together as it helps achieve results

e) Need for a team to be involved in earlier phases from design to development is important as failures can happen at any point of times

f) Pfmea becomes too text heavy, so use phrases that are easier to understand and concise, assign rating to numbers to explain severity which must be clearly communicated

 

.

 

[Hari Babu M 0]

New Issues with no history of occurence - 10 

High Critical Failures ( Not able to fix) caused due to new updates or new implementations or new ideas in a process -9

Moderate Failures ( Not Immediate fix) caused due to new updates or new implementations or new ideas in a process-8

Small failures (Have immediate fix )caused due to new updates or new implementations or new ideas in a process-7

Repeated Failures in the process-6

Occasional Failures in the process-5

Isolated Failures in the process-4

Rare issues in a similar process or steps-3

Observed issues in a similar process or steps-2

Errors having suitable check methods to address-1

Note: Numbers represents Rank

Edited August 27, 2019 by Hari Babu M
Foreget to mention how ton address

[Rushi Solanki 0]

1. Severity can be dependent on person or group assessment and his or her knowledge level. it will not cover all source of severity. for example, if we select 4 people they will have limited thinking and based on that all risk may not be covered.

2. Also severity is based on assumption, like if x fail because of y , this can z might happen actually we dont know whether Y is the actual root cause or not.

3. Also  good engineering is better than Fmea.

4. It doesn't take in to account statistical data.

5. It only prioritizes and doesn't make it right

6. It acts as reactive tool

7. it does not include supplier and customer

8. Just a document

9. Not linking with control plan, mistake proofing

10. No update once made

11. People may not be serious about the process.

[Indrani Poddar 1]

Severity is nothing but the impact or seriousness of a failure. When it comes to PFMEA, Severity assessment or ranking is of high importance as it is one of the factors affecting RPN score.

 

Some of the common challenges we face while doing Severity assessment could be -

 

a)     Severity ranking is provided basis immediate or short-term view without considering long-term view. For instance, there are balances being accumulated month on month in suspense accounts. If one provides a low Severity rating keeping in mind a short-term perspective as there is no impact on overall financial reporting. However, as a long term impact such balances retained would be aged. In an Insurance company if the suspense account is for client funds, retaining aged balances will have critical implication due to Escheatment laws and other regulatory requirements hence severity rating should be high. Thereby, long-term impact should definitely be considered.

 

b)     Severity ranking is often determined from the immediate perspective instead of end user/customer perspective. For example for invoice processing the processing team is only interested in meeting SLA target hence in such cases invoice processing or first touch would have a  higher severity rating however invoices pending in exception queue would have a lower severity rating.  However, in reality, high exception invoices leads to lower paid on time, hence vendor dissatisfaction and escalation and often a threat on production. Hence considering end user/ customer view for severity assessment is critical.

 

c)     Severity rating is determined based on a limited group of individuals’ perspective and whims of the participants. If participants creating the FMEA are experienced, knowledgeable and determines the severity rating basis the organization’s nature of business, objectives, priorities then such FMEAs would have a more accurate severity assessment.

 

d)     Upstream activities has severe impact on downstream activities. Hence, severity assessment would grossly go wrong if processes are considered in silos and not with an end-to-end view. For example; the accounts payable team has to close certain aged Open POs within a certain time frame as per the policy of the company. Only considering this activity in silo might determine a low severity rating. However when seen from an end-to-end perspective all open POs determine the accrual to be booked by the Reconciliation team. If the accrual entry is incorrectly done then the financial reporting is incorrect which has high implications hence both upstream and downstream processes are to be considered.

 

e)     Severity ranking cannot be established if the failure effect is not well defined. Often, all scenarios of effect of failure mode is not captured leading to one severity ranking for the failure mode. Below is an example, which shows how severity ranking can change basis effect of failure for the same process step and failure mode.

 

Process Step	Failure Mode	Failure Effect	Severity Ranking
Duplicate account is created for same supplier	More than one account is created for same supplier when the team receives creation request from different requestors	1. Duplicate account leads to duplicate payment.	8
		2. Decreases auto match process since invoice might get processed in X account and GR might get booked in Y account	4

 

 

 

 

 

 

Complexities and disagreements arise when the severity ranking is based on an improperly defined effect or inadequate severity scale. Hence, failure effect must be structured, explained and defined well which determines a more accurate severity scale. Severity rating must be determined basis effect of failure mode and not the failure mode by itself.

 

The above examples clearly explains why we end up with inaccurate severity rankings and how one can overcome the same while doing a process FMEA.

[Vijayalakshmi R 0]

 

1) Setting severity scale for the effects :
 This is most common challenge , if we are setting a low severity for a real effect/criteria , it might impact the business critically and can give an adverse effect when occured. 

2) Missing out the correct level :
 When the effect of failure has to be considered by the team in manufacturing or assembly level itself, but missed at these stages and identified in the later levels the impact of the effect could be worse and might end up with higher severity rating. 

So looking at the effect/criteria a little more closer and also in detail can give us a real picture, this might help in setting the correct severity scale and ranking. 

[Rupinder N 31]

The winning answers are for Prasanna Pokhrel and Natwar Lal. Both the answers provide a good holistic view of how assigning Severity Rating on FMEA can be challenging and how the challenges can be mitigated.