Q 12. While pursuing Business Excellence, given a choice, which error will you prefer over the other - A false alarm or a missed alert?

 

Explain your answer with suitable examples. 

 

Note for website visitors - Two questions are asked every week on this platform. One on Tuesday and the other on Friday.

• All questions so far can be seen here - https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/
• Please visit the forum home page at https://www.benchmarksixsigma.com/forum/ to respond to the latest question open till the next Tuesday/ Friday evening 5 PM as per Indian Standard Time.  
• The best answer is always shown at the top among responses and the author finds honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

I would like to call out the definitions with related example scenarios first.

 

False Alarm (also known as False Positive):

Where you receive a positive result for a test when you should have received negative results.

Some examples of false alarms:

• A cancer screening test comes back positive, but you don’t have the disease.
• Anti-Virus software on your computer incorrectly identifies a harmless program as a malicious one.

Effects of False alarms: Agony, Apprehensions, re-tests, Increased costs.

 

Missed alerts ( also known as False Negative):

 

You may miss the alert completely of the potential effect. In other words in case of a  test, get a negative test result, but you should have got a positive test result.

Example scenarios:

• Quality control in manufacturing; missed alert means that a defective item passes through the cracks.
• In IT  security, it would mean that a test/ product designed to catch something (i.e. a virus or potential intrusion attack) has failed.
• In the Justice System, when a guilty suspect is found Not Guilty and allowed to walk free.

Missed alerts create a false sense of security and potentially dangerous situations. For example, a crippling computer virus can wreak havoc if not detected, or an individual with cancer may not receive timely treatment.

 

From the above example scenarios, it is clear that given a choice businesses would prefer False alarms over Missed alerts.

As the last example, I would like to quote many Tsunami Warning systems that have been built globally with state-of-the-art systems, end up providing False alarms (which is based on surrounding facts). We all know the peril of having missed alert in case of Tsunami.

In Business Excellence, false alarm and a missed alert are two aspects that need focus.  

 

Missed Alert would be the key aspect for Business Excellence. However false alarm might have some impact in industries such as healthcare, finance, insurance. Reduction of false alarm will improve the motive of striving for Business Excellence.

 

While a missed alert would mean probably a missed Service Level Agreement (SLA) resulting in a probable customer dissatisfaction or customer penalty for breach of SLA and which can also business growth, false alarm would create unnecessary panic to the relevant stakeholders and put stress on individuals, in some cases. The objective should be to reduce the false alarms, as much as possible.  Let us see how this can happen, starting from the definitions followed by some examples

 

Alarm: An alarm is basically set to ensure that in case of any unwarranted events/steps that arise out or if there is a deviation from the standard process/routine/procedure, then it can notify the relevant stakeholders by sending a signal or sound to induce any manual intervention or necessitate any sort of automated processing actions, to resurrect things back to normalcy.  

Eg: Fire alarm in buildings

 

Different ways of Triggering Alarms:

 

In many cases, alarm triggering can happen in 3 ways – on real conditions, for alarm drill purpose and false alarm.  While we can schedule alarm for drill purpose, in real situation and false alarm, we cannot predict when the alarms might trigger.  

 

Example 1

Consider Fire alarms in buildings. Alarm can get triggered if there is a fire that catches the building. The smoke/heat detector in the office premises can catch hold of the fire and the alarm will get triggered. This is in a real time scenario. But there can be a planned mock fire drill, which can be used to assess how strong is the process for human beings to survive from the clutches of the fire. But there can be a false alarm which could have happened because the smoke detector malfunctioned or the alarm system or the communication system to the alarm malfunctioned.

 

Example 2:

Consider the case of water level in a dam. Because of incessant rains, the level of the water has almost reached to the optimal level beyond which the government authorities have to release it. In this case, the real time scenario is the only way of triggering the alarm. Mocking/Mimicking of the situation can be done with a simulation effect but there is no room for false alarm here

 

False alarm:

A false alarm is one in which all the sequence of steps followed in an alarm are followed , except for the fact that rather than happening naturally it is triggered because of either a human error, or a malfunctioning system error (hardware/software) or communication error between disparate systems or a combination of all these errors.

 

Now again the considering fire drill alarm example, we can see many instances, there could be a false alarm due to any of the above combinations. All we think about is the momentary anxiety and the relief we get after it becomes a hoax alarm. But it does give a thought about as how far our alarm system is that. How naturally we acted upon and how was our self-discipline during those momentary period.

False alarm serves as a blessing in disguise in that you feel that it is real but the impact is not there.  But many a time, organisations quickly announce that false alarm was made and hence employees return to their normal work place in no time.  So to simulate real time situation, organisations on a periodic schedule, conduct mock drills (in this case fire drills) but without any intimation (to the staffs/employees) and see how the people part/process respond to that. 

 

During Fire drills (usage of fire alarm systems), say there could be a guidance that in 2 minutes, all the people in a 8-floor building should come down to the designated point at a ground level using Emergency Exit doors (by foot-steps) .  Normally mock drills, will operate in false alarm mode. Instead of accidentally fire catching up the office floor(s), a lighting of a cigarette or lit up of a match stick could be purposefully done to trigger the smoke/heat detector, which can inject the fire alarm system.  This is an indirect false alarm system as it is an induced alarm system but used for finding out the effectiveness of the system.

 

Missed Alert

 An alert in a business environment, means that a notification message would be sent from the system (product/service/application) to the stakeholders (it could be sent to an individual customer, or to any mgmt key stakeholders of the customer company or any other relevant stakeholder), in most cases on a scheduled time.  In some cases, alert could occur on a conditional basis. It can happen in the event of some other activities/process(es) happening

 

 If the service providing organisation does not produce the alert or delays the alert or misses the alert or reacting to it , for whatever reasons, then it becomes an alert which is missed out. This directly leads to breaching of SLAs and potential loss of customer if it keeps happening and a dissatisfied customer.

 

Eg:1  Banking Domain: Suppose ABC Company is the service provider for a banking customer

 

In the banking application, created by ABC, alert text messages would be sent to the bank manager  which can tell him/her some facts about the due date status  for ‘home  loan interest’ of the bank customers at the beginning of each month or just 2-3 days before the actual due date for a bank customer(end-user) . With this alert, the bank manager would know how much loan amount needs to come to the bank on a routine basis.   In case, if the alert is missed , then the bank manager will not know the loan amount the bank should get for that period and this can result in heavy loss for him/her, in terms of planning for any other monetary work. The manager would get irate and this can lead to his/her dissatisfaction at the service provided by ABC Company.  This has to be rectified quickly and avoided in the future, so that ABC company does not jeopardize its business with the bank.

 

So missing an alert can make you breach the SLA, which can result in customer dissatisfaction and a possible business loss, if this continues in the longer run. This will be the case in any industry.  For instance, if you take a healthcare industry, missed alert cannot be there. If a patient’s hear-beat is monitored, then all the programmed alerts should happen and no alert (meant to the Doctors and Nurses) should be missed out. If any notification message is missed out by the service provider, it will not help the doctors and the patient (end-user) will suffer. Here Missed SLA is costly and will affect the human being.

 

So avoiding ‘Missed’ alert scenario is of paramount importance when we strive for business excellence. However in some circumstances or in rare cases, focus should also be given to false alarm.

 

Why False Alarm is important in Few cases:  

Let us take healthcare industries.  Assume a person is put in observation for a probable disease of specific nature and the patient is being monitored for that illness. The patient is undergoing various procedures, blood pressure, sugar test, urine test and other routine procedures.   Now when all these things are fed to a system and if one of these happen to be at a level lower than the normal level, then an alarm(signal) is triggered (read automated) which in turns sends some processing instructions , one of which could be a report to the chief doctor. 

 

But imagine that there was a processing error (human) while doing one of these procedures or while collecting the data, there was a momentary glitch in the system and the alarm system did not capture all its collected data properly. Because of this, a false alarm signal went and Chief doctor got a report.   What if erroneous data had been there in that report about the Sugar levels or what if all the data were right but just due to some other factor the alarm got triggered .

 

If the report contains incorrect data (sample) , then the patient may be told about his/her problems and that could lead to wrong dosages given to him/her.  The patient could get emotionally stressed out and this can lead to some other medical problems.  So its imperative that in industries/domains such as healthcare, false alarm should be reduced to achieve the motto of business excellence.

 

Conclusion:

Thus , we can see how Missed alert impacts Business Excellence and also understand how reducing false alarm can help you in your objective of striving for business excellence in industries like healthcare

While practising business excellence there should be endeavour to minimise both false alarm as well as missed alert.However it primarily depends on the nature of business and the quantum of risk that the business can afford..for eg. In aviation industry..during flying if the operator gets false alarms it can be rechecked and assured by exploiting other parameters but in case there is a critical warning or alert which is not shown or detected or missing can lead to catastrophic consequences.Another example could be securing the international border of our country.If there is a false alarm it can b checked and acted upon..but if there is infiltration and is missed then it is not acceptable.

Immediate choice between "False alarm" and "Missed Alert" will be "False Alarm" rather than "Missed alert"  - as the saying goes "Better to be safe than to be sorry". However situations where there are too many false alarms can instill a detrimental psychology. For example, the fire alarms in our high rise buildings. Whenever the alarm goes off, all the employees are expected to get up, leave their work and quickly walk in an orderly fashion towards the staircase. Some of them are also expected to ensure evacuation of people from restrooms and other areas. We have seen the alarm going several times - either for a mock drill or it would have gone off due to a system fault. It is very unlikely that we would have experienced it going off for a real fire (and of course, we wish it never happens!). Often we see that the alarms are not taken seriously at all and several employees assume with high certainty that each incident must be a false alarm or mock drill. However these are situations where such "False alarms" are highly necessary to prepare for an emergency that might be very unlikely, but could be disastrous if it occurs.

 

Control charts operate to give alerts on situations that require an action to be taken when a process mean shifts. However they are developed based on the statistical probabilities and there is always a chance that a point that fell outside the control limit need not be a result of mean shift. However the decision rules have been worked with such probabilities that the chances of a false alarm would be much unlikely than a genuine alert. Similar is the situation with most other statistical tools viz. Tests of hypothesis, Acceptance sampling plans. However the statistical methods give us a confidence level, that provides a quantification of 'how safe' do we want to play, and this can be chosen depending upon the criticality.

 

I would like conclude that in most situations, there are always risk factors associated with getting False alarm and Missing an alert. However we should plan to manage the probability of occurrence of false alarms taking into considerations the severity of consequences. For situations with high severity risk viz. involving life, health, financial, it is acceptable to build high factor of safety, that would expect us to put up with certain "False Alarms".

Quite tricky though. Here is what I interpret –

A false alarm is a situation when the system has the right conditions to prompt us to a fault, yet recovers itself without a failure. It raises an alarm, however the proceedings are still under control. The measuring system and its resolution are to be checked for the same to ensure that accuracy and precision of the deliverable are effective.

 

For instance, consider the weather forecast, earthquakes, Global warming reports, stock market and other highly unpredictable human undertakings. The false alarms are quite regular as the profound knowledge of system dynamics, variations and sensitivity turns out to be quite huge. With the advent of stronger statistical tools we are able to avert dangers and learn.

 

Another case is when the customer raises an alarm of bad treatment despite the operator following the protocol of acceptable customer service and satisfaction measures. The numbers might indicate a DSAT score, nevertheless it could be one odd instance and should have special treatments in place for the future.

 

Missed alert is a situation when there was a fault shaping into a failure and the relevant mistake proofing/preventive measure has missed identifying the same. Constant review of the control chart, mistake proofing principles, training the staff and optimizing the process should give us considerable certainty.

 

For instance, catastrophe to the Challenger shuttle in 1986, where the alerts of low temperature at the launch pad, the gas leak and the worn out rings were alerting the mission control and yet been ignored.

 

Employee dissatisfaction and not taking up the cause of creating an effective working environment. It is the most common indication of poor team formation and neglected by the leadership at times as they are focused on the business gains and the SLAs.

 

One can still wake up by a false alarm but shouldn't sleep off with a missed alert.

False Alarm and Missed alert both have negative impact. 

 

False alarm may increase the rejection % and may increase the unnecessary rework just because of confusion. It may directly impact on cost as well as time. 

 

At the same time missed alert may increase the probability of accepting wrong or defect products. Which directly impact on customer satisfaction and brand value. 

 

For any low cost product, if we want to avoid any rework then certainly we will not want any false alarm. 

 

Similarly for any critical, high value process like banking or manufacturing of air craft etc, we will certainly not accept any missed Alert. 

 

A false alarm:- Rejecting a good product. As the no. of false alarm increases rework/rejection increases. So it’s a producer’s risk.  Most of the producers will be keen on reducing false alarms. It can also be referred as Type 1 error.

 

Missed alert:- Accepting a rejected product. When a bad product is accepted and reaches customer/consumer. It is consumer’s risk. It can also be referred as type 2 error.

 

Which error to be preferred over other depends on the criticality of the product/parameter that is in consideration?

In food industry:- there are many parameters that are checked.

 

One of the main parameter is addition of allergen/non allergen ingredients.

 

In this parameter, missed alert will have more impact. As if an allergen product is consumed by a person, it can lead to his death also.

 

Taking one more parameter:-  Color of product. If we were not able to produce the product as per target color, and it is off by just a bit. In this case more emphasis is to be given on false alarms. As too many false alarms will lead to rework/rejection.

I would prefer a false alarm rather than a missed alert. Because

 

False alarm:  This is any small shift in the process behavior will be detected (that’s good) but also  you will have too many points outside the limits just by chance when there is no special cause to address (that’s bad). That is called a false alarm. 

 

Missed alarm: Missed alarm is that we have already missed the special cause which is suppose to be detected. Already the loss happened, cannot be recovered.

 

So false alarm is better than missed alarm.

I would prefer a false alarm over missed alert as false alarm means my 'good' thing is being shown as 'bad', this is something internal to the business and a corrective action can be taken, whereas a missed alert means that something 'bad' has been passed on to the next stage as 'good' and it may be passed on to the customer too, which will lead to customer complaint and may be to loss of goodwill. However this answer is valid as long as the frequency of these false alarm or missed alert is less or within a reasonable limit, otherwise too many false alarm will also result in loss of business.

Background and Concept:

 

False Alarm and  Missed Alert are better understood with the two types of errors that are possible in statistical Hypothesis testing. Dealing with them with reference to test of hypotheses will provide more insights than otherwise. 

 

Any hypothesis test is begun with the assumption that the null hypothesis is correct. Null hypothesis is the default position and corresponds to the idea that "one is innocent until proven guilty".

 

False alarm or Type I errors or False Positives (α): They happen when we reject a true null hypothesis.

Missed alert or Type II errors or False Negatives (β): They happen when we accept (fail to reject) a false null hypothesis.

 

Which error will you prefer over the other?

 

The answer to this question depends on the problem and the worst that could happen if either a Type1 or Type 2 error was committed.

 

Example 1: Person accused of Murder awaiting Death Sentence.

 

Null Hypothesis: Person did not commit murder.

 

Type 1 error: Person did not commit murder but pronounced guilty. (Rejected true Null Hypothesis)

Type 2 error: Person committed murder but pronounce Not guilty. (Accepted false Null Hypothesis)

In this example, though Type 2 error is not favorable to society, but hanging an innocent person is far worse. So

Type2 error or a Missed alert is preferable.

 

Example 2: Person being screened for a disease to prescribe further tests.

Null Hypothesis: Person does not have the disease.

 

Type 1 error: Person does not have the disease but recommended for further tests. (Rejected true Null Hypothesis)

Type 2 error: Person has the disease but recommended for no further tests. (Accepted false Null Hypothesis)

In this example, Type 1 error might cause the patient to undergo further tests but might finally reveal that he does not have the disease. A type 2 error would prevent a legitimate patient from undergoing further tests.  But a legitimate patient can re-do the test if the symptoms persist, and it is fine for a person to do some further tests even if he does not have the disease. So Type1 error or False alarm is preferable.

 

Example 3: Person being screened for a disease (presence of which has a good rate of survival and normal life) to prescribe a delicate specialised surgery that has poor success rate. 

Null Hypothesis: Person does not have the disease.

 

Type 1 error: Person does not have the disease but recommended for surgery. (Rejected true Null Hypothesis)

Type 2 error: Person has the disease but not recommended for surgery. (Accepted false Null Hypothesis)

 

In this example, Type 2 error might cause the legitimate patient to not have the surgery which is bad, but it is much worse to have a person without the disease undergo the delicate critical surgery. The legitimate patient may re-do the tests, if he still feels the symptoms of the disease and may be re-diagnosed to undergo the surgery. In this case, a Type2 error or a Missed alert is preferable.

False alarm - there is no cause of concern but a concern has been flagged. So when this happens, the responsible person can validate the concern is not valid and no specific action may be required. However when false alarms occur frequently or repeatedly, there is a productivity loss in validating false alarms.

For eg. 1. weather department raised an alarm of Hurricane Irma hitting Florida. As a result mass evacuation was ordered. However if Irma doesn’t cross Florida then it is false alarm – effort or cost of evacuation is a waste.

2. quality department raised alarm about a specific issue in Space Shuttle Challenger, so engineering team inspects the issue reported and confirms that this is not an issue... this is a false alarm.. in this case Space Shuttle Challenger will not face specific issues in its mission, though internally a cost overhead will be borne due to false alarm. 

 

Missed alert – an alert was needed but was missed to raise or an alert was sounded but person responsible missed to take action on it. In both cases the case was a genuine concern and due to lack of alert or lack of action on alert, concern was not attended to, leading to damages.

For eg. 1. weather department didn’t raise an alert or alert was raised but no action was taken to inform public or evaluate about Hurricane Irma hitting Florida and Hurricane Irma actually hits Florida… this would be a disaster and result in heavy loss of life…

2. quality department misses to flag a major defect in the Space Shuttle Challenger or defect was reported but it was missed, then Space Shuttle Challenger will run into an issue in its mission... now this would an issue and may result in severe losses... 

 

The missed alert situation is definitely more alarming than false alarm and can result in more significant losses. However repeated false alarms can result in loss of productivity or result in cost overheads. 

What is False alarm and Missed Alert?

 

False Alarm – refers to a system that triggers unexpected warning / remainders, which leads to unnecessary actions / response to be taken from our end.

 

Missed Alert – when a system fails to trigger warning / remainders at the scheduled time or when required, so that necessary actions / response will be taken to mitigate or plan course of action

 

Given a choice, I would prefer False Alarm over Missed Alert; because in such scenario my actions will go in vain whereas in case of a missed alert I would rather missed to act which leads to unnecessary risks or put me troublesome situation.

 

Example:

False alarm triggered from Fire bells will only lead to unnecessary evacuation being done. However, if the Fire bell does not trigger Alarm then it would to lead to unnecessary consequences.

On a lighter note, as is understood, False Alarm is a “Mistake in finding a Mistake”, i.e. A product/process is good/ correct but the alarm or check system has pointed it out as bad/ incorrect. The customer may not be able to know about this mistake, and hence unaffected, in most of the cases.

Whereas, Missed Alert is when product / process is actually not good (NG)/ incorrect but passes through the check system and the Customer has experienced the impact of the bad product or incorrect Process.-Thus it is a “Mistake of not finding a Mistake”

 

Though at first thought , it seems that A false alarm is no problem at all , as the customer seems to be unaffected, Too many or untimely false alarm can cause bad experience to the customer , whereas a Missed alert is definite cause for a bad customer experience.

Hence, both the situation are lapses in the process and have their individual negative impact on the operations.

 

To understand how a more “safe looking” False alarm can cause bad experience to the customer, Let us consider a Kitchen in a QSR, wherein , products to be cooked are pre frozen and stored in cartons. The Cartons are arranged by date of manufacture to ensure that a First in First Out (FIFO) process is followed, so that all products can be used within expiry and thus wastage is reduced.

During Peak rush period- lunch time- If the person who is responsible to get the boxes from the Freezer to the production area, suddenly believes that he may have brought in a box that was out of the FIFO- It may be a false alarm- The time taken to sort out the confusion and then to get another box in the right FIFO order, can create delay in serving the customer. The extra effort to catch up with the lapse in Takt time could lead to further defects in the process (Food Production)-

However, in this same situation, maybe an immediate call can be taken to use the box and then correct the arrangement of the box in the low rush time to ensure that the FIFO is followed in the future. This way ,the time delay is reduced and the damage can be mitigated.

Thus, a false alarm, if happening occasionally, or if handled well tactically, will not cause much loss to company, most of the times ,and can be corrected or the damage can be mitigated.The chances of the same false alarm happening can be minimised by reinforcing the check points.

Having said that, it is also very obvious that a false alarm can lead to process adjustments when none is required in an existing correct process and cause many NVA.

 

 

However, if an alert is missed, it may affect the customer directly- For example in the Kitchen of a QSR, fried products are maintained in a Heated holding cabinet for a maximum Specified time (USL), beyond which the quality of the product deteriorates below the set standards of the product. The Holding cabinet has a timer that alerts once the time limit is over. If this alert is missed, a below standard product passes on to the customer, as there is no physical (Touch) check on the patty thereafter- very seasoned staff is required to visually detect a patty which is not in proper quality.

This can lead to customer dissatisfaction and breach of expectation- Loss of reputation and eventually loss in business.

Thus when asked a preference between the 2 errors, it is like choosing between the devil and the deep blue sea- However, with the above explanation and also given that Statistically for attribute measurement < 2% missed rate is considered as good and < 5% false alarm are considered in good category as general guideline, the space for false alarm acceptability is higher-

 I will “Prefer” the false alarm!!

 

Alarms:
These are to notify the user that there is something wrong with the process and needs attention. They could be manual or automatic. These both could result into a 'False Alarm' or 'Missed Alarm'.

 

Now, both false and missed alarms are based on specific criteria. False alarms can help users to make sure that they are attentive and are able to respond in time in case of any actual "Alarm".

 

While, Missed alarm may led to notify the user's authorities that there was an alarm and an important one that was missed by the user and could turn fatal. In both the cases a missed alarm could result in losing the business and too many false alarm may also result in losing the business.

 

However, one should always define the criteria of how a false alarm will be triggered and what needs to be done if an alarm is missed. What is the criteria that is "Ok" for an Alarm to be Missed and recurrence for a False Alarm. This will be defined only when there is sufficient data and continuous monitoring of data that is received in the process.

 

This will also define the Business Excellence as well.

While pursuing Business Excellence, given a choice, error which I would prefer would be a False alarm over a missed alert.

 

A missed alert would mean that a mistake has happened but the trigger was not given paid heed at that time.

 

A false alarm on the other hand means that alert was raised though the mistake was not there.

False Alarm vs Missed Alerts

 

The acceptance of both is subjective to business scenarios. In general cases, it appears that false alarms are more acceptable considering they do not directly/immediately affect customers in financial terms while missed alerts could be fatal depending on criticality of the parameter in question. It is a trade-off decision to set thresholds for both false alarms and missed alerts considering the industry and scenario. This trade-off produces two different cognitive states.

 

A false-alarm prone process reduces 'compliance' and may lead to all alarms including true ones being responded to late or possibly not at all.

A miss-prone process reduces 'reliance' and it will warrant to use spare monitoring capacity for the alerted event in the course of correction.

 

A false-alarm prone situation might be acceptable when there is evidence that it does not impact overall effectiveness with the presence of other triggers / standard process behavior of monitoring the event. In the scenario of health-monitoring related automations ( e.g. system to detect possible seizures) , a false alarm is often taken as more preferred compared to missed alerts. This will impact the quality of the underlying algorithm though but patient wellbeing takes higher priority than system effectiveness.

 

A missed-alert prone situation might be acceptable if there are other means of  fault detection/mitigation available. A study was done in the case of unmanned air vehicles and it concluded that other triggers like visual monitoring took up larger attention over system generated alerts and mitigated the risks to great extent. In this study, a false alarm prone system proved to be more 'costly' compared to missed-alert prone system. 

 

[Source: http://www.aviation.illinois.edu/avimain/papers/research/pub_pdfs/hfes/dixonwickchang.pdf ]

False alarm is more preferred over missed alert because false alarm is a detection technique where defect has already happened whereas missed alert is a prevention technique where we have missed some steps in process of preparing end product which is a warning which might lead to defect but defect hasn't happened

When our toast burns and or there is smoke from a burnt food in oven and smoke alarm fails then it's type 1 error or false alarm. 

When there is a fire in house and smoke alarm fails then it's type 2 error or missed alarm. 

Though type 2 error is very rare assume the functioning proper functioning of smoke alarm but it comes at cost of too many type 1 error

In the scenarios of Cyber security operations, too many false alarm may lead to desensitized security personnel where there is chances of critical alerts getting missed. Where as few missed alert may not be significant in comparison to too many false alarms. Also in Health care centers where there are alerting system like blood pressure machines, ventilators and heart monitors, there is risk of too many false alerts sounds making the brain of the personnel in charge getting tuned to the  sounds of alert where as few missed alerts in comparison would not be significant.

First let us know about what is exactly false alarm & missed alert?

 

False alarm is nothing but suggests that the object is actually good but alarm is showing as bad object. on the other hand a missed alert suggests that object is actually  not good but passed.

 

I would prefer false alarm over missed alert for pursuing business excellence.  A false alarm occasionally may be okay or we may never want it. it depends on situation to situation. However, too many false alarms can lead to the assumption that something is wrong leading to an unwanted change in a well behaving process. A missed alert may never be acceptable or may be sometimes okay, it also depends on situations. Let us understand it with some examples below:

 

1.      A false alarm might cause inhouse rejection increase but customer is not bothered about it on the other hand missed alert will pass the NG object to customer and based on the severe nature of the NG , a customer complaint may arise.

 

2.      Alarms are set based on specific criteria. Based on some trend observed in past false alarms it can further be calibrated to improve the predictability. The end objective is that we do not end up in a situation that can lead to unwarranted consequences. On the other side A missed alert will lead to a negative outcome in most cases. If the alerts (control system) do consider adequate buffer it will help us to recover but not in all scenarios. Although practical solution will depend on various factors like cost of deploying the mechanism, maintenance, criticality of process being monitored, etc.

 

3.      A false alarm will not cause any loss to company rather than creating some confusions on shop floor, but if a alert is missed, it may affect the economy and heavy loss can be seen(if alert of coolant is missed in CNC and is not turned off, the overfilling of coolant may cause loss for company).
Thus, according to me we should prefer a false alarm and not a missed alert.

 

4.      An occasional false alarm is fine but too many of these can lead to loss of business or even closure of a company. As an extreme example, let us consider an automotive company that decides to do 100 percent inspections on all parts leading to high cost of product which finally leads to loss of market.

We need to also consider that a false alarm can lead to process adjustments when none is required in an existing good process.

 

5.      A false alarm is highlighted to everyone. However, excessive alarm is something that one need to keep an eye on it. A missed alert is something like a loss is coming for e.g. financial, mechanical, human or anything, but negligence or a lazy attitude did not trigger the awareness about the same.

 

6.      Though false alarms can increase the incidence of rechecks, it can definitely avoid reworks. If we see healthcare industry missed alerts can be fatal.

 

7.      False alarm is nothing but defect in defect identifying method, and missed alert can be handled by error report developed in the application so missed alert should not be that big issue. An outlook giving me a false alarm about a meeting that is cancelled. I can review and correct it. Too many false once means there is a series that has been cancelled and I am receiving the alert unwantedly. On the other hand an alert that should remind me of a meeting getting missed cannot be taken..A team huddle getting missed because of it missed.

A False alarm is an issued alert without a subsequent conflict. It occasionally may be okay or you may never want it. It depends. However, too many false alarms can lead to the assumption that something is wrong leading to an unwarranted change in a well behaving process.

 

Conversely, a missed alert is a conflict with no prior issued alerts. A missed alert may never be acceptable or may be sometimes okay. It depends.

 

Both are measures of reliability of conflict detection approach.

 

Missed alarm are considered as an extreme hazard conflicts leading to serious damage to the system or process, while false alarms are considered as nuisance alarms resulting in unnecessary escape maneuvers.  Hence minimizing false alarms and eliminating missed alarms are the crucial design requirement for any process it matters.

 

As per many textbooks and many instructors, type 1 (false positive ) error is more serious  than type II error(false negative).The honest answer is “it depends” on the situation. Also it depends on the null hypothesis that we set to test the data.

 

In statistical test theory, the notion of statistical error is an integral part of hypothesis testing. The statistical test requires an unambiguous statement of a null hypothesis (H₀), for example, “this person is healthy”, “this accused person is not guilty” or “this product is not broken”.   The result of the test of the null hypothesis may be positive (healthy, not guilty, not broken) or may be negative (not healthy, guilty, broken).

If the result of the test corresponds with reality, then a correct decision has been made (e.g., a person is healthy and is tested as healthy, or the person is not healthy and is tested as not healthy).  However, if the result of the test does not correspond with reality, then two types of error are distinguished: type I error and type II error.

Type I Error (False Positive Error)

A type I error occurs when the null hypothesis is true but is rejected.  Let me say this again, a type I error occurs when the null hypothesis is actually true, but was rejected as false by the testing.

A type I error, or false positive, is asserting something as true when it is actually false.  This false positive error is basically a “false alarm” – a result that indicates a given condition has been fulfilled when it actually has not been fulfilled (i.e., erroneously a positive result has been assumed).

Let’s use a shepherd and wolf example.  Let’s say that our null hypothesis is that there is “no wolf present.”  A type I error (or false positive) would be “crying wolf” when there is no wolf present.  That is, the actual condition was that there was no wolf present; however, the shepherd wrongly indicated there was a wolf present by calling “Wolf! Wolf!”  This is a type I error or false positive error.

Type II Error (False Negative)

A type II error occurs when the null hypothesis is false but erroneously fails to be rejected.  Let me say this again, a type II error occurs when the null hypothesis is actually false but was accepted as true by the testing.

A type II error, or false negative, is where a test result indicates that a condition failed, while it actually was successful.   A Type II error is committed when we fail to believe a true condition.

 

Continuing our shepherd and wolf example.  Again, our null hypothesis is that there is “no wolf present.”  A type II error (or false negative) would be doing nothing (not “crying wolf”) when there is actually a wolf present.  That is, the actual situation was that there was a wolf present; however, the shepherd wrongly indicated there was no wolf present and continued to play Candy Crush on his iPhone.  This is a type II error or false negative error.

A tabular relationship between truthfulness/falseness of the null hypothesis and outcomes of the test can be seen in the table below:

 

	Null Hypothesis is true	Null hypothesis is false
Reject null hypothesis	Type I ErrorFalse Positive	Correct OutcomeTrue Positive
Fail to reject null hypothesis	Correct outcomeTrue Negative	Type II ErrorFalse Negative

Examples

Let’s walk through a few examples and use a simple form to help us to understand the potential cost ramifications of type I and type II errors.  Let’s start with our shepherd/wolf example.

 

Null Hypothesis	Type I Error / False Positive	Type II Error / False Negative
Wolf is not present	Shepherd thinks wolf is present (shepherd cries wolf) when no wolf is actually present	Shepherd thinks wolf is NOT present (shepherd does nothing) when a wolf is actually present
Cost Assessment	Costs (actual costs plus shepherd credibility) associated with scrambling the townsfolk to kill the non-existing wolf	Replacement cost for the sheep eaten by the wolf, and replacement cost for hiring a new shepherd

 

 

Note: I added a row called “Cost Assessment.”  Since it can not be universally stated that a type I or type II error is worse (as it is highly dependent upon the statement of the null hypothesis), I’ve added this cost assessment to help me understand which error is more “costly” and for which I might want to do more testing.

 

Let’s look at the classic criminal dilemma next.  In colloquial usage, a type I error can be thought of as “convicting an innocent person” and type II error “letting a guilty person go free”.

 

Null Hypothesis	Type I Error / False Positive	Type II Error / False Negative
Person is not guilty of the crime	Person is judged as guilty when the person actually did not commit the crime (convicting an innocent person)	Person is judged not guilty when they actually did commit the crime (letting a guilty person go free)
Cost Assessment	Social costs of sending an innocent person to prison and denying them their personal freedoms (which in our society, is considered an almost unbearable cost)	Risks of letting a guilty criminal roam the streets and committing future crimes

 

 

Let’s look at some business-related examples.  In these examples I have reworded the null hypothesis, so be careful on the cost assessment.

 

Null Hypothesis	Type I Error / False Positive	Type II Error / False Negative
Medicine A cures Disease B	(H0 true, but rejected as false)Medicine A cures Disease B, but is rejected as false	(H0 false, but accepted as true)Medicine A does not cure disease B, but is accepted as true
Cost Assessment	Lost opportunity cost for rejecting an effective drug that could cure Disease B	Unexpected side effects (maybe even death) for using a drug that is not effective

 

Let’s try one more.

 

Null Hypothesis	Type I Error / False Positive	Type II Error / False Negative
Display Ad A is effective in driving conversions	(H0 true, but rejected as false)Display Ad A is effective in driving conversions, but is rejected as false	(H0 false, but accepted as true)Display Ad A is not effective in driving conversions but is accepted as true
Cost Assessment	Lost opportunity cost for rejecting an effective Display Ad A	Lost sales for promoting an ineffective Display Ad A to your target visitors

 

 

The cost ramifications in the medicine example are quite substantial, so additional testing would likely be justified in order to minimize the impact of the type II error (using an ineffective drug) in our example.  However, the cost ramifications in the Display Ad example are quite small, for both the type I and type II errors, so additional investment in addressing the type I and type II errors is probably not worthwhile.

Summary

Type I and type II errors highly dependent upon the language or positioning of the null hypothesis. Changing the positioning of the null hypothesis can cause type I and type II errors to switch roles.

 

It’s hard to create a blanket statement that a type I error is worse than a type II error, or vice versa.  The severity of type I and type II errors can only be judged in the context of the null hypothesis, which should be thoughtfully worded to ensure that we’re running the right test.

 

I highly recommend adding the “Cost Assessment” analysis like we did in the examples above.  This will help identify which type of error is more “costly” and identify areas where additional testing might be justified.

False Alarm Vs. Missed alert.docx

On ‎9‎/‎18‎/‎2017 at 10:08 PM, Vishwadeep Khatri said:

Q 11. While pursuing Business Excellence, given a choice, which error will you prefer over the other - A false alarm or a missed alert?

 

Explain your answer with suitable examples. 

I would prefer an error of  False Alarm over an error of Missed alert.

 

If an error happens in the form a Missed alert – we stand to lose out an opportunity to avoid a potential quality issue/ risk/service outage.

An early warning signal is missed out.

This could result in loss of credibility/warranty costs etc.

 

If an error happens in the form a False Alarm- we tend to hit the emergency recovery procedures. We may end up evacuating buildings/stopping production lines/redoing the quality procedures  etc. But we may end up with no actionable items.

 

Fact is in  both the cases- there would be cost implications.

 

However, a Missed alert is a missed opportunity to fix something within our possible control/influence.

A False alarm only end up in wasted resource/schedule and some frustration internal to an organization with no real damage to credibility/customer experience.

 

Another way of looking at this is-

A Missed alert has no immediate internal impact, but is a potential risk and can have external credibility impact.

A False alarm is a short term internal inconvenience/internal impact and less/no external credibility impact.

 

False alarm is ok as it make us to stay alert. But, Repeated false alarm will lead to loss in business, as this will require lot identification or rectification of the problem which is not true and this rectification is cost to the business. False alarm will only lead to high rechecks.

Example: False alarms in an automated company will have to check from the scratch to identify the mistake which is of high cost to the company.

 

A missed alarm will lead to a negative outcome in most cases. In most of the cases negligence or a lazy attitude did not trigger the awareness about the same.

Example: Missed alert in a financial company about the system issue can lead to data loss which is unacceptable.

 

However, both has its own importance with regards to the situation. A false alarm taken seriously will lead to creating an effective system(Natural causes like tsunami or hurricane). But a missed alert can cause fatalities(alerts about tsunami or hurricane)

Hence,

can not select over the other.