Q 683. What is a Botched Run in Design of Experiments (DOE)? How does it affect the output of the study? What steps will you take to prevent them during DOE?

 

Note for website visitors -

• This platform hosts two weekly questions, one on Tuesday and the other on Friday. 
• All previous questions can be found here: https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/. 
• To participate in the current question, please visit the forum homepage at https://www.benchmarksixsigma.com/forum/.
• The question will be open until Tuesday or Friday at 5 PM Indian Standard Time, depending on the launch day. 
• Responses will not be visible until they are reviewed, and only non-plagiarised answers with less than 5-10% plagiarism will be approved. 
• If you are unsure about plagiarism, please check your answer using a plagiarism checker tool such as https://smallseotools.com/plagiarism-checker/ before submitting. 
• All correct answers shall be published, and the top-rated answer will be displayed first. The author will receive an honorable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

• Some people seem to be using AI platforms to find forum answers. This is a risky approach as AI responses are error prone as our questions are application-oriented (they are never straightforward). Have a look at this funny example - https://www.benchmarksixsigma.com/forum/topic/39458-using-ai-to-respond-to-forum-questions/

• We also use an AI content detector at https://crossplag.com/ai-content-detector/. Only answers with less than 15-20% AI-generated content will be approved.

Botched Runs: Throwing a Wrench in DOE Results:

 

While Design of Experiments (DOE) is a powerful statistical technique used to understand the cause-and-effect relationships between different factors and how they impact a process or system, there are Achilles’ Heel to DOE project successes as well like Botched Runs, unforeseen phenomena, limited resource etc.

 

Let’s deep dive on Botched Run:

 

Botched run refers to an experimental trial where something goes wrong, causing the data collected to be invalid or unreliable for analysis. A Botched run occurs when an experiment with in a DOE project deviates from the planned conditions. This can happen due to various reasons impacting the overall results of your DOE study.

 

Reasons for Botched Runs: Below are some of the reasons for Botched Runs

• Equipment malfunction: A faulty sensor, incorrect calibration, or unexpected equipment failure can lead to inaccurate measurements or incomplete data collection.
• Human error: Misreading instructions, setting incorrect parameter values, or mishandling materials can all contribute to botched runs.
• External factors: Changes in ambient temperature, unexpected power fluctuations, or even contamination of materials can introduce unwanted variables into the experiment.

Impact of Botched Runs:

• Inaccurate/unreliable outcome: Since the data from a botched run isn't representative of the actual effect of the factors being studied, it can skew the overall results. This can lead to misleading conclusions and hinder the ability to identify the optimal settings for your process.
• Reduced Statistical Power: DOE studies rely on a statistically significant number of valid data points for robust analysis. Botched runs effectively reduce the usable data, potentially weakening the statistical power of the study and making it difficult to draw definitive conclusions.
• Wasted time and resources: Botched Runs leads to "Muda of rework", this will lead to waste of time and resources as the outcome is not reliable and the whole experiment has to be repeated. 
• Loss of Momentum and Morale: The frustration caused due to Botched run may demoralize the research team. This can lead to a decrease in their motivation and potentially hinder their efficiency in moving forward with the project.

 

Below are some precautions to be taken to minimize the occurrence of botched runs in DOE:

• Thorough Planning: Clearly define the experiment's objective, factors, and their levels. Plan the number of runs considering potential for errors or unexpected events.
• Meticulous Setup: Ensure equipment is properly calibrated and functioning correctly. Double-check the settings for each run to avoid errors.
• Data Recording and Monitoring: Record all observations and measurements meticulously. Regularly monitor the experiment to identify any deviations or unexpected behavior.
• Standardized Procedures: Develop clear and detailed protocols for conducting the experiment. Train personnel involved to ensure consistent execution.
• Pilot Run: Consider performing a pilot run with a limited number of trials to identify and rectify any potential issues before the main experiment.
• Contingency Plans: Have a plan for handling minor equipment malfunctions or unexpected situations. This might involve having spare parts or rerunning specific data points if possible.

By implementing preventive measures and having a plan for handling Botched Runs, companies can ensure the integrity of their DOE studies and make informed decisions that optimize their processes and avoid costly mistakes.

DOE helps us to identify the relationship between cause and effect. It provides an understanding of interactions between causative factors and helps us to determine the levels at which the controllable factors need to be set to optimize reliability.

It is mostly used by the engineers in the manufacturing industry to maximize yield and decrease variability.

Botched Run is when the experiment is not conducted properly. This results in datasets which are inconclusive and not possible to analyze.

 

Effects on the output of Botched run:

·        Inaccurate data

·        Unexplained variability

·        The output of the overall experiment may not be accurate

 

Steps to prevent Botched run:

·        The experiment needs to be planned thoroughly, considering all possible variables

·        SOP’s should be created for all experiment procedures

·        All the people involved in the experiment should be well trained

·        Monitor the experiments continuously and document any variations

 

Botched Run in designed of experiment refers to kid of mistakes which change original recipe and may affect the test results. for example we testing effect of 7 Different spices on taste of a Briyani Rice. Lets say in one of our test we accidently add too much chilies, more than the recipe called for. This mistake changes original recipe and may affect the test results. However as long as the changes are not too extreme, we can still analyze and learn from experiment. This kid of mistakes called as " Botched Run" .

 

In DOE, Deviation from planned settings can impact the study's output in serval ways.

1. It may altered results by reflecting true effect of the factor being studied.

2. precision of the experiment's result might be reduced.

3. Overall validity and reliability of experiment can be questioned.  

 

To prevent these errors in  Design of Experiment ( DOE) you need to take following preventive measures.

 

1.  SOP for experimental procedure. 

2. Training and detailed planning on the SOP and execution of SOP.

3. Pilot run if possible

4. Regular monitoring of experiment in progress to catch deviations early and correct them.

5. System to report, highlight errors as soon as they occur.

 

Above steps can help to maintain reliability of experimental results.

The winning answer has been provided by Jayanth. Well done!

 

P.S. Nowadays most statistical packages have the ability to deal with botched runs (in case one occurs in your DOE). So we might not be required to redo the whole experiment.