Q 776. In the Measure phase, while measuring the process—like tracking patient wait times in a busy hospital—how do you pick numbers that actually show what’s going on, not just what’s easy to measure? What tricks do you use to catch bad or incorrect data before it throws off your whole project?

 

Note for website visitors -

• This platform hosts two weekly questions, one on Monday and the other on Thursday. 
• All previous questions can be found here: https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/. 
• To participate in the current question, please visit the forum homepage at https://www.benchmarksixsigma.com/forum/.
• The question will be open until Monday or Thursday at 5 PM Indian Standard Time, depending on the launch day. 
• Responses will not be visible until they are reviewed, and only non-plagiarised answers with less than 5-10% plagiarism will be considered for winner selection. 
• If you are unsure about plagiarism, please check your answer using a plagiarism checker tool such as https://smallseotools.com/plagiarism-checker/ before submitting. 
• All correct answers shall be published, and the top-rated answer will be displayed first. The author will receive an honourable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

• Some people seem to be using AI platforms to find forum answers. This is a risky approach as AI responses are error-prone because our questions are application-oriented (they are never straightforward). Have a look at this funny example - https://www.benchmarksixsigma.com/forum/topic/39458-using-ai-to-respond-to-forum-questions/

• We also use an AI content detector at https://quillbot.com/ai-content-detector. Only answers with less than 45-50% AI-generated content will be considered for winner selection.

Traditinnonally, ones will try to balance leading an lagging indicator like respectively number of patient arriving per hour and average waiting time for exemple running time series plot or triangulating different metrics.

 

But using AI could turn the lagging to leading indicator by using realtime ploting and dynamic analyse.

 

This will avoid acting late, using wrong data source or variation on sampling.

 

In Measure phase when we are tracking patient wait time, the easier metric to measure would be the time taken from registration to moving into the doctor's office. Instead, the key metric to be tracked must be the "Total time from patient arrival to seeing a doctor" or "time of arrival to admission" or "Time of arrival to discharge". This will help to identify the E2E process efficiency. 

 

To catch incorrect or bad data, some of the key measures that can be taken are:

• Clearly define what each metric measurement means. For ex: Patient wait time -- What is the starting point & what is the ending point. What are the stages and how do we ensure that data recording is consisitent
• Once the stages are identified, define the data collection at multiple points to ensure that there are no misses, people who collect data are calibrated. 
• Before deployment of the data collection plan, do proof of concepts to identify ambiguities in definitions, issue with data collection methodologies/tools
•  Once data is collected, data visualization techniques can help to identify outliers and patterns. In addition data collection errors also can be quickly identified. 
• Cross referencing the data across multiple stages. For ex: In a linear process of arrival, registration, tests, etc, the arrival time cannot be later than the registration time. Cross referencing helps identify anomalies. 
• Train everyone involved in data collection on the definitions, collection methods. 

 

Very basic thing in the lean six sigma to count on measure phase which is one of an important component. The goal of business excellence's Measure Phase is to quantify present performance in order to pinpoint areas that require improvement. At the beginning of continuous process improvement creating process maps, generating baseline measures, defining pertinent metrics (KPIs), gathering trustworthy data, statistical analysis, detecting performance and quality gaps, recording results, and involving stakeholders are all important tasks. Organizations may obtain insights that enable targeted changes and overall performance enhancement by measuring processes consistently. Collecting an accurate data sets and focusing only on process performance along with efficiency and quality improvment is a main objective of measures phase.
To get a comprehensive insight of process performance and pinpoint areas for improvement, the Measure Phase is essential. Organizations may create a strong basis for further stages of development by carefully carrying out this phase, making sure that any adjustments are data-driven and in line with strategic objectives. More client happiness, higher quality, and more operational efficiency are the ultimate results of this.

How to Pick the Right Numbers:

When I measure patient wait time, the temptation is to just pull what is easily available (example: Time from check-in to doctor consult). But that may miss important context or root causes.

 

To ensure my metrics reflect the real process:

• Start with the SIPOC & Process Map
  • This helps to visualize where delays happen, so I don’t just rely on one data point. May be the wait starts before check-in.
• Use the Voice of Customer (VOC)
  • If patients complain “I waited for long time before anyone response me in helpdesk”, then I need to measure time from entry to first contact, not just from check-in.
• Focus on CTQs (Critical to Quality elements)
  • Understand “What makes or breaks the experience?” A single long wait might impact satisfaction more than average wait times.

Break Down the Wait Time into Key Components:

Instead of measuring one overall figure, I would track:

• Time at registration
• Time waiting for nurse
• Time waiting for doctor

This breakdown helps identify specific bottlenecks in the process.

 

Balance Leading and Lagging Indicators

• Lagging: total wait time
• Leading: number of staff on shift, patient volume/hour

This gives early warning signs.

 

Tricks to Catch Bad or Incorrect Data

• Gemba Walk or Time Study
  • I will observe the actual process for a few patients. Compare this to what is recorded in the system. Discrepancies are red flags.
• Check for Missing or Impossible Values
  • Negative wait times? 0 minutes for a complex step? We could spot these using basic filters in Excel.
• Run a Control Chart or Histogram Early
  • Abnormal spikes, flat lines or bimodal distributions might mean inconsistent measurement or data entry errors.
• Logical validate
  • Compare time stamps from two sources.
  • For example:
    • EHR vs manual logs
    • Front-desk data vs patient-reported times
• Use Operational Definitions
  • Everyone must record data the same way. “Check-in time” must be clearly defined. When patients enter the door, or when they are logged into the system?
• Pilot Before Full Measurement
  • I will try data collection method on a small sample. Fix issues before scaling.

Quick Example:

If we track only “average wait time” from EHR timestamps, but the nurse doesn't log start time until after triage, we miss the true wait.

 

Solution:

Add a “patient greeted” timestamp by the front desk as a new data point based on what we learned from process observation and VOC.

Measure phase is the crucial aspect in a Lean Six Sigma project. It can be tempting to choose easily measurable data, but if it doesn't accurately represent the actual process, our improvement efforts could fail.

Let's break this down into two sections:
-How to Select Metrics That Truly Reflect the Situation
-Begin with the Voice of the Customer (VoC) and Process Mapping

Utilize VoC and process mapping (like SIPOC or detailed workflows) to determine critical-to-quality (CTQ) metrics.
For instance, when monitoring patient wait times, the CTQ should be “the time from check-in to seeing a doctor,” rather than just “the time from check-in to triage,” which may be easier to obtain but less relevant to patients.

Concentrate on Process Pain Points and Bottlenecks
Measure at known stress points (e.g., the wait time between discharge orders and patient transport) as these often reveal inefficiencies more effectively than overall metrics.

Balance Data Quantity and Quality

Don't only concentrate on averages

Keep track of outliers, medians, and ranges Relying solely on the mean obscures operational concerns, for instance, if the average wait time is 20 minutes but varies from 5 to 90 minutes.

Select Leading and Lagging Indicators Together

Number of patients awaiting beds is the leading factor

Average Length of Stay (LOS) is lagging indicator

Techniques to Spot False or Inaccurate Information Before It Impacts Your Project Give each metric a precise operational definition. For instance: "The EHR timestamp indicates that the patient wait time starts when the patient checks in at the front desk and concludes when a nurse begins triage." This stops different data collectors from interpreting time points differently.

Conduct a Data Validation Drill
Select a small sample (10–20 records) and manually check them against source data (like medical records or EHR logs) to catch discrepancies early.
Look for Data Anomalies
Be alert for impossible values, such as negative wait times or wait times exceeding 12 hours

To find outliers, use box plots or histograms

Execute a Gemba Walk

Watch the procedure unfold as data is being gathered. This often uncovers deficiencies in the data that the machine has gathered or concealed alternatives.

Put a Data Collection Plan into Action

Implement by identifying who is responsible for measurement, what should be measured, the methods of measurement, and the sources of the data. This makes data sourcing more consistent.

Check for Repeatability and Reproducibility (R&R)

Have two people measure the same data points on their own If their findings diverge, it suggests that terminology or procedures need to be clarified

Summary

The usefulness of data should not be confused with its availability. The existence of a time field in the EHR does not guarantee that it is a suitable or trustworthy measure for the CTQ.

Pick metrics that reflect the customer experience and process goals (e.g., total wait time, not just time at reception). Use sampling, data validation rules, and cross-checking with other sources to catch incorrect data early.

 

 

We should make sure the metrics being measured align with the hospital goals and objective of the project.

Get stakeholder buy-in.

Define key indicators like eg -patient satisfaction and relate that to the metrics being measured.

Ensure that data input is standard and consistent, train and correct where needed

Audit the data in the initial phases to see if it aligns to the output

Automate where possible for reducing human error

 

For this situation, In measure phase we can do below -

1. Focus on metrics which are directly associated with CX, eg lead time, wait time,

2. Avoid putting too much efforts on metrics which do not have significant relationship to process, eg- # of patients in a day

3. Use a combination of metrics to identify relationships. Eg- paitient wait time along with Patients VoC and Diagnosis.

 

To catch bad/incorrect data-

- The data entry tools for staff can be forced to accepted data basis historical experience/trends. Any outliers must be highlighted by system at time of entry an requires a forced entry, if needed.

- Such anomaly of data must add a flag in the system for subsequent stages of patients journey for review and validation.

 

In the Measure phase it’s easy to grab whatever data is easiest to collect, like timestamps from a system. But that doesn’t always tell the full story. To really understand what’s going on, we may need to focus on metrics that matter, not just ones that are convenient.

For example, instead of just tracking how long patients wait from check-in to being seen, we might also want to measure how long they wait after being seen, or how long it takes to get test results. These might require a bit more effort to track, but they paint a much clearer picture of the patient experience.

As for catching bad data before it messes everything up—one way is to always do a quick sanity check. If a number looks way off (like a patient supposedly waited 0 minutes or >5 hours), flag it. Also, look for patterns: if one nurse’s entries always seem faster than everyone else’s, maybe there’s a data entry issue. Visualizing the data—charts and graphs can make outliers and inconsistencies jump out in a way raw numbers don’t.

In the given example scenario, tracking wait times in a busy hospital.

In this case the problem statement to be defined, let's say, is "Customer dissatisfaction because of long wait time during the visit."

 

The values that are easy to measure in the process, such as

1. Number of patients seen per hour states that total volume of patients

2. Time from check-in to discharge states total time of the patient in the hospital includes wait time, treatment time, test time, etc.

To pick the meaningful number in the process, let's break down the total wait time of patients when they enter the hospital.

1. Time of arrival at reception

2. Patient Vital Test

3. Waiting room (to consult specialized doctor)

4. Pre-Consultation Time

5. Medical Test Time (if required)

6. Post-Consultation Time

7. Pharmacy Wait Time

8. Discharge Time

By collecting the data in this format, it will show where the wait time gets increased in the routine cycle.

It will be used to analyze and improve the process by reducing the possible wait times at each section.

Keep monitoring the KPI and KRI in the process.

KPIs such as average wait time to see a doctor, average wait time for a particular test and results, and number of patients who leave without being seen.

KRI such as number of nurses/doctors on duty, lab turnaround times, and patient volume by hour.

 

Tricks to catch bad data:

1. Ensure the person who is collecting the data is well trained and has a complete understanding of the process so none of the data will be missed during collection. A pilot test run of the complete process is to be done to validate it before implementing it in the live environment.

2. Use data validation logic at the time of data processing; for example, a patient's discharge time cannot be before their arrival time, and a patient's wait time can't be negative.

3. Randomly collect data and check its accuracy.

4. Analyze the data in the chart format to check the pattern that does not fit the process. (Usually the wait time of the patient to see the doctor is 20 to 40 minutes, but some of the data shows 200 minutes.)

5. Automate the data recording process and validate it from time to time. If any data entry is manual, train the staff to record accurately.

 

The journey of pateints need to be map to understand the touchpoints - before picking the metrics study the whole floe from arrival to dischage

Pick Metrics with Context Avoid generic “averages.” Instead:

Use median to reduce the impact of outliers.

Add percentile metrics (like 90th or 95th) to understand the worst-case experience.

All Layer in stage-wise times (e.g., waiting for triage vs. waiting for a doctor)—these pinpoint bottlenecks.

Include Volume and Capacity Measures Numbers like patients per hour or staff coverage ratios give  to wait times. A spike in wait might not mean a process failure—it could be understaffing.

The Measure phase is the one of the steps in the DMAIC framework i.e. Define, Measure, Analyze, Improve, Control.

Goal of the measure phase are - Map the process, identify what to measure, collect accurate data, Establish a baseline


Two points matter for measure phase are -

1)Choosing the right metrics 
* Align metrics with process goal - For patient wait times, the focus is typically patient satisfaction, clinical outcomes, or operational efficiency.
* Use VOC approach
* Don’t settle for what's easiest if it doesn’t reflect reality.


2) Catching Bad or misleading data
To avoid being misled check below points-
*Audit your data early
*Cross-check sources
*Use visual tools and direct observation to validate it.

When you are measuring a process like patient wait time in a hospital. It is easy to collect data like scheduled v/s actual appointment times but its difficult to know the true patient experience or identity area of loopholes.

 

For having the correct picture and improve the quality of hospital services, we need to have strong clinical governance system where we have a systematic process for patient visit. We need to Map the entire Process from check in to seeing the doctor including all steps such as registration, doctor consultation and lab test etc. We need to have a mechanism for time in and time out for each step and should be measured with benchmark.

We should have a seamless feedback mechanism to understand the pain point of patient which can be used as tool for improvement of the customer experience.

 

We should have an appointment scheduling system to reduce waiting time and upgrading the waiting time experience by digital update about their number and so on.

We need to Analyze patient wait time and enhance their experience by regular updates and feedback sessions.

 

The most important steps to catch bad or incorrect data before it throws off your whole project is to regularly monitor that the steps implemented is followed and audited on regular basis. We should plan surprise and internal audit regularly. We should not rely on one source of truth and have cross mechanisms.

 

Nidhi

In my opinion, in a hospital (that too a busy one) to check and track patient wait times there would be multiple sources to feed-in the data we are looking for.

 

A patient's viewpoint, a hospital staff's view, and finally a third party's viewpoint. All these viewpoints may differ in experience and priority.

 

for example:

1. For a patient there may be multiple wait time segments - waiting the lobby, waiting in the examination room, waiting in the treatment room etc.

2. The hospital staff like doctors, nurses, receptionists, facility management folks, etc. would have their own observation from their point of view as to which section or segment of someone attending to the patient is causing a hindrance/ obstacle

3. A neutral party like ours will have a different view altogether from what we see, when the data is collected with date and time at each section/ station right from the reception (lobby area), waiting in the examination room and finally attended by a doctor to further collecting medicines from the pharmacy.

 

Once these various readings are taken a consolidation would give us a wider factor that contribute to the wait times.

 

Tips, tricks, guidelines:

* All numbers/ time slots need to be counted in one unit (e.g., minutes only,  hours, hours only, seconds only)

* Sampling needs to be done - time of the day, number of sample patients, days of the week etc.

* A standard process simulation should be done with all operators/ data collectors so that no to less room for error even at the data collection phase

* Double check/ review the reviewer how all operators are collecting the data - whether they are following a standard process as discussed in the above pointer

* Conduct a mini simulation of the outcome/ observation and share with the hospital staff before considering the same with a larger data for the project to avoid any outliers/ missed out steps/ etc.

Way to decide on what metrics to considered for Measurement:

• Measure the complete process starting form patient check-in to discharge from hospital.
• CSAT survey results or ratings based on patient experience
• Define a target time and measure the average wait time of patients. This will help to re baseline and plan improvements to reduce the wait time .

Recommendations to catch the bad data :

• Any major fluctuations (drop or raise) in wait time.
• Compare the data with other internal sources to cross verify the individual data collected.
• Conduct pilot runs and analyze the data ,before rolling out complete deployment in one go.

Interesting insights provided by all respondents to pick the right numbers and metrics in the Measure Phase.

 

The best answer has been written by Thaiyeb Hussain. Well Done!