No data but good judgement

Anil K · 2015-12-01T05:57:17+00:00

.demo { border:1px solid #C0C0C0; border-collapse:collapse; padding:5px; } .demo th { border:1px solid #C0C0C0; padding:5px; } .demo td { border:1px solid #C0C0C0; padding:5px; } Vishwadeep Khatri There are situations where data is hard to collect or the occurrences are too infrequent. Highly experienced managers, however, are happy providing their guess about data through rating scales. They believe that the ratings are pretty close to real data. As a problem solver, would you go with such ratings to find causes? Partho Banerjee LSSBB I don't think that ratings alone would be sufficient. In such cases. I feel a further drill down is required. Besides we can look out for measurables which are closely related (pre or post stage) to get some idea of the affected process. Shamik Kumar I understand that "not using human intelligence/ experience" is one of the wastes in lean. So we need to appreciate people having experience and providing feedback based on that. Now as a problem solver i would rather prefer to take their experience into consideration and try to validate it, if possible, than to simply ignore or believe it. Mayank Gupta I have done this for one of my projects with pretty good results. This is what we did * Validated the domain expertise of the agent providing the ratings * Took ratings for all the potential Xs where data was not available (prevented the bias towards a particular X) * Cross verified the ratings with a focused group So as a problem solver, I would go with such ratings, but one needs to proceed with caution and there are additional checks and balances that need to be taken care off. Manoj Singh In absence of data always trust your experiential logical regression ( domain knowledge). It really works Satish Karivedha When data is not available to understand root causes, then we relying on prediction theory. There is nothing wrong in going with ratings. While considering rating, ideally one should look at inputs coming in from SME's. Secondly larger samples will give good insights. In this entire context of findings, degree of risk is relatively is on the higher side as there is no data to prove. We are going with assumptions. Partho Banerjee LSSBB I agree with Satish and Manoj but what would be our approach to get more sampLEs as in this scenario the rating and that too from those who belong to the function or are affected by it(stakeholders) since they have already given their ratiNg. Who else should we look forward to for our study. Beverly Daniels If you don't have any data how do you know you improved the performance? I worked on a Problem that resisted 'solutions' for over 20 years. People guessed, used ranking, fishbone diagrams and voted on the most likely solution, called in subject matter experts, etc. None of the actions made the performance better. They didn't use any data. We got data. It took a bit of time, but we baselined the failure (Define phase), Determined that we could in fact measure it with some repeatability (Measure), ran a couple of experiments to recreate the failure in-house and understand the causal mechanism (Analyze), determined a solution that we proved with data would eliminate the failure mode (Improve), then added a set of controls on the critical inputs and the output performance to ensure that it didnt' re-occur. It took time to get the data but no where near 20 years of failed guessing without data. Now I'm not against the quick fix based on past experience or subject matter expertise as long as it's easy, cheap and quick to implement and I'll know rather quickly if the solution worked, but we need data for that last part don't' we?

December 1, 201510 yr

Quote

We had a recent discussion on this topic. All forum members are invited to continue this discussion here.

	Vishwadeep Khatri There are situations where data is hard to collect or the occurrences are too infrequent. Highly experienced managers, however, are happy providing their guess about data through rating scales. They believe that the ratings are pretty close to real data. As a problem solver, would you go with such ratings to find causes?
	Partho Banerjee LSSBB I don't think that ratings alone would be sufficient. In such cases. I feel a further drill down is required. Besides we can look out for measurables which are closely related (pre or post stage) to get some idea of the affected process.
	Shamik Kumar I understand that "not using human intelligence/ experience" is one of the wastes in lean. So we need to appreciate people having experience and providing feedback based on that. Now as a problem solver i would rather prefer to take their experience into consideration and try to validate it, if possible, than to simply ignore or believe it.
	Mayank Gupta I have done this for one of my projects with pretty good results. This is what we did * Validated the domain expertise of the agent providing the ratings * Took ratings for all the potential Xs where data was not available (prevented the bias towards a particular X) * Cross verified the ratings with a focused group So as a problem solver, I would go with such ratings, but one needs to proceed with caution and there are additional checks and balances that need to be taken care off.
	Manoj Singh In absence of data always trust your experiential logical regression ( domain knowledge). It really works
	Satish Karivedha When data is not available to understand root causes, then we relying on prediction theory. There is nothing wrong in going with ratings. While considering rating, ideally one should look at inputs coming in from SME's. Secondly larger samples will give good insights. In this entire context of findings, degree of risk is relatively is on the higher side as there is no data to prove. We are going with assumptions.
	Partho Banerjee LSSBB I agree with Satish and Manoj but what would be our approach to get more sampLEs as in this scenario the rating and that too from those who belong to the function or are affected by it(stakeholders) since they have already given their ratiNg. Who else should we look forward to for our study.
	Beverly Daniels If you don't have any data how do you know you improved the performance? I worked on a Problem that resisted 'solutions' for over 20 years. People guessed, used ranking, fishbone diagrams and voted on the most likely solution, called in subject matter experts, etc. None of the actions made the performance better. They didn't use any data. We got data. It took a bit of time, but we baselined the failure (Define phase), Determined that we could in fact measure it with some repeatability (Measure), ran a couple of experiments to recreate the failure in-house and understand the causal mechanism (Analyze), determined a solution that we proved with data would eliminate the failure mode (Improve), then added a set of controls on the critical inputs and the output performance to ensure that it didnt' re-occur. It took time to get the data but no where near 20 years of failed guessing without data. Now I'm not against the quick fix based on past experience or subject matter expertise as long as it's easy, cheap and quick to implement and I'll know rather quickly if the solution worked, but we need data for that last part don't' we?