Leaderboard

As LSS professional often we find ourselves in a tight situation and complete deliverables under pressure. There is a chance that due to several external factors around us we end up Data dredging, also referred as "data fishing" which means analyzing data in such a manner so that possible relationships between data are somehow demonstrated. The effects are harmful because it defeats the purpose of true hypothesis testing. Some of the other terms of data dredging are “p-hacking”, “data snooping”, “fishing trip” and so on. For instance, we want to prove a hypothesis during a pre-project and post project improvement analysis, however the data doesn’t reveal so and we use a “cherry picked” sample which helped prove the point of improvement statistically. This would result in data dredging! In LSS world, unless factors are statistically significant, it doesn’t have the “value” and to prove the hypothesis using a statistical test quickly may end up in data dredging. Sometimes, unintentionally; more often a move made to close the case with some bias. It is easy to access large data set and perform analysis to come up with various relationships at random. Sometimes, data dredging may result in accidental correlation which otherwise may not have been identified. However, in our endeavor to research/analyze, it is important to recognize a valid relationship and focus on unbiased data set to arrive at accurate conclusions. The end results can be harmful in many ways: · Proved a hypothesis as statistically significant which may be later be proved as ‘not significant’ · Solutions are framed around a “so-called” significant cause whereas it may not help resolve the issue thereby becoming a questionable move later · Time/Effort spent would be a waste and end up being anti-LSS (Lean says reduce waste!) · Credibility of the professional may go down if practiced frequently and may put the entire organization in the wrong spot We can avoid data dredging by adopting practices like: · Ensuring data set is sufficient, relevant, representative, and not just a mere “subset” · Negotiate for adequate time, effort required for analysis and not perform under pressure, if we must turn around something quickly, we do so with a caution statement and not conclude too soon · Make data capture process accurate, robust, and exhaustive · Question the extreme values · Go with a balance of “data door” and “process door” approach in the project so that all possibilities are explored, and data/information are better presented for operational consumption without getting stuck in hypothesis testing · Keep it simple, use business sense as well to justify causation once we see correlation A scenario: The project lead shared the following data towards the end of project end for a review with the mentor: Pro-Project (AHT in mins) 20 Post Project (AHT in mins) 13 A better view for the project mentor would be the below table to mitigate data dredging as assess sustained performance: Pre Project (AHT in mins) 20 16 23 22 18 24 17 Post Project (AHT in mins) 12 16 13 12 14 13 11 Response is drafted basis relevance of Data Dredging typically in business process outsourcing.