â€œA point of view can be a dangerous luxury when substituted for insight and understanding. â€“ Marshall Mcluhan â€
We all have come a long way from industrial revolution where mass production and consumption was the order of the day. Over the years, the once not so knowledgeable consumer has transformed into a living storehouse of information. What has changed over the years? Why did the companies who were the industry leaders at one point of time failed miserably later on? What changed? The answer to all this lies in data. In our thirst to find newer markets for products and differentiate them from our competitors, we pushed the consumer towards seeking knowledge and then when he demanded, we failed to live up to standards.
We pushed the customer to give more importance to value i.e. value today has become emergent as compared to the old times when it was measurable. Now the new age customer wants more and he/she will switch, the moment he/she finds more value from another product or another competitor. How do we live up to this? Given the rapid changes in technology and the rapid adoption of management information systems by almost all industries, we need not look further. We have the solutions but we are too thick skinned to look at it. Todayâ€™s customer is like a walking hard drive with copious amounts of data. What we need to do is analyse it, find patterns where none exist, combine it with the experience of mankind and predict the needs and demands of future backed by strong data analysis.
If we stop and look back into the history what will be distinctly visible to us is the fact that companies that were present in the first list of Standard & Poorâ€™s 500 only 74 remain in operation today. Almost half of the 25 companies that passed the rigorous test for inclusion in Tom Peters and Robert Watermanâ€™s 1982 book â€œIn Search of Excellenceâ€, have gone bankrupt or have shut down due to poor performance.
The reason being that when organisations are thriving and successful, it creates an illusion that whatever gave them success in the past will work in the present and still remain valid in the future. They forget the fact that the world is constantly changing, constantly evolving and to survive they need to evolve with it too. Today there is no dearth of examples of companies that failed after achieving success on a grand scale, to quote a few, Wang Labs, Digital Equipment, Borders and Blockbusters and many more; of which many either fell after achieving industry leadership position or shrunk in size.
The reason is not that they failed to act or didnâ€™t know what was happening but the fact that they had all the data but didnâ€™t analyse it. The patterns were there, had they analysed the data they already possessed they would have seen it. They believed that experience can upstage data but they failed to comprehend that with todayâ€™s rapidly changing global economy, itâ€™s impossible for the human mind to absorb all factors that go into making sound decisions. Statistical analysis often uncovers correlations that no one could have predicted on the basis of their past experiences. We wonâ€™t be wrong in accepting the fact that data and statistical analysis isnâ€™t perfect but who hasnâ€™t made poor decisions on pure gut instinct!! It is said that sustainability is based on transforming data into analyzable information for insights and decision-making. This is where data driven decision making comes in and plays its part. (Cokins)
Take for example the case of Wal-Mart as reported in New-York Times. This occurred when Hurricane Frances was moving across the Caribbean, threatening to directly hit on Floridaâ€™s Atlantic coast. Residents were making a rush for the higher ground, but at the same time, far away, in Bentonville, Ark., executives presiding over the Wal-Mart Stores decided that the situation opened a great window of opportunity for their newest data-driven weapons, Predictive technology. A week ahead of storms predicted landfall date, Linda M. Dillman Wal-Martâ€™s CIO at that point of time asked her staff to come up with forecasts based on what happened when Hurricane Charley struck several weeks earlier. Their experts mined for data and found that the stores would need certain products more than the usual batteries, flashlights, and bottles of water.
All this brought out some very interesting facts about the pre-storm behaviour of consumers. Strawberry pop tarts and beer were the top-selling items. By predicting what was going to happen rather than waiting for it to happen, Wal-mart was able to deal with this unexpected situation in a holistic manner.
It is said that if we can improve our ability to estimate for a given customer, we stand to gain by applying the same logic to millions of customers. This applies to many other areas where intense application of data science and data mining is seen like direct marketing, credit scoring, financial trading, help-desk management, fraud detection, search ranking, product recommendation and many more. An example for this would be the instant recommendations of Amazon.com and Netflix which are a result of split-second advertising decisions based on the items a user has previously seen or is viewing right now, which has resulted in increased revenues for both of them.
The increasing amount of data generated by humans while doing their daily chores has lead us into a world where the purchases and habits of a consumer give us an insight into the behaviour of the consumer, how he reacts to various offers , what are his preferences, Is the customer a sort of person who will pay back loans. The algorithms have been designed and implemented which predict accurately how likely a person is to get diabetes. All this has a potential to give rise to platform of customization for every consumer i.e. mass customization.
A recent study by MIT professor Erik Brynjolfsson has brought out the fact that most revolutions in science begin with better methods of measurement. When we can see new things, we are driven to seek answers and thus build new ways of thinking and operating. The main underlying part of his findings was the fact that organisations that relied on data driven decision making, enhanced their performance by 4 â€“ 6 % vizâ€“a-viz their peers in the same market. This figure has remained robust despite taking into account the contributions for labour, capital, purchased services and traditional IT investment. (Lohr)
Another research conducted by Mr. Erik along with Mr. Lynn Wu, an assistant professor from Wharton Business School used publicly available web search data to predict housing-price-changes in metropolitan areas across the United States. They didnâ€™t possess any special knowledge of the housing market when they began their study, but they reasoned that virtually real time search data would enable good near term forecasts about the housing market- and they were right. In fact their predictions proved to be more accurate than the official ones generated by the National Association of Realtors which had far more complex model but relied on slow changing historical data. (Brynjolfsson)
Now if we analyze the business that exist today and have been running on intuition, tradition and convenience, we observe that they have scattered staff development programs, the budgetary decisions are based on prior practice and programs, Staff assignments are based on interest and availability and goal setting is based on votes , favourite initiatives or fads. In contrast, the companies that have adopted data driven decision making for the staff development programs are more focussed as an improvement strategy to address the problems identified by the analysis of data. Staff assignments are based on skills required for future as well as the present requirement. Goal settings are done on the basis of problems identified by data analysis and their possible explanations.
One factor that we must consider is that with the ever increasing rapidly changing pace of technology, companies today have multiple points of contact with the customer i.e. blogs, call centres , customer reviews or simple comments on social media platforms which are all a valuable source about client segments , insights and behaviour. Today it can be observed that increasing competition, decreasing margins and availability of information has left many companies struggling for survival. The only way out seems to stem from the fact that it is very important today to analyse the vast amount of data being generated daily and convert it into actionable knowledge so that client needs are addressed adequately. (Mike Lynch)
An example worth mentioning here is that of researchers at John Hopkins School of Medicine, who provide a classic example of how simple models and big data win over more elaborate analytic approaches. They used the data from Google Flu Trends, which is publicly available, to forecast surges in flu related emergency visits a week before similar warning were received from the Centres of Disease Control. Similarly, twitter updates provided accurate records similar to the official reports that were tracking the spread of cholera in Haiti after the 2010 earthquake. (Brynjolfsson)
From the numerous examples quoted above, it can be substantiated that data analytics unlock huge amounts of useful information, which can be made more transparent in nature by use of Big Data. It is often seen that we search for data in forms other than electronic and then transfer it to electronic form. This effort represents a huge inefficiency; if we could the information directly from other forms to electronic form without human intervention it could save us a lot of resources and time.
Our online financial transactions or any transaction for that matter, if and when collected over a period of time can bring out interesting facts about our choices and behaviour. Now if we apply the same to a huge organization, what we have is copious amount of data that can provide us with very accurate and detailed information about various aspects of the organization. This further helps the middle level and upper level management make meaningful and informed decisions regarding day-today decisions to designing long term plans for the organizations along with requisite data to support it.
Data collected from consumer transactions can help us predict the behaviour of the customers and help the organisations in maintaining requisite inventory levels. They can also tailor and customize the products and services according to the needs of an individual customer without the customer making an additional effect in telling the provider about it. It also leads to narrower segmentation of customers.
Data obtained when combined with power analytics techniques can be used to find hidden patterns which otherwise would remain hidden and help in predicting future needs , improvise after-sale services and meet the expectations of the consumers in a holistic manner. In short the transfer of information between consumer and manufacturer can be defined as the art of communicating; without actually communicating; as no formal/informal communication takes place between the partiesâ€™ involved; just observation, which leads to predicting the needs and aspirations of consumers. We need to understand that the data is not the only tool for the decision making. If applied in isolation without experience, the end result is a Frankensteinâ€™s monster of business process and best practices woven together. As value increases its dependency on unique knowledge and human experience, objective data sources must be supplemented with perspectives from organizational cultures, communities of interest and other relationships. Only then can we take holistic and sound decisions.
Bibliography (Debates, 2011)
Bonabeau, E. (n.d.). Harvard Business Review . Retrieved 11 21, 2013, from hbr.org : http://hbr.org/2003/05/dont-trust-your-gut/ar/5
Bono, E. D. (n.d.). Search Quotes. Retrieved 11 21, 2013, from searchquotes.com: http://www.searchquotes.com/quotes/author/Edward_De_Bono/3/
Brynjolfsson, A. M. (n.d.). Harvard Business School . Retrieved 11 21, 2013, from Harvard Business School Website : http://hbr.org/2012/10/big-data-the-management-revolution
Christensen, C. (n.d.). Design Intelligence. Retrieved from di.net: http://www.di.net/articles/the-innovators-dilemma-when-new-technologies-cause-great-firms-to-fail/
Cokins, G. (n.d.). analytics-magazine. Retrieved 11 21, 2013, from analytics-magazine.org : http://www.analytics-magazine.org/may-june-2012/577-corporate-decision-making-why-do-large-once-successful-companies-fail
Debates, D. (2011). When it comes to important business decisions , should you trust your gut or follow the numbers . Deloitte Development LLC.
Lohr, S. (n.d.). New York Times . Retrieved 11 21, 2013, from New York Times Website : http://www.nytimes.com/2011/04/24/business/24unboxed.html?_r=0
Mike Lynch, c. e. (n.d.). Financial Times . Retrieved 11 21, 2013, from Financial Times Website : http://www.ft.com/cms/s/0/62f37a4a-931c-11de-b146-00144feabdc0.html#axzz2lITs0aA5
Plisch, B. G. (2004, 02 04). WTN News . Retrieved 11 21, 2013, from WTN News Website : http://wtnnews.com/articles/557/
Tim McGuire, J. M. (n.d.). Ivey Business Journal . Retrieved 11 21, 2013, from iveybusinessjournal.com: http://iveybusinessjournal.com/topics/strategy/why-big-data-is-the-new-competitive-advantage#.Uo47PsXlbrN
Wladawsky-Berger, I. (n.d.). Irving Wladawsky-Berger. Retrieved 11 21, 2013, from Irving Wladawsky-Berger blog: http://blog.irvingwb.com/blog/2013/09/data-driven-decision-making-promises-and-limitations.html
Zwilling, M. (n.d.). Forbes . Retrieved 11 21, 2013, from Forbes.com: http://www.forbes.com/sites/martinzwilling/2012/03/25/too-many-leaders-still-rely-on-their-golden-gut/
Note - Visitors shall not be able to comment on this article until they are logged in.