Skip to content
View in the app

A better way to browse. Learn more.
Benchmark Six Sigma Forum

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.
Click here to know about CAISA - Certified AI Solution Architect

Articles

Manage articles
Show all categories

Induction program with quality session

I observed that the scope of six sigma has increased it is now applicable in many industrial sectors. When Bill Smith created this methodology his main focus was defect reduction and improvement which brings excellence and process improvement, later on Quality formed the core of other methodology/framework too.
 
I have seen that the approach of employees toward quality management has not yet changed, they still consider this as the job of quality unit, as for them quality means compliance, reporting, QMS manual and most importantly Audits. I found that organizations are spending good amount of money in quality related training/workshops/paperwork etc. but still are not successful in changing the approach of individuals. At high level, every organization’s policy talks about quality and its importance but at ground level due to fixed processes, departmentalized structure it become a task of small number of people, like managers and specifically of those who belongs to quality department, since it’s their job.
 
Induction program- By definition an induction program is an important process for bringing staff into an organization. It provides an introduction to the working environment and the set-up of the employee within the organization. An induction program is part of an organizations knowledge management process and is intended to enable the new starter to become a useful, integrated member of the team.
 
But I believe now organizations have a need to adopt a new approach of induction program:
Organizations should provide Process Excellence/Quality training to each and every employee at the time of induction, this training they can design as per the requirements of their LOB (Line of Business).  
 
Also when induction of an employee gets conducted at that time we can tell employee that they are suppose to share their views (positive or negative) after six months from now about quality of process/operation to which they belongs (preferably through a presentation) so that this training of quality not become just a training and they feel involved with the things they have learned.  
This kind of treatment at induction will make employee responsible for his own quality as well as quality of his team and organization.
 
Why this should be done at induction program because when we start with induction:
 
1. The new employee will feel himself linked with the quality department.
2. This will make new employee feel that he is responsible for the quality of operation to which he belongs.
3. This will convey a message to employee that organization has a serious approach towards quality.
 
This approach will bring mutual benefits and this also links with the high level objective of each and every organization that is “alignment of individual objectives with organization’s objectiveâ€.
 
Now the question comes since induction is for new employee what will happen to those who are already working? For this we need not to do anything but to ask them to attend this “Quality session†which we are conducting for new employees and they can join this from there and then.
 
I believe that each and every individual should feel himself responsible for quality and there is a need to make quality a routine habit rather than an event.
DevsenaMishra

SMART GOALS

SMART Goals
Once you have planned your project, turn your attention to developing several goals that will enable you to be successful. Goals should be SMART - specific, measurable, agreed upon, realistic and time-based.
A goal might be to hold a weekly project meeting with the key members of your team or to organise and run a continuous test programme throughout the project.
The acronym SMART has a number of slightly different variations, which can be used to provide a more comprehensive definition for goal setting:
S - specific, significant, stretching
M - measurable, meaningful, motivational
A - attainable, agreed upon, achievable, acceptable, action-oriented
R - relevant, realistic, reasonable, rewarding, results-oriented
T - time-bound, time-based, timely, tangible, trackable
This provides a broader definition that will help you to be successful in both your business and personal life.
When you next run a project take a moment to consider whether your goals are SMART goals.
To quote renowned American philanthropist Elbert Hubbard:
SMART Goals Specific
Well defined. Clear to anyone that has a basic knowledge of the project.  
Measurable
Know if the goal is obtainable and how far away completion is. Know when it has been achieved.  
Agreed Upon
Agreement with all the stakeholders what the goals should be.  
Realistic
Within the availability of resources, knowledge and time.  
Time Based
Enough time to achieve the goal. Not too much time, which can affect project performance.  
 
 

Keval

Gut Feeling Vs. Data-Driven Decision Making: Is There Actually A Battle?

I tried to keep the topic neutral because neither did I want one to decide against reading the article based on gut feeling nor based on previous data analysis of topics. I myself am a firm believer of the two approaches; however, I believe people tend to take either of the sides without realizing that the two approaches are not at odds.
A situation that is frequently cited by proponents of use of intuition for management is when the decision-makers do not have access to extensive data. This may be the case when the environment has changed significantly (a simple scenario that may be relevant at an individual’s level can be a decision to invest money). The decision in such situations may not seem to be driven by data due to non-existence of structured data. However, what often seems like intuition is well-rehearsed behavior that comes from the years of experience. Not being able to present a pattern or inability to articulate a recognized pattern (in cognition) does not imply that it is a gut-based decision. The cognitive pattern is built upon the extensive data of experience and knowledge accumulated over the years by the decision-maker. However, data-driven decisions may not always be the way out of a situation. People who are not brain-washed or are not entirely influenced by the data can see things in an entirely different perspective and can generate ideas that expertise may drive out of one’s brain. This may be seen in situations of business crisis – where to turnaround a business the top management is changed and new people from within the company or outside is brought to bring in an entirely fresh perspective.
 
The problem with gut feeling is that we all already know of a lot of cognitive biases and related flaws which may impact the decision making. There is a lot of research that would yield “gut-feeling†approach to be highly susceptible to these flaws:
confirmation bias – one’s difficulty in believing and remembering evidence that contradicts one’s beliefs
fallacy of centrality – this is very important for the decision-makers to understand where they believe that they will know about anything and everything that is important.
 
Although I believe in gut-feeling, the biases make me apprehensive of using the same for decision making – I fear my unknown biases creep in the decisions. Thus, intuition, according to me, works only for the people who act on their beliefs, but always doubting what they know. Such decision-makers would always be on a look-out for some contradictory evidence. This would in turn create an environment where others around them would also challenge their beliefs and result in constant updating of the “basis†of intuition. Few decision-makers that may fit this particular mindset could be Brad Bird from Pixar and Andy Grove - CEO of Intel. However, there are other interesting cases which, unlike the popular belief, have had seen errors in their judgment or in decisions based on gut-feeling. Steve Jobs made this mistake. Google believed that they were going to crush Firefox with Chrome, but it has not happened. The market share of Chrome still remains modest. This is not to bring anything negative about these two decision-makers or others. All of them have extremely impressive track records, but needless to say their experiences also include some failures which are usual for a human as well as a human-run organization. We need to understand that there are conditions where judgment may fail, and it holds true for the best of the leaders and managers. As is the case for intuition, similar arguments would hold true for data-driven decisions where decisions solely based on data may fail in absence of intuition.
 
Finally, in my view, gut-feeling and data-driven analyses do not oppose each other. These may be looked at as tag team partners to achieve the best decisions. Hunches may be necessary to generate the initial hypotheses or even alternate decisions, but evaluating the same based on hard evidence (both quantitative and qualitative data) would help to achieve better results vis-à-vis decisions solely based on hunches and ingrained behaviors. Following gut-feeling mindlessly may result in believing that the process is impervious even after observing clear signs of failing. The gut-feeling may be captured in the qualitative data analyses, but in a much more structured and streamlined manner which also helps in easy identification of a pattern and repeated usage of the same, if required.
 
To be driven by gut-feeling does not imply foregoing the advantage that structured data can provide. A move towards data-driven decision making while not shifting from the gut-feeling process is important in this highly dynamic environment.
 
References: http://bobsutton.typepad.com/my_weblog/2009/11/intuition-vs-datadriven-decisionmaking-some-rough-ideas.html
 
 
 
Note - Visitors shall not be able to comment on this article until they are logged in.
prateekchaturvedi

Data Driven Decision Making - Injecting Rationality In Your Gut Feeling

 
Data Driven Decision Making - injecting rationality in your gut feeling
 
 
 
 
Sector: Banking Sector in India
The data centricity of banking industry is the universal truth. Traditionally banking has been the one sector which handled the maximum data about any person that are particularly critical and dear to those persons as they deposit their trust with the banks in the form of their finances. In the recent times this has been further reinforced with the RBI implementing the “Know your Customer†mandate that aims to compulsorily maintain the customer data which should be relevant, concurrent & authentic. Despite the proliferation of such data, effective analytics and data mining techniques has been at its elusive best.
 
 
 
The information industry has grown leaps and bounds and the remarkable advances in analytics software and its processing power aided by the cloud computing systems is just the tip of huge iceberg of potential that such data is capable of achieving.  
As the industry tries to grow out of the recent financial crisis towards the shady future of uncertainty, banking and retail banking in particular must inculcate the power of analytics in them to be able to improve decision making, indulge in constant innovations which ought to become the bread and butter for survival & be more compliant with the stringent financial regulatory environment that the RBI is supposed to impose for greater control. The siloed approach to banking should give way to enterprise wide resource planning (data being the most critical resource) for fostering greater transparency, efficiency & effectiveness through integration and unified image of the entire sector. This will also help in garnering greater customer trust & rejuvenate customer relationship which is the single most critical factor for survival in times of uncertainty, mistrust & risk.  
The recessionary trends have forced the clientele of banks to a more frugal approach to managing their funds. Careless consumption has been replaced by need based one and “ROI†has suddenly become the buzzword which never had such a great reputation except amongst the business houses. However it is interesting to note that despite reduced spending the world has not stopped itself from the adoption of latest technologies. Be it smart phones or social media presence the huge numbers are truly defiant of the existing economic conditions and its implications. Such behaviour re-confirms the value of innovations in today’s society besides such channels could provide source of huge data tapping which can help retail bankers to provide a more rewarding experience to their customers enhancing their brand loyalty. The usage of “Big Data†as the new window to the world of increased productivity, innovation & competition is important to be considered here.  
The rapid adoption of analytical tools would help banks process the information they have into market knowledge which would enable them to differentiate themselves through service excellence. It may sound contradictory that previous paragraphs talked of unified image and integration and now differentiation is promoted. Well, competition has been and will always be the root for future growth without which the need for existence of mankind comes under the radar. Rather we should look at a new dimension of competition – “Competition through Cooperation†where competitors would be on the same page with respect to technology and new inventions yet they would have to constantly evolve themselves to be relevant.  
Advanced analytics provides the banks with a new path of continuing business by overcoming the obstacles of risk and uncertainty, the prime growth drivers being stricter regulation, better risk management, effective strategising and stronger CRM. The various ways to achieve data salvation is revealed as below: The analytics software would speed up the financial and risk reporting services as required by the new norms as and when implemented by the government ensuring service delivery with no or minimal cost. The usage of Enterprise wide data architecture would provide a single version of banking creating transparency and restoring customer confidence. Data crunching would enforce better risk management by identifying malicious transactions and preventing its recurrence. Usage of technology to combine past and current data can help in predicting future scenarios with greater accuracy and provides an opportunity to face the shady future in a planned manner with confidence. Besides the data analytics tools may be used to boost revenues as well like, Customer data analytics – enhance service and bring more clients Investment analytics – improve lending process Process analytics – find process inefficiencies and take corrective measures thereby reducing costs, to name a few.  
[*]Data collection from various sources like KYC, social media websites etc... and it analysis using Big Data and relevant technologies can help in providing customized banking solutions, new financial products to suit customer needs & gather feedback on marketing campaigns launched. This would lead to greater customer satisfaction and tighter relationship.
 
 
Mobile banking is the new brainchild of the banking sector that allows customers carry out transactions on the move. This means greater volume of transactions to be handled and the usage of analytic software to integrate data across channels become essential. Also multichannel banking is constantly evolving with the endeavour of providing cross channel banking across websites.
So far the discussions lead us to the conclusion that usage of Data Driven Decision Making through Data Analytics & ERP is imperative to the future competitiveness in banking industry. But there are major speed breakers in the path to this rediscovery, which are as follows:
Modifying existing IT infrastructures and the corresponding data migration might incur substantial initial costs Using analytics at the strategic level would require identification of relevant data and standardization of processes and data structures Resolving frequent data issues & inconsistencies that exists in the customer data in the baking domain Required expertise in analysing data points, process expertise & technical expertise is important Support & Initiative of key stakeholders  
Finally the embracing of analytics as a service depends on the internal culture and dynamics of the organization. Hence to successfully implement the same nurturing of the employees to convince them of the power of data driven decision making is very crucial. However the conviction in employees can be developed only if the leaders & top management of the enterprise believe in the vision of “Analytics as the future of bankingâ€. Hence the purpose of this article would be to inspire the top management, so that they can realize the importance of using data in their organizational decision making and inject rationality in their decision making.
 
Note - Visitors shall not be able to comment on this article until they are logged in.
Partha Mitra

Data Driven Decision Making (Dddm)

 
Data driven decision making (DDDM)
 
 
As it is rightly said by Plato, “Necessity is the mother of inventionâ€; it comfortably fits in the life and business perspective. In today competitive world, we cannot even think of succeeding over others if we are not better than them. It is applicable in both day to day lives as well as in Businesses. In day to day life, we have numerous examples of us competing with our colleagues for ranks, posts or various other things. But it is the candidate which has better skills and qualities will always succeed. This thing is also applicable to a larger extent to businesses where every day lakhs and crores of transactions are performed. Any organization should use all its data available judiciously and all the decisions should be based on these data instead of personal beliefs. This process of making decisions based on data is called Data-Driven Decision Making.
In a recently published article, “Data Science and its Relationship to Big Data and Data-Driven Decision Making,†Foster Provost and Tom Fawcett define Data-Driven Decision Making as “the practice of basing decisions on the analysis of data rather than purely on intuition.†Equally succinctly, they view data science “as the connective tissue between data-processing technologies (including those for big data) and data-driven decision making.†This DDDM is being viewed as a tool to help people make smarter, more effective decisions.
 
Also according to Electronic Learning Assessment Resources (ELAR), a DDDM focus uses student assessment data and relevant background information, to inform decisions related to planning and implementing instructional strategies at the district, school, classroom, and individual student levels. Even the concept of Data literacy meaning “a person possesses a basic understanding of how data can be used to inform instruction†is closely inter-weaved with DDDM. We can say Data Literacy as an underlying technique of use of DDDM.
 
Considering the reference from the research paper “Strength in Numbers: How Does Data-Driven Decision making Affect Firm Performance?†by Erik Brynjolfsson, MIT & NBER Lorin Hitt, University of Pennsylvania and Heekyung Kim, MIT. A detailed survey data on the business practices and information technology investments of 179 large publicly traded firms, it was found that firms that adopt DDD have output and productivity that is 5-6% higher than what would be expected given their other investments and information technology usage. Such surveys and studies have every now and then showed the importance of data in taking important managerial decisions.
 
Even the share-market is not luck or belief based. It is based on various complex logics which have to interpreted using different other factors. So it is the need of the hour to work on the data analysis for having better forecasts, demands and market scenarios. If we see all the prospering companies of the world, they go by numbers. It is the challenge on the part of the management to lead the organization towards data-driven decision making. This DDDM is important because of the following reasons:
 
 
 
1. Commodity priced computing  
2. Massive file system storage and retrieval technology  
3. Bandwidth  
4. Smart devices: Records are everywhere  
Keeping in mind these important factors, it become necessary for any company to take decisions very precisely as each and every decision has very long term effects on the company and its revenues. Recently, so many technologies have evolved including Big Data which have made the analysis of data far easier as it was earlier. Now even small information which is sort out of crap data is very useful for the organizations in taking future decisions. In last few year many new organizations had come up which provide services in the field of data analysis which indirectly helps the companies hiring them. Lots of social networking sites provide some data which are used by these analysts to provide related advertisements to the people. This explosion of decision making from personal instincts to data driven can be largely attached to Big Data.
 
With the advent of Big Data, this has come out even more drastically and most of the companies shifting towards it. Lot of money is being invested in getting meaningful data out of bulk of data available in the companies. It’s not surprising that data-driven decision making is one of the most promising applications in the emerging discipline of data science. It has an explosive growth.
 
There are large numbers of characteristics of Data which have to study before taking certain decision. These include variety, volume, velocity, veracity, variables and sources. In finding meaningful information from the raw data the following steps are to be followed:
 
 
 
1) Collect all data from various sources  
2) Create a file of raw data and arrange properly  
3) According to predefined index, interpret the data to make a data file  
4) Analyze this data file generated  
This complete procedure is a basic process of DDDM. It has to be followed if the accurate analysis is required.
In a nutshell, we can conclude that this data driven decision making is the need of the hour and each and every company should move towards it as soon as possible. It may be looking a tedious and unnecessary at the present moment but its long term effects are very soothing and beneficial for the entire company.
 
References:
· http://blogs.wsj.com/cio/2013/09/27/data-driven-decision-making-promises-and-limits/
· http://online.liebertpub.com/doi/full/10.1089/big.2013.1508
· www.a51.nl/storage/pdf/SSRN_id1819486.pdf‎
· https://www.950.ibm.com/events/wwe/grp/grp017.nsf/vLookupPDFs/Michael%20Kowolenko%20Presentation/$file/Michael%20Kowolenko%20Presentation.pdf
 
Note - Visitors shall not be able to comment on this article until they are logged in.
Aakash Varma

Data Driven Decision Making

In the FMCG sector, our company is rated among the top 5 organisations both by the consumers and industry. Much of the success is attributed to the increased consumer confidence and round-the-clock shelf SKU availability of our product line. The role of our Purchase Division has always been at the core for achieving this mark. Over the years our team under your guidance has been able to keep the inventory at optimum levels thereby helping the Production Unit to make delivery on time. It is due to the careful execution of our fundamental function - 'Material Management' which ensures that raw material is procured from the suppliers at the right time and in right quantities. This is the key to success in our sector and thus any minuscule improvement towards this can play an important role in raising our organization's bar.
Our well maintained database of daily Material Management activities can help us in this direction for making more accurate estimations. An approach-'Data Driven Decision Making' can be applied in which assessment data and background information can be used to take decisions related to planning activities. According to a study by MIT Center for Digital Business, organizations driven most by data-based decision making had 4% higher productivity rates and 6% higher profits. With data driven decision making we can deploy Just-In-Time as approach and Material Resource Planning as the method for waste minimization in the purchase cycle.
 
To demonstrate the strategic, operational and financial advantages of DDDM approach, we can consider the following conventional process being carried out at our unit. Under usual circumstances, we have to keep a safety stock of inventory items so as to counter the market and supply chain uncertainties. These may include logistics delay, plant failures, supply side variability, demand side variability and others. This leads to estimations made with the help of brainstorming methods within the team and experiential knowledge that cumulatively determine the amount of inventory to be ordered. With the help of estimations, performance is delivered in terms of fulfilling various conditions like on time delivery, safety stock, safety time related to inventory items. But there could occur estimation errors during the process due to which stores or warehouses can have lower or excessive amounts of inventory. This can affect the overall process efficiency both in monetary and operational terms. To improve this scenario, Data driven decision making mode can be deployed. Considering the situation that we faced last month. The production unit had given a demand forecast of 100 packs for one of our products and keeping safety stock levels in mind, we had ordered 120 packs. But we received only 115 packs from the supplier due to production based variability at his end which had not been a factor of consideration in our process. Also, the actual demand turned out to be 118 packs which was closer to our safety stock levels. So, overall we faced a shortage of 3 packs. From this we can identify that there is a chance of around 4% variability in delivery from supplier's end. This data if incorporated in taking decisions before ordering again from the same supplier can help us in achieving more accurate outcomes. This was an example of countering lower inventory received from the supplier where DDDM could have helped otherwise. Considering another situation that occurred 6 months back where we had ordered around 200 packs where as the production had ordered for 120 packs only but finally picked up around 150 packs from the warehouse. We had a rough idea of up-scaling from demand side as well as in the costs from the supplier's end due to which we increased the order. Though a more accurate estimation using DDDM would have given an additional gain by reducing the wastage of the remaining underutilized 50 packs as well.
 
Using DDDM as one of our modus operandi would be convenient and beneficial too as its output depends on the quality of data gathered and a well managed database already in place can help us in reaping maximum benefits out of this investment. Further its effectiveness depends on defining the questions to be considered before analyzing the data and with your experience level in this area, we can easily frame pertinent questions to get relevant results from the data. Data driven decision making would give us added advantages of faster processing, refocusing our resources to increase the yield, relevant data backing to explain our rationale behind purchase decisions to the management, foreseeing the opportunities and threats in the market and overall supply chain. Over a period of time, this approach can also lead to building a reliable group of suppliers giving us a competitive advantage gained by adopting a data backed strategic purchasing model. An early adoption of Data driven decision making would bring maturity to our supply chain infrastructure and resilience towards unforeseen circumstances so that we can quickly respond to them without compromising on financial and operational aspects.
 
Note - Visitors shall not be able to comment on this article until they are logged in.
ssinghal

Data Driven Decision Making

Data in today’s world, means a lot to people and their lives as such. Data represents the raw information (dimensions and facts, in technical terms)
related to a particular task or an activity obtained from various sources which helps the user to summarize an event, to provide meaningful information
and to derive at a particular decision or a conclusion. People have become increasingly dependent on the data provided by many sources, that whatever
decision they take up in their lives be it purchasing a furniture or a high end car in the market, depend on the data that support their decision. Thanks to
the advent of internet tools which allow people to make informed decisions hassle free. If the word “data†is put under search in Google Ngram viewer
we could observe that more and more people have incorporated data in their decision making processes.
Data has enormous potential for analyzing what has happened in the past, what is currently happening (including minute to minute updates) and what is
gonna happen in the future. What Oren Etzioni was able to do in 2003 is a classic example of the power of intelligent usage of data and its hidden
potential. He published a paper “To buy or Not to buy - Mining Airfare Data to Minimize Ticket Purchase Price†which could help people when to
buy an air ticket at the lowest price in a time period. Microsoft immediately acquired the idea and the website and the idea is now part of Bing Travel.
What our own Raghuram G.Rajan was able to predict in 2005 about the then impending Economic Crisis is also an example of the uncanny ability to
unlock patterns which form part of the real world.
 
Just as people started using data, companies also started doing so and benefited by retrieving efficient results in their businesses. Today a number of
software packages are available to suit the needs of companies of all sizes. And the way data has been collected, stored, analyzed and presented has
changed throughout the years making the power of data and information inevitable for running even smaller businesses. Technologies
have improved so much that companies are switching over from small ETL (Extract - Transform - Load) packages to complex ERPs
(Enterprise Resource Planning) softwares and Big Data tools like Hadoop, unmindful of the costs involved, just because of the fact that meaningful
information processing is critical for their revenue and business models. It is imperative that behind every companies’ move to take any decision in
their day to day or long term operations, there is enough data to support the same. This is quite evident from the number of IT service solutions
companies burgeoning every day helping multi-national corporations of all verticals to perform efficiently.
 
Thus decision making till some time back was an art, but now it is becoming a science which every company is trying to master at with the help of
data. And in this competitive environment where businesses are fast paced, dynamic and ever-expanding it is of paramount importance that data driven
decision making should be part of any business to operate, expand and flourish.
 
 
Note - Visitors shall not be able to comment on this article until they are logged in.
Vignesh Venkataraman

An Age Of Truckload Information

Once when asked what he does, if the data does not support his decision,
John Maynard Keynes replied – “I change my opinion. What do you do?â€
In a haystack of information today, that one thing which helps organizations take sound decisions is the ‘analysis of data’. Often, companies find themselves in situations where from a variety of choices, they need to pick one. In such cases, data-driven decision making enables following a systematic procedure. A successful completion of any process begins from a decision well made. It forms the first step of any execution process, and is thereafter followed by modification, as and when changes in information arise.
 
The question that one would ask then is – ‘If it is data that is needed, then exactly how much of it?’. Authors like James Taylor and Stephen Covey, in their writings explain keeping ‘the end in mind' before undertaking a course of action. They say that the goal is never to build on the data; rather it is to use the facts to make work easier. Thus, the perfect quantity would be one which helps an organization make ‘timely’ as well as ‘correct’ choices.
 
But even a manager’s power to predict can do this job, isn’t it? On digging deeper we realize that the ‘intuition' we are referring to is nothing but the gut feeling that arises based on a manager’s experiences of the past, and thus its own roots lie in data as well. What people actually follow is ‘informed intuition' – that uses previous occurrences as its basis. This is justified, since not only is complete information necessary, but also alongside is corporate alignment and clarity.
 

 
These days, a term that managers often hear is ‘big data’ – which refers not only to the volume of data available, but also to the variety of it and the rapid pace at which it alters. Big data brings with itself the complexities of processing and interpretation, causing confusion and delays. It is here that just-the-right-kind of filtration is needed, to separate what is relevant and what is not.Infact sometimes, even lesser amounts of data can lead to better decisions being taken. As they say, the ‘first impression’ can indeed be beneficial if taken as the ‘last impression’. This is exactly where the use of instinct comes into picture. Whether it be studying a consumer insight or predicting the future, it is a blend of analysis and a manager’s intuition that leads to the apt solution. In a post in Forbes, Robert Carraway, a professor at the Darden School of Business said that big data and the increasing use of frameworks require not less, but a higher amount of managerial insight to accompany them.
 
There have been faults based on judgment (remember Google claiming to overtake Firefox when launching Chrome?) ; and so exist popular crisis due to over-reliance on data. The idea hence is to strike a balance between these two seemingly different ways that managers use to reach a conclusion. The more an association can accommodate diversity in terms of style, emotions and experiences, the higher is the probability of improved performance.
 
In a nutshell, if a corporation can make sure that it has as members both left-brain and right-brain thinkers – it can strike the nail on its head!
 
(*source for the cartoon : Google images)
 
Note - Visitors shall not be able to comment on this article until they are logged in.
  • Teaser Paragraph:

     

     

     

     

SUHANI POPLI

Data Driven Decision Making (Dddm): An Introduction To How It Affects An Organisation Future

How do companies make decisions? A simple answer to this crucial question can be: in two ways – one based on experience and intuition; and the other backed by data to validate its credibility. Whereas the former is a traditional way of making decision, but is wrought with drawbacks like inaccuracy, incomplete, the latter is a rather newer concept (called Data-Driven Decision making).
  Definition  
A recently published article by Foster Provost and Tom Fawcett define Data-Driven Decision Making (DDDM) as:  
 
“The practice of basing decisions on the analysis of data, rather than purely on intuition.† 
To explain it, an example of Wal-Mart can be cited, where it used past data to make decisions about what items to stock in the stores before an oncoming storm, based on past purchase records in other Wal-Mart stores before oncoming storm in that area. A simple intuition based approach would very well have predicted that merchandise items like torches, batteries and bottles of water would be sold most before storm – “common senseâ€. What it would fail to predict would be the amount of increase in sales due to the hurricane, i.e the quantity to be stocked. This is where past data would come to the rescue. Also, there could be astonishing discoveries made during the analysis of past data, e.g. the sale of strawberry Pop-Tarts increased by as high as seven times the usual in other stores. And that, the pre-Hurricane top selling product was Beer - a finding, nothing but a data driven approach can make.
 
Productivity Increase
Data-driven decision making (DDDM) is usually undertaken by an organization as a means of gaining a competitive advantage. A study conducted by a team of researchers of MIT on 179 companies concluded that “firms that adopt DDDM have output and productivity that is 5-6% higher than what would be expected, given their other investments and information technology usage.â€
This distinction of 5% is substantial enough to separate winners from losers.
 
P&G, which is the world’s largest FMCG company did sales worth US$84 billion last year, is a data driven business. The company touches the lives of close to 4 billion consumers every day. One cannot achieve this feat merely by experience driven approach, without relying on data. Visualising data with comprehensible patterns helps the heads of the respective country, category or brands identify which products are working, and which aren’t, thus finding places where new opportunities lie, all this in real-time.
 
The success of the data-driven decision making approach is dependent upon two factors: i) the quality of the data gathered; and ii) effectiveness of the analysis and interpretation of the results.
 
There cannot be more striking examples of companies, whose business is driven by DDDM, than Google and Facebook. Their advertising is based on examining, analysing, and exploiting the online data. Be it displaying ads of products and websites similar to past search history on Facebook, or be it sponsored ads on Google related to the search query, all credits to the data analytics.
 
That’s not all. Software giants like IBM, Microsoft, SAP and Oracle have sensed the future of the DDDM industry, commonly called “Analytics†or “Business Intelligence (BI)â€, wherein companies help others to find patterns in their data, to generate meaningful information. IBM expects this industry to grow to US$16 billion by 2015.
 
Applications
In addition to the Google or Facebook examples, where the data is used for marketing purpose, DDDM helps a business in all of the following areas:
 
1. Marketing: DDDM helps a business to compute returns from a particular promotion, fine tuning spending to yield better ROI, tracking social media, etc.
2. Sales: Finding best path and best practices, cost of customer acquisition, process improvement, Y-O-Y turnover, sales analysis, etc.
3. Inventory: As with the Wal-Mart example above, it helps project to a considerable accuracy the inventory to be maintained, monitoring and adjusting inventory levels.
4. Human Resource: Managing recruitment process, monitoring and managing attrition rates, employee turnover, etc.
 
Another example, from a very different end of business spectrum, is the one shown in the movie Moneyball, wherein the manager of a baseball team uses statistical data to analyze players he picks and place value on them accordingly. The team makes straight 20 wins, confirming the role of Data in making winning decisions.
 
Zipcar, a US based carsharing company is the world's leading car sharing network. Founded in 2000, it is valued at US$500 million now. The company has more than 810,000 members and offers nearly 10,000 vehicles throughout the US, Canada, the UK, Spain and Austria. The feat was reached only by analysing the data generated at every touchpoint between the company and its members, and crunching the data to generate useful information to make personalised future contacts with the customers.
 
Incorporation with Traditional approach
Does that mean there is no role of experience, intuition, gut feelings, personal beliefs in decision making? Gabriel Eckert, CAE, executive director of a leading organisation and co-author of From Insight to Action: Six New Ways to Think, Lead, and Achieve, in an interview to Associations Now says,
 
 
 
 
 
“...decision making is where statistics meet the senses. It’s where research combines with gut reaction. It’s asking ourselves, What does the data mean? ... There’s really three different ways intuition comes into play... It’s leveraging your experiences, it’s recognizing patterns, and it’s understanding the whole of the connections among decisions that we’re making . † 
This means that the best approach is dynamic decision making process, which incorporates the data analytics with the experience of those making the decision.
 
Conclusion
Data-driven decision making is the need of the hour for any aspiring organisation. When the world is moving very fast, not embracing the newer technology will leave our organisation in the dark ages.
Our Unit has a perfect blend of individuals with a great experience in their field and good analytical skills. Hence, incorporating this contemporary mode of decision making into the unit, and thereby into our organization will be the best step towards prosperity.
 
Note - Visitors shall not be able to comment on this article until they are logged in.
rohang

Data-Driven Decision Making

A write-up to influence my manager – a gut-feel decision maker with a decade of experience – who is a non-MBA yet has a good skill-set, towards adopting DDDM. Key decisions are made by unstructured brain-storms and personal belief, and DDDM is vehemently opposed.
Over the last ten years, you have been doing a wonderful job as a manager. This is especially remarkable, considering your advocacy of instinct-based decisions. The bulk of today’s managers wouldn’t be as lucky though, to be endowed with such a definitive gut for decision-making. Nevertheless, there are several of them – with much lesser experience and exposure - who have apparently climbed the ladder faster and have stayed steady in their lofty roles. And there is only one factor that seems to have been the difference – devotion to evidence over instinct.
 
I totally understand your apprehensions about data-driven decisions, and with due respect, I request you to hear out my explanation. You might be an adversary of data-driven decision making, but any dignitary of your capacity will certainly be a strong adherent of reason and evidence. All of what I say is based on just pure and plain reason, more than anything else, and you will agree with me that data-driven decision-making is nothing but corporate pragmatism in its most palatable form.
 
The primary problem with data, much to the annoyance of managers, is the hefty costs incurred in collection and analysis of evidence. However, the paybacks and benefits offered by data analysis hugely offset the initial costs of analysis and databasing. It can be safely asserted that all multi-national companies have adopted data-analysis practices right from the operational level up to the strategic level. This has been the cornerstone of their survival stories along varied cultures, business models and tactical transformations across geographies. What might be a sunk cost on the balance sheet might actually be an asset to the fortunes of the company and its long-run sustainability.
 
There is a very valid argument about how today several companies fail in spite of using data-driven decision-making – I’m sure it has been a discerning insight in your mind too. It is one thing to gather and store data, and a completely different thing to analyze the collected database. Poor analysis of data is akin to searching for something in the wrong place. Carefully devoted time and effort on data-analysis has fueled the recovery and resurgence of several Fortune 500 companies from the brink of bankruptcy.
 
On closer observation, I find that the root of aversion to data-driven decisions has been more of an inexpressible psychological factor than anything else. The technicalities involved and directness expressed in evidence-analysis has fueled a feeling of insecurity among traditional decision-makers, who are staunch believers in their strong management foundations drawn by rich experience.
 
In reality, adoption of data-based decision making initiatives is not a means of discounting, but a strong facilitation of core management practices. Any gut decision backed up by appropriate data makes for a solid case since it adds a measure and credibility to the result. Data does not define the direction of action; it merely indicates the consequences of all possible paths taken. It is not a traffic signal, but a map. Also, data-driven decisions are not necessarily defensive. This is another common misconception; all that data-analysis does is to help take calculated risks.
 
You will appreciate that today’s businesses are steered by forces that are completely different from those a decade ago. Competition is getting tighter and survival tougher as newer and more agile enterprises are mushrooming in a bid to cash in on hitherto unseen growth-rates. Flexible operations and non-conventional marketing tactics are enabling these companies to gradually eat into the market-share of the incumbents. The only way to survive, therefore, for an established firm like ours, is to constantly top our performance by maintaining our competitive advantage. And the first step towards identifying and nurturing our competitive advantage is data-analysis on our core competencies.
 
We have to face the fact that whatever be the domain of profession, analytical models work better than human calculations and perceptions. Any sort of development or application of models begins with collection and analysis of data. With the advent of affordable technology, devices and programs have become increasingly more compatible to design models and programs. This has led to an unprecedented level of information processing, leading to a knowledge race, rather than a technology race, amongst competitors. As a result, firms that are too rigid to evolve from their legacies are under severe threat – invariably, these threats percolate to key stakeholders in each organization. Going global will demand us to learn and understand the tone of modern business. And all of today’s businesses across the globe speak one language: data.
 
Have you ever wondered why a bulk of our review meetings and internal conferences tend towards stalemates and end up with you having to take a call on the stakes? Any viewpoint without a firm backing of data is deemed as baseless and subjective. Even a surmountable, innovative solution will cease to be appreciated simply because people’s perceptions are different. An evidence-based approach, on the other hand, makes a strong pitch as it directly presents the consequences in plain, direct numbers – in black and white. Arguments and alternatives are better received, and every conference will lead to a consensus, not just a conclusion. Disillusion and skepticism at our board meetings will be replaced with confidence and clarity. After all, what is an organization but a set of reasonable men working towards a common goal? It is only natural that our reasoning be shaped by rationality and pragmatism, in the best interests of our stakeholders.
 
Decisions made based on data-analysis exude reason and rationale. This is the bare truth.
 
Thank you for hearing me out! I’m assured that if you have thoroughly considered my statement, you have already begun to appreciate the imperative to take a data-based approach to decisions. It is in the cause of our sustainability: as individuals, and as an organization.
 
Note - Visitors shall not be able to comment on this article until they are logged in.
srikanth sriram_58698

Data Driven Decision Making Towards Gaining Competitive Advantage

Data Driven Decision Making Towards Gaining Competitive Advantage
Being an MBA student for the past one and a half years I have learnt that it is extremely important to have a point of view about everything and it is equally important to be able to put forward it in front of people, effectively. But how do we do it? We are able to do it when we have with us substantial arguments to support our point of view. Some might believe that the best decisions are sometimes made based on one’s “gut feeling†but we should not forget that disasters happen more often than not, when due diligence is not done before arriving at a decision. Decisions that are made on the basis of data are easier to carry forward and defend in case of opposition. Also, the person making the decision is more confident of attaining the goal because the decision is supported by a thorough research and analysis. As we gain experience in a job, the decisions are taken increasingly on the basis of experience. We are likely to become more complacent, and expect others to follow whatever they are told. But we must ask a question to ourselves- Is our past experience enough, to enable us, to make decisions in this ever changing business environment?
The truth is while we want people to follow us, we have to become good followers first and the interesting part is that we do not have to necessarily follow a person but a thought. A thought can be to always follow some rules before making a decision. Once we fully submit ourselves to these rules, we can become someone who will be readily followed by others. Even if a person feels that there is no need to follow rules due to immense experience and power that one possesses, one should try to substantiate his/her arguments with the help of data because ultimately he/she is working for the growth of the organization which has many stakeholders apart from him/her. Hence, the decisions involving so many people should not be made on pure instincts.
 
Another advantage of Data Driven Decision Making (DDDM) is that it always helps us in deciding the next step in the project that we have undertaken. If we follow instinctive approach decision making then it is quite likely that we will take an instinctive reaction at each and every stage of our assignment. Also, DDDM reduces the surprises that we encounter moving forward when we implement our decisions and thus removes the element of uncertainty that might force us to take unaccounted decisions. It helps you to land up in a familiar territory where we can rely on the data we possess and reduce the chances of committing errors.
 
Data gives us the power to represent and then implement our thoughts and beliefs. In a meeting also, it is always better to have a discussion on the basis of some data rather than an unstructured brainstorming. True, brainstorming helps in finding out unbiased solutions to problems but data gives a direction to the discussion, and ensures that the solutions remain within the framework of being relevant.
 
Information is available everywhere and we must try to gain as much of it as possible, because it is never too much. A learned person commands respect and obedience from one and all. The importance of data is so much that some companies like Bloomberg have made a business out of providing relevant data to companies, and companies that recognize data’s importance do not mind paying the price for acquiring additional knowledge, because they acknowledge the competitive advantage that data will provide while facing their competitors.
 
It is always easier to follow our instincts and make decisions quickly but what is difficult but more logical is to take a more pragmatic approach and invest time in researching about the consequences of different decisions we take.
 
Following a DDDM approach also makes people more accountable as the results of their actions are recorded in a data warehouse and is available for viewing of the manager at any given time. Hence, it is easier to figure out where the bottlenecks are and corrective steps can be initiated. This approach is far more superior to the random unstructured approach of evaluating performance. DDDM brings about an element of transparency in functioning of the company and in turn increases the employee commitment as they know that their work will be recognized because of presence of independent data driven method of performance evaluation. When employees believe that their growth is linked to the company’s growth they work harder to achieve the stated goals.
 
Whenever a company targets another company for an acquisition, it is preceded by a thorough due diligence process wherein the acquiring company studies the finance structure, culture, cash flows, and future synergies among other factors before it makes up its mind to go ahead with the acquisition.
 
It is a fact that one cannot predict the future but what is also true is that we can always prepare for the future, looking at the past. Hence DDDM is a wise approach to proceed because we never know what we may encounter in your journey. The precedents that have been set help us in avoiding the fatal mistakes that have been made in the past. “I know all†approach is not a trait of a good manager and as it is known to all that the day one becomes complacent, he/she is doomed.
 
In conclusion I would say that no decision is right or wrong on the onset, it is the consequences that decide its correctness. But a decision made on back of the supporting data is far more likely to sustain itself in the turbulent times.
 
Note - Visitors shall not be able to comment on this article until they are logged in.
simplysam17

Data Driven Decision Making

The fortunes of any form of business on this planet are correlated with the accuracy of the management decisions. Ironically, the toughest job of the management is to make an accurate decision owing to volatile economic conditions, cut throat competition, ever changing consumer demands, regulatory frameworks and constant technological transformations.
Intuition and Experience have long been the primary weapons in a manager’s arsenal to help him reach to a decision. But it was essentially because of the absence of sufficient and reliable data and information. In the present era, where the volume, variety and velocity of data is massive, personal experience and insights are no longer sufficient in reaching a conclusion.
 
 
 
 
 
 
 
 
The following conclusions can be drawn based upon the above results:-
Personal intuition influences decision making to a great extent 80% of the time, more than any other decision making process factor Decision makers seek validation from peers and colleagues 50% of the time 1 out of 3 times, decisions makers make decisions without any data support or analysis  
Intuition, experiential judgement and gut feeling have their own place in decision making but if we couple it with data analysis, it can help the managers to identify the most optimal and smart decision leading to new business opportunities and competitive advantage. The significance of data driven decision making is highly evident from the fact that out performing organizations recognize and pursue data driven decision making much often than on par and under-performing organizations. Not only their profitability is high but the associated business risks and uncertainties are also minimal.
 
 
 
 
 
 
From the above graph, it is clear that successful and leading organizations are leveraging upon data driven decision making, especially at enterprise level, at a much larger scale then the under-performing organizations. Intelligent enterprises leverage information to reach optimal decision in a faster manner and moreover to the outcomes that are most expected.
 
 
 
 
 
 
 
 
Again it can be observed that out performing organizations recognize the value of information improvements and are hungry for more. All the critical divisions of an under-performing organizations are stuck in the bottom left quadrant where perceived benefits of information improvement as well as perceived gaps in information are low. This leads to a stagnant growth with lack of innovation and the organizational becomes unsustainable in the long run.
 
However this potential source of competitive advantage is now increasingly being recognized by organizations at all levels and the focus is now shifting from the traditional decision making approach to a new modern approach as show below:-
 
 
 
 
 
 
 
 
With the possession of right kind of data available, application of smart data analytics lead an organization to:-
Discover insights by capturing real time data and synthesizing information from variety of sources Predict Outcomes by consolidating business sub systems and establish an integrated view of the business Act faster by understanding consumer preferences and environmental changes  
The above discoveries enable an organization with threefold immediate and strategic advantages:-
Profitable Growth by improving opportunities to discover new customer base, uncovering their preferences leading to better customer relationships, identification of new potential markets and development of right product and services Cost optimization and efficiency by optimal allocation and deployment of resources and capital leading to higher operational performance Proactive risk management by better predicting the outcomes and identification of risk at early stage which gives the organization enough time to respond and act  
Organizations with focus on data and extracting key information and patterns out of it are already ahead on the growth curve as compared to their competitors. They are inherently employing the below depicted framework to breakaway from the clutter and uniquely differentiate themselves with respect to their supply chain, marketing, financial, human resources and other processes. The six dimensional framework for a data driven decision making organization are represented and explained below:-
 
 
 
 
 
 
 
Aware - Gathers and senses all the structured and unstructured information from each point in the organization. Also helps in gaining insights from previous unquantifiable and unusable data. Linked - Integrates internal and external functions across all geographies, departments, business units and teams to align business activities from front to back of the organization. Precise - Uses the most relevant, updated and quality information to make timely decisions by the organizational management. Questioning - By deriving new insights and uncovering hidden patterns, it challenges the status quo and creates new opportunities for the organization. Empowering - Enables and extends the employees' memory, insight and reach as well as the authority to decide and act. Routine decision making is left to the automated systems enabling the employees to make higher value decisions. Anticipating - Instead of just reacting to the environment or correcting the mistakes, it focuses on the future to impart resilience and steers the organization amid uncertainties by evaluating trade offs.  
 
 
 
 
To conclude, we can say that the role of data and information shall shape the future of businesses through smart and intelligent decision making by enabling deeper customer understanding, enterprise visibility through a robust supply chain, optimize real time decisions through better predictability, maximizing customer service, reducing business risk etc. Organizations have already began to seize the opportunity to utilize data and information analytics for business advantage. Leaders are no longer relying just on intuition to fill information gaps. Instead they are coupling their management acumen and expertise with the unique data insights to make decisions in an entirely different way. They are able to draw out the specific, precise, relevant and contextualized information to predict the most likely outcomes of their key decisions. Given the nature of today's highly competitive business environment, no business can afford to leave benefits like these on the table. Only those organizations that can skillfully adopt, integrate and deploy the benefits of enterprise wide data analysis and optimization will be prepared to shape their own future.
 
 
Note - Visitors shall not be able to comment on this article until they are logged in.
Tarun Kumar Jain

My Experiments With Data Driven Decision Making (D3M)


As a Computer Illiterate growing up in the new millennium, I had tremendous difficulty making sense of how to do most of my stuff. Most of my decisions were snap, on-the-moment and intuitive ones. By the time I was in my teens, I had learnt my way around the big computer problem. My experiences and the Great Indian “Jugaad†mind-set had provided me with a sufficient enough method wherein I only worked on the computer when absolutely necessary. If there was any other way to do the task, any escape route, I gladly followed it.
By now you must have realised that I was and am sceptic and data-averse. Data to me was always those unreadable files that eat a hell lot of my storage space. I was gaining proficiency in getting around this data problem and along came Big Data. Everyone from Google to Barrack Obama was using it.[1] As a student at one of the most prestigious B-Schools in the knowledge market, not only was I supposed to know what it was, but was expected to be able to tune in and utilise it to make a difference. Life has strange ways of getting back at us, mere mortals, and here was my customised gut wrenching sucker punch. The fighter in me knew I had to do this, but I did not find a way I could.
 
A 3 month long research in the field led me to Data Driven Decision Making or D3M. Simply put, it is the pleasure of sipping a coffee in your office chair while you watch your computer work up some algorithms and provide you invaluable decision making tips to face some of your most frequently encountered problems.
 
Immediately after, I experimented with the different sorts of decisions that can be made with D3M. The results were a true eye-opener. If we think of decision making as a broad spectrum from operational decisions at one end to strategic decisions at the other. Operational decisions can be characterized as highly structured, routine, short-term oriented and increasingly embodied in sophisticated software applications. On the other hand strategic decisions are taken by the top management and serve to set the long-term directions, policies and procedures of an organization. They tend to be complex and unstructured because of the uncertainty and risks that generally accompany longer term decisions. In between these two extremes, we can have varied decisions including non-routine ones in response to new or unforeseen circumstances beyond the scope of operational processes, and tactical decisions dealing with the necessary adjustments required to implement longer term strategies. [2]

 
Half a decade ago, D3M could have only helped you with the more structured forms of decision making but no more. With the advent of Big Data, machines know much more about humans and human behaviour than humans themselves. Sample this, personal analytics can actually allowed me to analyse my Whatsapp chat history and find why most of the girls I chatted with refused me a date. The word cloud usage showed me that my texting skills were uninspiring to say the least with the most common words being “ok†and “yaâ€. I realised I needed to be more creative and engage better. What did not help was that most girls chatted in the window of 8-12pm whereas I had the habit of taking a short nap at the time. Also it helped me identify my most productive work hours and helped me plan out my work better.
 
Thus far, all seemed well with D3M but on further research most early moving corporates into the field were still undecided on its benefits. A closer look exposed a distinctly similar pattern in them. Most early moving managers thought that D3M will help them save money or time or both. Research shows that they are misguided to say the least. D3M does neither, atleast not in the short term. What D3M does allow though is discovering solutions you never knew existed. What it will do though is finding needles in the haystack consistently. Another interesting insight was that D3M depends a lot on the data collection. Great collection leads to great results. All we are required to do is ensure data cleanliness, variety and velocity.
In his book, Data Driven: Profiting from Your Most Important Business Asset, Prof. Thomas Redman summarizes the whole decision making via data as “Good decision makers follow at least three Bayesian principles. First, they bring as much of their prior experience as possible to bear in formulating their initial decision spaces and determining the sorts of data they will consider in making the decision. Second, for big, important decisions, they adopt decision criteria that minimize the maximum risk. Third, they constantly evaluate new data to determine how well a decision is working out, and they do not hesitate to modify the decision as needed.â€
 
After months of thorough investigation and experimentation, I have arrived at this conclusion, “To stay a step ahead in this ever competitive world, using Data Driven Decision making is a must. So let D3M take care of all the external data that you need to work with and let your mind focus on understanding “the data from insideâ€. That will surely lead you to not just success but contentment.
 
References
1. http://swampland.time.com/2012/11/07/inside-the-secret-world-of-quants-and-data-crunchers-who-helped-obama-win/
2. http://blogs.wsj.com/cio/2013/09/27/data-driven-decision-making-promises-and-limits/
 
Note - Visitors shall not be able to comment on this article until they are logged in.
NACHIKETA .

Data Driven Decision Making

Decision making is the inevitable truth of the life of every practicing manager. The best of the corporate leaders constantly emphasize on the fact that the litmus test of a manager is to take correct decisions in the difficult times. Management styles of the managers have varied widely and there has been always a contest between the merit of data driven decision making and the intuitive method of decision making. So any attempt to reach a conclusion about the greater efficacy of one over the other or the chance of success of one over the other lies in accessing the arguments made in support of each of them.
The intuitive managers would argue that this style of decision making is justified because it is not a fluke. Rather it is the gut feeling of the manager which he has acquired out of experience. Further such decision making is quick and less rigorous to reach conclusively for the decision maker. Also there are examples which prove that this method has produced unexpectedly great results. But does this certify the method beyond doubt?
 
Even when claim of intuitive manager regarding generation of unprecedented, unexpected great result is accepted yet this argument is incomplete. This is because the frequency of the success rate in the above stated intuitive method can always be questioned. The decisions taken intuitively are equally likely to produce highly good and highly bad outcomes. On the other hand the decisions taken based on data are likely to produce outcomes with lesser variance. In their case the outcomes are expected to be far more standard. An analogy can be drawn with the individuals trained in the discipline of management and individuals who do the business based on their experience and not on the basis of tools, techniques or methods taught to them. While the latter might produce very high outcomes, the former can be expected to produce standard set of expected outcomes on all instances. The risk involved in the case of the former is bracketed and is well within bounds.
 
 
This helps us further argue the case that the data driven decision making helps organization to create a standard process for carrying out decisions making. This makes the company less dependent on skillsets of a given manager rather it makes the skills ubiquitous through the designed processes.
 
We have examples of successes in decision where the manager is guided by gut feeling and intuition. Hitler in the battlefield, a manager at war or Indian cricket captain M.S. Dhoni, a manager in the stadium or Steve Jobs, a manager spinning the invention wheel without any empirical evidence of demand are all such examples. But there are also examples that did just the contrary like Napoleon on the battlefield, Nate Silver in the baseball match and modern India’s entrepreneur like Snapdeal founder Kunal Bahl. It should be concluded here that data driven decision making helps create a framework which allows validation of a standard method by repetition.
 
Data driven decision making has produced remarkable results in several domains and has helped to solve historically unsolved problems.
 
In the field of education, the top notch management colleges have expressed concern on not being able to create the best of peer groups that could maximize peer learning. The problem was essentially a consequence of intuitively formed groups with seldom any recognition of the educational background, past academic performance; data based analysis of individual’s strength and weaknesses etc. Modern research has shown that application of decision based system has resulted in remarkable increase in the peer learning. This study was based on the assumption of keeping all the externalities equal and the examination results were taken as the proxy for the improvement in performance.
 
In the field of crime mitigation and managing police administration the USA has shared reports that could startle any intuitive manager. The manager in this case is Anne Milgram, former Attorney General, State of New Jersey who was trying to put a check on the amount of crime rates. For all the time prior to Anne the decision was based on intuitive skills of the officers concerned. Anne revolutionized the entire methodology. She instructed to collect data in greater detail and based on the profiling of the people and the locality she redesigned the entire schedule for the police petrol and beet police. The impact was a sizable decrement in the crime rates.
 
Another example that validates the claim is the research of Prof. David Pall of Wharton School. In relation to e-commerce the professor says that attempt to push sales by an online enterprise should be based on some empirical findings. He says that the information about the social capital and the buying behavior of the customers are positively correlated with a correlation of 0.4. This correlation might appear to be small but even this much insight into the buying behavior of the customers has far reaching consequences on the company’s strategy. This gives the sales team a sizable knowledge that where their efforts in selling goods are more likely to fructify.
 
This example becomes clearer when we see it in the light of the failure of an e-commerce company that worked without any regard to data based decision making. This company, boo.com based in U.K. had established offices in the elitist of the locations for which it spent more money than Amazon and eBay and yet failed. The reason primarily was making decisions that weren’t corroborated by data and facts. They launched products unmindful of regulations in respective countries, did absolutely no market research on the expectations of target population in terms of getting the touch and feel of the products that they offered. The final nail in the coffin came when they expected very high volume of customer orders on the Christmas when they actually received just eight orders.
 
This clearly shows that data based decision making has greater significance and are likely to have a strong impact on business.
 
 
Note - Visitors shall not be able to comment on this article until they are logged in.
Hari Shankar Shashvat

Data Driven Decision Making: Helping You Decide

You have been making decisions on the basis of your gut; it has never led you astray. Undoubtedly, you have been hearing this buzz about using data to make decisions around you. Can Data Driven Decision Making (D3M) be trusted? Is it worth all the hoopla surrounding it? Let us find out.
As the name suggests, D3M is literally making decisions based on the analysis of data or information. Primarily, the information that must be analysed has to be relevant and there has to be a sufficient amount to help you make your decision. We shall discuss more about the nitty-gritties later as we dissect this term further. The concept of D3M was born out of necessity. Today, any action by a human causes data to be generated. According to an article on TechnologyReview, in the year 2012, 2.8 zettabytes of data was generated. That is almost equal to 3 billion hard disks of 1 terabyte each. A one terabyte hard disk can store 250,000 songs. Mind boggling, is it not? Even if you were to deal with the smallest fraction of the human race, there is still too much data for a mere mortal to handle. Enter the concepts of Big Data and D3M; these concepts have perhaps single-handedly revolutionized the way we make decisions.
 
Surprisingly, the concept of D3M has been around for a really long time; organisations started using it as early as the 1980s. It is just the scale and corresponding technology that has evolved. It is best stated in an article by Pamela Wheaton Shorr back in 2003 where she states that making a decision without data is similar to flying a plane in the dark without any radar. She also states that using D3M makes sure that everyone is held accountable; in her words-“ there is nowhere to hide†because the analysis of collected data can find out the most minute of discrepancies ensuring you do not lose any money to such irregularities.
 
Contrary to what many people believe, D3M is not that different from the methods that we have been using to make decisions. For starters, it is used all across the organization in different capacities as stated by Irving Wladawsky-Berger in this article on the Wall Street Journal. Henceforth, I aim to make it clear that D3M is an extension to the current intuition based decision making and not a replacement. As mentioned by several experts, every decision is made two times-
a.) Intuition: We make a decision on the basis of our intuition and past knowledge
b.) Data driven: Once data is available, we decide to continue with or forego the decision altogether.
The benefit of D3M is that you can combine these steps into one and get better value for the time spent in making that decision.
 
Even though it is not apparent, what we call intuition and belief are often a sum total of data. Our belief is born out of our past experiences; that too is data evaluated by our brain. So in essence when we talk about D3M, we have brought out the workings of our brain into the real world. Just like our brain, the D3M too has points which if not watched out for may cause incorrect decisions to be made.
 
 
 
i.) Time consistency : Any decision that we make is of use to us only within a limited time frame. Similarly, for any decision made through the D3M system, every decision is good as long as it is made on time. What causes a delay in decision making? The answer is rather simple when you think about it. Just like in life, here too, you cannot have the perfect answer. It is important to draw the line somewhere. Before making any decision using the D3M, you have to ask yourself- “Whether you want the correct answers – quickly if possible or do you want the answers quickly – correct if possible?†Something we must do in life as well.  
ii.) How much data is enough data ? : While making decisions conventionally, it is quite possible for us to overthink things and in trying to account for everything, we fail to make an apt decision in time. The equivalent of this in terms of D3M is the above mentioned question. Here, experts advise us to use our instinct and decide the bare minimum amount of data that we need to make our decision. Most experts give a fairly simple mantra to highlight the importance of this concept- “ Garbage in, garbage out â€; this phrase highlights that the better the quality of data you put in for analysis, the better the results. It is here that your formidable intuitive skills will be useful the most.  
iii.) Confirmation : More often than not, we tend to pre-formulate an opinion or decision in our minds and instead of arriving at that decision through analysis; we use that analysis to verify our decision. Most of the times, we tend to ignore any discrepancies between the suggested decision and our pre-decided decision. This causes us to make the wrong decisions. An easy way out is to be ready to make changes to our initial assumptions. This is similar to the way we streamline our decisions in real life based upon feedback.  
In the discussion so far, I have tried to cover all the important details about D3M. The purpose of presenting even the possible issues above is to show you that this system is just an extension of what you have been doing till now. The benefit of D3M is that it gives you the ability to quantify your decision and you have absolute control over the decision making process. You do not have to take my word for it; just refer to the links provided within the article and make sure for yourself.
 
References:
1. Decision-Driven Data Management: A Strategy for Better Decisions with Better Data (SAS White paper)
2. Other references mentioned inline
 
Note - Visitors shall not be able to comment on this article until they are logged in.
Aseem Shandilya

Data Driven Decision Making - A Holistic Approach To Decision Making

“A point of view can be a dangerous luxury when substituted for insight and understanding. – Marshall Mcluhan â€
We all have come a long way from industrial revolution where mass production and consumption was the order of the day. Over the years, the once not so knowledgeable consumer has transformed into a living storehouse of information. What has changed over the years? Why did the companies who were the industry leaders at one point of time failed miserably later on? What changed? The answer to all this lies in data. In our thirst to find newer markets for products and differentiate them from our competitors, we pushed the consumer towards seeking knowledge and then when he demanded, we failed to live up to standards.
 
We pushed the customer to give more importance to value i.e. value today has become emergent as compared to the old times when it was measurable. Now the new age customer wants more and he/she will switch, the moment he/she finds more value from another product or another competitor. How do we live up to this? Given the rapid changes in technology and the rapid adoption of management information systems by almost all industries, we need not look further. We have the solutions but we are too thick skinned to look at it. Today’s customer is like a walking hard drive with copious amounts of data. What we need to do is analyse it, find patterns where none exist, combine it with the experience of mankind and predict the needs and demands of future backed by strong data analysis.
 
If we stop and look back into the history what will be distinctly visible to us is the fact that companies that were present in the first list of Standard & Poor’s 500 only 74 remain in operation today. Almost half of the 25 companies that passed the rigorous test for inclusion in Tom Peters and Robert Waterman’s 1982 book “In Search of Excellenceâ€, have gone bankrupt or have shut down due to poor performance.
 
The reason being that when organisations are thriving and successful, it creates an illusion that whatever gave them success in the past will work in the present and still remain valid in the future. They forget the fact that the world is constantly changing, constantly evolving and to survive they need to evolve with it too. Today there is no dearth of examples of companies that failed after achieving success on a grand scale, to quote a few, Wang Labs, Digital Equipment, Borders and Blockbusters and many more; of which many either fell after achieving industry leadership position or shrunk in size.
 
The reason is not that they failed to act or didn’t know what was happening but the fact that they had all the data but didn’t analyse it. The patterns were there, had they analysed the data they already possessed they would have seen it. They believed that experience can upstage data but they failed to comprehend that with today’s rapidly changing global economy, it’s impossible for the human mind to absorb all factors that go into making sound decisions. Statistical analysis often uncovers correlations that no one could have predicted on the basis of their past experiences. We won’t be wrong in accepting the fact that data and statistical analysis isn’t perfect but who hasn’t made poor decisions on pure gut instinct!! It is said that sustainability is based on transforming data into analyzable information for insights and decision-making. This is where data driven decision making comes in and plays its part. (Cokins)
 
Take for example the case of Wal-Mart as reported in New-York Times. This occurred when Hurricane Frances was moving across the Caribbean, threatening to directly hit on Florida’s Atlantic coast. Residents were making a rush for the higher ground, but at the same time, far away, in Bentonville, Ark., executives presiding over the Wal-Mart Stores decided that the situation opened a great window of opportunity for their newest data-driven weapons, Predictive technology. A week ahead of storms predicted landfall date, Linda M. Dillman Wal-Mart’s CIO at that point of time asked her staff to come up with forecasts based on what happened when Hurricane Charley struck several weeks earlier. Their experts mined for data and found that the stores would need certain products more than the usual batteries, flashlights, and bottles of water.
 
All this brought out some very interesting facts about the pre-storm behaviour of consumers. Strawberry pop tarts and beer were the top-selling items. By predicting what was going to happen rather than waiting for it to happen, Wal-mart was able to deal with this unexpected situation in a holistic manner.
 
It is said that if we can improve our ability to estimate for a given customer, we stand to gain by applying the same logic to millions of customers. This applies to many other areas where intense application of data science and data mining is seen like direct marketing, credit scoring, financial trading, help-desk management, fraud detection, search ranking, product recommendation and many more. An example for this would be the instant recommendations of Amazon.com and Netflix which are a result of split-second advertising decisions based on the items a user has previously seen or is viewing right now, which has resulted in increased revenues for both of them.
 
The increasing amount of data generated by humans while doing their daily chores has lead us into a world where the purchases and habits of a consumer give us an insight into the behaviour of the consumer, how he reacts to various offers , what are his preferences, Is the customer a sort of person who will pay back loans. The algorithms have been designed and implemented which predict accurately how likely a person is to get diabetes. All this has a potential to give rise to platform of customization for every consumer i.e. mass customization.
 
A recent study by MIT professor Erik Brynjolfsson has brought out the fact that most revolutions in science begin with better methods of measurement. When we can see new things, we are driven to seek answers and thus build new ways of thinking and operating. The main underlying part of his findings was the fact that organisations that relied on data driven decision making, enhanced their performance by 4 – 6 % viz–a-viz their peers in the same market. This figure has remained robust despite taking into account the contributions for labour, capital, purchased services and traditional IT investment. (Lohr)
 
Another research conducted by Mr. Erik along with Mr. Lynn Wu, an assistant professor from Wharton Business School used publicly available web search data to predict housing-price-changes in metropolitan areas across the United States. They didn’t possess any special knowledge of the housing market when they began their study, but they reasoned that virtually real time search data would enable good near term forecasts about the housing market- and they were right. In fact their predictions proved to be more accurate than the official ones generated by the National Association of Realtors which had far more complex model but relied on slow changing historical data. (Brynjolfsson)
 
Now if we analyze the business that exist today and have been running on intuition, tradition and convenience, we observe that they have scattered staff development programs, the budgetary decisions are based on prior practice and programs, Staff assignments are based on interest and availability and goal setting is based on votes , favourite initiatives or fads. In contrast, the companies that have adopted data driven decision making for the staff development programs are more focussed as an improvement strategy to address the problems identified by the analysis of data. Staff assignments are based on skills required for future as well as the present requirement. Goal settings are done on the basis of problems identified by data analysis and their possible explanations.
 
One factor that we must consider is that with the ever increasing rapidly changing pace of technology, companies today have multiple points of contact with the customer i.e. blogs, call centres , customer reviews or simple comments on social media platforms which are all a valuable source about client segments , insights and behaviour. Today it can be observed that increasing competition, decreasing margins and availability of information has left many companies struggling for survival. The only way out seems to stem from the fact that it is very important today to analyse the vast amount of data being generated daily and convert it into actionable knowledge so that client needs are addressed adequately. (Mike Lynch)
 
An example worth mentioning here is that of researchers at John Hopkins School of Medicine, who provide a classic example of how simple models and big data win over more elaborate analytic approaches. They used the data from Google Flu Trends, which is publicly available, to forecast surges in flu related emergency visits a week before similar warning were received from the Centres of Disease Control. Similarly, twitter updates provided accurate records similar to the official reports that were tracking the spread of cholera in Haiti after the 2010 earthquake. (Brynjolfsson)
 
From the numerous examples quoted above, it can be substantiated that data analytics unlock huge amounts of useful information, which can be made more transparent in nature by use of Big Data. It is often seen that we search for data in forms other than electronic and then transfer it to electronic form. This effort represents a huge inefficiency; if we could the information directly from other forms to electronic form without human intervention it could save us a lot of resources and time.
 
Our online financial transactions or any transaction for that matter, if and when collected over a period of time can bring out interesting facts about our choices and behaviour. Now if we apply the same to a huge organization, what we have is copious amount of data that can provide us with very accurate and detailed information about various aspects of the organization. This further helps the middle level and upper level management make meaningful and informed decisions regarding day-today decisions to designing long term plans for the organizations along with requisite data to support it.
 
Data collected from consumer transactions can help us predict the behaviour of the customers and help the organisations in maintaining requisite inventory levels. They can also tailor and customize the products and services according to the needs of an individual customer without the customer making an additional effect in telling the provider about it. It also leads to narrower segmentation of customers.
 
Data obtained when combined with power analytics techniques can be used to find hidden patterns which otherwise would remain hidden and help in predicting future needs , improvise after-sale services and meet the expectations of the consumers in a holistic manner. In short the transfer of information between consumer and manufacturer can be defined as the art of communicating; without actually communicating; as no formal/informal communication takes place between the parties’ involved; just observation, which leads to predicting the needs and aspirations of consumers. We need to understand that the data is not the only tool for the decision making. If applied in isolation without experience, the end result is a Frankenstein’s monster of business process and best practices woven together. As value increases its dependency on unique knowledge and human experience, objective data sources must be supplemented with perspectives from organizational cultures, communities of interest and other relationships. Only then can we take holistic and sound decisions.
 
Bibliography (Debates, 2011)
Bonabeau, E. (n.d.). Harvard Business Review . Retrieved 11 21, 2013, from hbr.org : http://hbr.org/2003/05/dont-trust-your-gut/ar/5
Bono, E. D. (n.d.). Search Quotes. Retrieved 11 21, 2013, from searchquotes.com: http://www.searchquotes.com/quotes/author/Edward_De_Bono/3/
Brynjolfsson, A. M. (n.d.). Harvard Business School . Retrieved 11 21, 2013, from Harvard Business School Website : http://hbr.org/2012/10/big-data-the-management-revolution
Christensen, C. (n.d.). Design Intelligence. Retrieved from di.net: http://www.di.net/articles/the-innovators-dilemma-when-new-technologies-cause-great-firms-to-fail/
Cokins, G. (n.d.). analytics-magazine. Retrieved 11 21, 2013, from analytics-magazine.org : http://www.analytics-magazine.org/may-june-2012/577-corporate-decision-making-why-do-large-once-successful-companies-fail
Debates, D. (2011). When it comes to important business decisions , should you trust your gut or follow the numbers . Deloitte Development LLC.
Lohr, S. (n.d.). New York Times . Retrieved 11 21, 2013, from New York Times Website : http://www.nytimes.com/2011/04/24/business/24unboxed.html?_r=0
Mike Lynch, c. e. (n.d.). Financial Times . Retrieved 11 21, 2013, from Financial Times Website : http://www.ft.com/cms/s/0/62f37a4a-931c-11de-b146-00144feabdc0.html#axzz2lITs0aA5
Plisch, B. G. (2004, 02 04). WTN News . Retrieved 11 21, 2013, from WTN News Website : http://wtnnews.com/articles/557/
Tim McGuire, J. M. (n.d.). Ivey Business Journal . Retrieved 11 21, 2013, from iveybusinessjournal.com: http://iveybusinessjournal.com/topics/strategy/why-big-data-is-the-new-competitive-advantage#.Uo47PsXlbrN
Wladawsky-Berger, I. (n.d.). Irving Wladawsky-Berger. Retrieved 11 21, 2013, from Irving Wladawsky-Berger blog: http://blog.irvingwb.com/blog/2013/09/data-driven-decision-making-promises-and-limitations.html
Zwilling, M. (n.d.). Forbes . Retrieved 11 21, 2013, from Forbes.com: http://www.forbes.com/sites/martinzwilling/2012/03/25/too-many-leaders-still-rely-on-their-golden-gut/
 
Note - Visitors shall not be able to comment on this article until they are logged in.
deepakgupta312

Not What Or Why, But How To Choose?

The most essential facet of being a manager, be it in any walk of life, is for sure the one skill that differentiates him/her from anybody else and that is: Decision-making. It is most definitely the power, rather the ability of a person to take prudent, relevant and effective decisions that set him/her a notch ahead in the race that the corporate run comprises. Thus, we not only need to know what or why, but how to make decisions, how to make the choices that we make, to be able to foresee what lies in store having made those choices and to be positive that the alternatives have an opportunity cost that is less likely to exceed the benefits from the plan A that we choose.
The mind always works on matter, matter that is assimilated, accumulated, observed, absorbed and created. This matter more often than not comprises facts and figures that we come across over a period of time and it is this that formulates our opinions on issues and circumstances and should guide us in choosing the way forward. One’s experiences are a key to learning and avoid mistakes, but life is all about new challenges and it isn’t necessary that one’s challenge will be the other’s or that one’s solution will work out for the other, problems, solutions and hence decisions, vary, with time and tide, to each his own.
 
Decision-making is a crucial skill for treading on the path to success. It is imperative thus that one take informed decisions, rather than impulsive or intuitive ones. This itself can be illustrated by the use of numerical figures in terms of probability of a decision being a thumbs-up or a downer based on whether it is based on gut-feeling or not, where a gutsy decision has a fair and square half chance of hitting the bulls-eye, an informed, data driven decision based on historical data can warrant a chance much greater than that of being a winner.
 
Data analytics, a field of research, an active component of all industrial data processing and reporting, has become the buzz-word in the world of business. It is owing to the stress on hard core historical data projection and information related to past trends, their relevance, their cyclical patterns, any relational change between parameters and many other such ways of engineering data to deduce important facts and identify occurrences, areas of improvement, effect of factors and even have a decent foresight by extrapolating trends, assuming at times ceteris paribus. It is the mere fact that this subject is being undertaken as a specialization in some of the newer and premiere business schools in the country that relays its importance, its applicability and the extent to which it can play a role in making key decisions easier for every managerial level all courtesy the heavy data dependency, real data building up virtual images only for a clearer picture albeit a surer one.
 
While the success of data driven decision making can itself be proved through data, its wonder lies in the fact that one can also prove the risk related to intuition based choices in a data driven manner.
 
There is a reason the rational mind which processes such data is called analytical. It overcomes the biases of the emotional mind and loosens the shackles of a narrow-minded attitude towards things or the safe zone preference based on conservative beliefs, prevalent practices and recency effect of happenings. Strong and concrete data has the power to shake people out of the herd mentality and of Utopian expectations based on rosy optimism rather than hard-hitting realism. Mostly intuitive decisions tend to land people in false situations, because they are taken either in haste or due to ignorance. Either way, it makes waste due to the hurry in which it is taken without any deliberation or any heed to the consequence, totally reliant on luck for the outcome, well if good, expected if bad. In case of ignorance, the risk is even higher as the stakeholder in question very conveniently underestimates the potential risk and can in fact land in a whole lot of trouble, based on the decision ending up seeming gullible, irrational and more often than not irresponsible and unreliable.
 
Thus we can say that data plays a key role, be it that of the number of germs that a disinfectant can kill or that of the amount of disposable income in the economy that drives consumption, be it the household consumer or the government bodies, be it the decision of keeping yourself healthy or the nation, it is data alone that can be relied upon for a less blurred vision of the future through the spectacles of the past. If such decisions were to be made on the flip of a coin, only probability data related to the outcomes of a coin flip can tell us which side we would be losing, the health of the nation or the its people, either of which states the vulnerability of intuition as a tool for making choices against rationality which is but obvious the only choice to make when deciding, anything, big or small, for the better and not remotely for the worse.
 
Note - Visitors shall not be able to comment on this article until they are logged in.
onam.priyadarshi13h

How Is Data-Driven Decision Making Changing The Way In Which Decisions Are Being Made?

Have you ever attended school? Well, I did and it’s laden with bitter-sweet memories. Sweet because it was that one time where I had this amazing thing called “Work-Life†Balance. Back then,, I had never heard about such a word although I had it in my life. And by the time I came to know about it, it was nowhere to be found in my life. What an irony!
Bitter were those memories where there used to be a peculiar event called “Parent-Teacher Meetingâ€. The good kids were appreciated, the bad ones were reprimanded. And what was the key metric that they used for this? Marks! “Did you see your son’s marks? How pathetic. He is not at all attentive in class and his marks shows. Your daughter is amazing. You don’t need to worry about her. Her marks are excellent. She will make you a proud parent indeed.â€
The point I am trying to impress upon you is the fact that data-driven decision making isn’t something of a recent origin. It has been there since ages. Why did Copernicus come up with the theory that earth revolved round the sun? How did America get discovered by the Columbus? Because these people relied on numbers and calculations, based on hard facts and not on what popular opinion lay at that time. Why did Vasco-Da-Gama travel all the way to India? Because he believed in data which showed that India was a spice rich country. Why were more and more spies sent behind enemy lines during World-Wars? Data, data, data.
 
Today’s world is no less competitive either. Business leaders are in a continuous battle trying to improve their topline, bottomline and also keep employees satisfied. Data holds the key unlock all of these questions. Data analytics can give an insight into patters which may or may not be easily visible. For instance, a firm that grows at 5% as compared to last year in a market that’s expanding at 10% is losing market share. Although a casual observation might show that topline and bottomline has gained, only a rigorous analysis done on the industry reports can give the whole picture. Similarly, a yearlong research and analysis by Harvard Business Review on the industries that did well during recession brought forth some key insights into the focus areas which could help the leaders of companies perform better in the future.
 
Such data driven analytics helps in better understanding of the situations and allows for more informed decision making. Another recent study on the internet usage in India showed that access to internet via mobile phones was at 62.5% as compared to 37.5% for desktop. Such data analytics helped Google in understanding where its future lay. And the recent ad on Indo-Pak friendship clearly depicted how google search on mobile could take care of the requirements of the users.
 
It’s not just Technology companies that such analytics. In fact, recently at a company event in San Francisco, Jeff Immelt, CEO of GE talked about how analytics is the “next holy grailâ€. He also talked about how such analytics was helping in cutting down “unplanned downtime†for its customers. Such data driven decision making not only helps you in making better informed decisions, it also helps in making those decisions faster.
 
In today’s fast paced world, waiting for the quarterly/annual financial reports and then taking decisions is not the correct way of running a business anymore. What we need today is real-time decision making that‘s dependent on strong analytical foundations. These changing trends could also give you a headstart in future businesses. The changing trends that brought about the fall of Kodak, Nokia could have been averted had they seen the changing trends. The next big trends are in E-cigarettes, LED lightning, 3-D printing which are all gaining momentum. And being able to tap into these and many such areas at the right time, at the right place allows a firm to outgrow its competitors. Such understanding and insights of the future trends could only be gained and developed by data driven analytics.
 
To sum it up, Data Analytics is very important tool that’s available with the decision makers who can tap into the vast amounts of data that's on the internet and analyse them, ask the right questions to understand where they can improve on their existing operations and where to invest their future into.
 
Note - Visitors shall not be able to comment on this article until they are logged in.
P JAYANT

Data Driven Decision Making (Dddm)

Decision
 
Decision is something that every organization has to deal with every day. From the smallest scale, like labor distribution, to the largest, as in targeting new markets, it is often the major reason for success or failure at that particular level and beyond. In modern times the managers and executives, who control the future and destination of the companies, are facing more and more complicated situations where they need to make decisions considering numerous criterion within limited time period. At the same time the business competitors are coming up with more and more innovative ways to deal with the situation.
DDDM
 
The information explosion from increasingly sophisticated computer tracking of shipments, sales, suppliers and customers, as well as e-mail, web traffic and social-network is a huge challenge for some and for others it is a great opportunity. The later base their decisions on the analysis of data rather than purely on intuition, this practice is called Data-driven decision making.
 
The benefits of data-driven decision making have been demonstrated in a study conducted by the economist Erik Brynjolfsson and his colleagues from MIT and Penn's Wharton School. They developed a measure of DDDM that rates firms as to how strongly they use data to make decisions across the company. They show statistically that the more data-driven a firm is, the more productive it is—even controlling for a wide range of possible confounding factors. And the differences are significant: 4–6% increase in productivity. DDDM also is correlated with higher return on assets, return on equity, asset utilization, and market value, and the relationship seems to be causal.
 
Drawbacks of Purely Intuitive Decision Making
 
Complex situations. Pure intuition may be fine for small decisions where your mind processes part or all of the information automatically, quickly, and without awareness of any details. But as the complexity increases it becomes impossible for human mind to process all the scenarios individually.
 
Time to build intuition. Good intuition takes a long time to build. Chess players, for example, need 10 years of dedicated study and competition to assemble a sufficient mental repertoire of board patterns.
 
Specific to environments. Some environments provide accurate indications about what’s going to happen next. They exist in poker and firefighting, but not in, say, stock markets. Feedback from the environment is information about what worked and what didn’t. Intuition doesn’t work well in environments where the results of decisions are vague and takes time to materialize.
 
Inconsistency in application. A person following intuitive decision making often makes largely different decisions for the relatively similar situations and if one decision works on a particular situation there is no guarantee that it will work the same in others.
 
Bad judgments. Intuition can easily be biased by overreaction to available information. Psychologists call this availability bias, which although the most prevalent is not the only form of bias that might exist, for eg. there is confirmation bias when people have a hard time believing and remembering evidences that contradict their beliefs. The decision makers could be a victim to many such biases.
 
Origin of idea. There’s no to know if a spontaneous idea is the result of legitimate expertise or of a pernicious bias.
 
The Way Forward
 
Intuition and DDDM are not necessarily at odds. There are many times when decision-makers don't have very good data because something is new, the situation has changed or because what might seem like intuition is actually well-rehearsed behavior that comes from years of experience at something, so even though people can't articulate the pattern they recognize, they still are acting on a huge body of experience and knowledge. And on the very other side of experience there are virtues to the gut reaction of naive people, as those who are not properly brainwashed may see things and come up with ideas that expertise drives out of their brains.
 
The tools and techniques provided by data science to process data can contribute a great deal by the following ways:
Extracting useful knowledge from data to solve business problems can be treated systematically by following a process with reasonably well-defined stages. Evaluating data results requires careful consideration of the context in which they will be used. The relationship between the business problem and the data analytics solution often can be decomposed into tractable subproblems via the framework of analyzing expected value. Information technology can be used to find informative data items from within a large body of data. Entities that are similar with respect to known features or attributes often are similar with respect to unknown features or attributes. Provides close attention to the presence of confounding factors, possibly unseen ones.  
Finding a balance between intuition and data DDDM is important not just for the top mangers and executives but throughout the organization. For eg. using data analysis in marketing would allow the business to provide the most desirable products to its prospective customers.
Many data focused companies employ data-science teams to bring advanced technologies to increase revenue and to decrease costs. Companies like Facebook and Twitter, along with many other “Digital 100†companies,5 have high valuations primarily due to data assets they are committed to capturing or creating.
 
Conclusion
 
Using data analytics for business processes is important, but we must think beyond the algorithms, techniques, and tools in common use. We must think about the core principles and concepts that foster success in data-driven decision making. Success in today's data-oriented business environment requires being able to think about how data analytics can be applied to particular business problems. Understanding this process and its stages helps structure problem solving, makes it more systematic, and thus less prone to error.
 
Note - Visitors shall not be able to comment on this article until they are logged in.
Kongkan

Demystifying The Decision Making: Getting It Right

"Whatever a manager does, he does through decision-making", said by none other than the man who invented management theory Peter Drucker. Decision-making is an indispensable component of the management process. It is a well-balanced judgment and a commitment to action. What a manager needs to do is rational decision making to achieve his objectives effectively.
If you have been successful in taking decisions with your intuition, which means that you have acquired the skill through experience. In the fast moving world where change is only constant, decision making requires the scientific approach .The time has come for you to acquire an additional skill if you want to be a competitive and an updated manager.
 
A decision should be rational because it may facilitate expansion of business and give more profit, goodwill and fame to a business unit. Decision making is continuous activity, mental/intellectual activity, based on reliable information, time consuming activity, and a responsible job. Good decisions are always taken by proper analysis of reliable information. The decision making process at all levels of the Organisation can be improved with the support of an effective management information system (MIS). This facilitates the manager to take a bold and confident decision.
 
Reasons to use data based decision making
Storing of data is becoming more cheaper and easier. The data is flowing with a characteristic of 3V's (Velocity, Variety and Volume). The velocity of data incoming to companies is very high. This is through various mediums like sensors, transaction systems, social media web sites, forums, research reports etc. This data is coming in variety of formats like text, numbers, audio, video and images. The volume of the data is a similar to tsunami. As of 2012, data that is created in a day is 2.5 Exabyte’s (2.5×1018). Every two years, the amount of data available on internet is doubled.
Companies boast of their growth in terms of
Customers grow at x% CAGR grow at y%  
So it is time to realize that data also grows at an even faster pace. Ignoring this vast amount of data is nothing but losing a wonderful opportunity to dive in and taking better decisions. Decision-making is an intellectual process which involves selection of one course of action out of many alternatives. The effectiveness of management depends on the quality of decision-making.
 
Here are some of the Loop holes in decision making through gut and intuition
· Decision depends on the mood of the manager
- The decision on same situation can be contradicting when a manager is in different moods if he is not using data.
· Especially in Uncertain environment, it is the most difficult task to take decisions. So without the use of data, you are in great risk.
· A situation may be encountered wherein a manager have too many choices. Moreover more than 1 seems to be right. Then the manager faces the dilemma of which choice to be selected.
· Personal element in decision-making is a common phenomenon in the process. Similarly, every decision-maker will have his own personal background in the form of personal beliefs, attributes, preferences, likes and dislikes and so on.
 
Need for data driven decision making
a) The higher u climb up the corporate ladder, the more data is available just before you. Remember the more power you get, the more responsible you become. Consider a situation wherein you were managing hundred people. You may know each and every name of them and how they work. You may be very comfortable to take decision on promoting a certain employee. But, when you are given a responsibility of a thousand or more employees under you. Then the importance of data comes into picture.
b ) A person at management position can be posted in any division of the company. The manager at different division who took decisions based on his experience may no longer be able to do the same.
c) Data driven decision making is always backed by logic by proper analysis of relevant data.
d) This power of analytics is realized and now, it is extensively used by top fortune companies of the world.
 
Ways of data based decision making
Big data is the emerging trend in the world now. Companies are investing heavily on the use of big data to get the powerful insights from data. This data may be of various formats as mentioned before. Many statistical tools like SPSS, SAS are available for analysis of data. These tools facilitate the manager to get important insights from the data. Open source tools like R and weka are gaining popularity these days.  
Finally, Nothing but Change is constant in this world. So the manager should be embracing change
 
 
 
 
It is in your moments of decision that your destiny is shaped.
 
 
 
Do it wise...
 
 
 
 
 
 
Note - Visitors shall not be able to comment on this article until they are logged in.
 
rahulbillapati

Lessons of the Square Watermelon

The story revolves around the growing of square watermelons in Japan.
Japanese grocery stores had a problem. They are much smaller than their US counterparts and therefore don’t have room to waste. Watermelons, big and round, wasted a lot of space. Most people would simply tell the grocery stores that watermelons grow round and there is nothing that can be done about it. But some Japanese farmers took a different approach. ”If the supermarkets want a space efficient watermelon,” they asked themselves, “How can we provide one?” It wasn’t long before they invented the square watermelon.
 
The solution to the problem of round watermelons wasn’t nearly as difficult to solve for those who didn’t assume the problem was impossible to begin with and simply asked how it could be done. It turns out that all you need to do is place them into a square box when they are growing and the watermelon will take on the shape of the box.
 
This made the grocery stores happy and had the added benefit that it was much easier and cost effective to ship the watermelons. Consumers also loved them because they took less space in their refrigerators which are much smaller than those in the US - which resulted in the growers being able to charge a premium price for them.
 
What does this have do with anything besides square watermelons? There are a five lessons that you can take away from this story which will help you in all parts of your life. Here are a few of them:
 
1.) Don’t Assume:
The major problem was that most people had always seen round watermelons so they automatically assumed that square watermelons were impossible before even thinking about the question. Things that you have been doing a certain way your entire life have taken on the aura of the round watermelon and you likely don’t even take the time to consider if there is another way to do it. Breaking yourself from assuming this way can greatly improve your overall life as you are constantly looking for new and better ways to do things.
 
2.) Question Habits:
The best way to tackle these assumptions is to question your habits. If you can make an effort to question the way you do things on a consistent basis, you will find that you can continually improve the way that you live your life. Forming habits when they have been well thought out is usually a positive thing, but most of us have adopted our habits from various people and places without even thinking about them.
 
3.) Be Creative:
When faced with a problem, be creative in looking for a solution. This often requires thinking outside the box. Most people who viewed this question likely thought they were being asked how they could genetically alter water melons to grow square which would be a much more difficult process to accomplish. By looking at the question from an alternative perspective, however, the solution was quite simple. Being creative and looking at things in different ways in all portions of your live will help you find solutions to many problems where others can’t see them.
 
4.) Look for a Better Way:
The square watermelon question was simply seeking a better and more convenient way to do something. The stores had flagged a problem they were having and asked if a solution was possible. It’s impossible to find a better way if you are never asking the question in the first place . Always ask if there is a better way of doing the things that you do and constantly write down the things you wish you could do (but currently can’t) since these are usually hints about steps you need to change. Get into the habit of asking yourself, “Is there a better way I could be doing
this?” and you will find there often is.
 
5.) Impossibilities Often Aren’t:
If you begin with the notion that something is impossible, then it obviously will be for you. If, on the other hand, you decide to see if something is possible or not, you will find out through trial and error.
tharun.shanker

Types of Errors in Testing of Hypothesis

 
 
The inductive inference consists in arriving at a decision to accept or reject a null hypothesis (H0) after inspecting only a sample from it. As such, an element of risk - the risk of taking wrong decisions is involved. In any test procedure the four possible mutually disjoint and exhaustive decisions are:
 
Reject Null Hypothesis when actually it is not true, i.e. when Null is false Accept Null when it is true Reject Null when it is true Accept Null when it is false  
The decision in (I) & (ii) are correct decisions while the decisions (iii) & (iv) are wrong decisions.
 
These decisions may be expressed in the following dichotomous table:

 
Thus, in testing the hypothesis we are likely to commit two types of errors. The error of rejecting Null when Null is true is known as a Type I error and the error of accepting Null when Null is false (i.e. Alternate is true) is known as Type II error
Remark: Type I & Type II error:
We make type I error by rejecting a true null hypothesis We make type II error by accepting a wrong null hypothesis  
If we make
P (Reject Null when it is true) = P (Type I Error) = Alpha (a)
P (Accept Null when it is wrong) = P (Type II Error) = Beta (b.)
 
Then a & b are also called the sizes of Type I error & Type II error respectively
 
In the terminology of Industrial Quality Control while inspecting the quality of a manufactured lot, the Type I error amounts to rejecting a good lot and Type II error amounts to accepting a bad lot.
 
Accordingly,
a = P (Rejecting a good lot)
b= P (Accepting a bad lot)
 
The sizes of Type I & Type II errors are also known as Producer's risk & Consumer's risk respectively.
 
An ideal test procedure would be one which is so planned as to safeguard against both these errors. But, practically, in any given problem, it is not possible to minimize both these errors simultaneously. An attempt to decrease a results in an increase in b & vice versa. In practice, in most of decision-making problems in business and social sciences, it is more risky to accept a wrong hypothesis than to reject a correct one, i.e., consequences of type II error are likely to be more serious than the consequences of type I error. Since for a given sample, both the errors cannot be reduced simultaneously, a compromise is made by minimizing more serious errors after fixing up the less serious error.
Thus, we fix a, the size of type I error and then try to obtain a criterion which minimizes b, the size of type II error.
 
Obviously, when Null is true, it ought to be accepted. Hence, minimizing b amounts to maximizing (1-b.), this is called the power of the test. Hence, the usual practice in testing of hypothesis is to fix a the size of type I error and then try to obtain a criterion which minimizes b, the size of type II error or maximizes (1-b.), the power of the test.
ssachin.mannan_1

How we address the Sample Size Dilemma?

1. Introduction
 
1.1 General
 
Sample-size determination is often an important step in planning a statistical study—and it is usually a difficult one. Among the important hurdles to be surpassed, one must obtain an estimate of one or more error variances, and specify an effect size of importance. There is the temptation to take some shortcuts. This paper offers some suggestions for successful and meaningful sample-size determination. Ever-changing markets are a difficult environment for testing. You need to control as many sources of variation as you can and then gather enough data for test results to rise above the noise. Sample size calculations are the tool to help you decide how much is “enough.” A big part of planning to succeed is figuring out how many observations you will need in order to meet the objectives of your project. Taking observations costs time and money, so we want to make sure we get just the right amount to make inferences about our outcomes of interest
 
1.2 Importance of Right Sample Size
 
Statistical studies (surveys, experiments, observational studies, etc.) are always better when they are carefully planned. Good planning has many aspects. The problem should be carefully defined and operationalised. Experimental or observational units must be selected from the appropriate population. The sample must be of adequate size, relative to the goals of the study. It must be “big enough” that an effect of such magnitude as to be of scientiï¬c signiï¬cance will also be statistically signiï¬cant. It is just as important, however, that the sample is not “too big,” where an effect of little scientiï¬c importance is nevertheless statistically detectable. Sample size is important for economic reasons: An under-sized sample can be a waste of resources for not having the capability to produce useful results, while an over-sized one uses more resources than are necessary. In an experiment involving human or animal subjects, sample size is a pivotal issue for ethical reasons. Sample size calculation is a more complex topic than can be covered in-depth here, but there are several key items you should start thinking about before you consult with a statistician or other researcher familiar with sample size calculations.
 
1.3 Previous Literature
 
For such an important issue, there is a surprisingly small amount of published literature. Important general references include Mace (1964), Kraemer and Thiemann (1987), Cohen (1988), Desu and Raghavarao, (1990), Lipsey (1990), Shuster (1990), and Odeh and Fox (1991). There are numerous articles, especially in biostatistics journals, concerning sample-size determination for speciï¬c tests. Also of interest are studies of the extent to which sample size is adequate or inadequate in published studies; see Freiman et al. (1986) and Thornley and Adams (1998). There is a growing amount of software for sample-size determination, including nQuery Advisor (Elashoff, 2000), PASS (Hintze, 2000), UnifyPow (O’Brien, 1998), and Power and Precision (Borenstein et al., 1997). Web resources include a comprehensive list of power-analysis software (Thomas, 1998) and online calculators such as Lenth (2000). Wheeler (1974) provides some useful approximations for use in linear models; Castelloe (2000) gives an up-to-date overview of computational methods.
 
2. Sample Size
 
2.1 There are several approaches to sample size. For example, one can specify the desired width of a conï¬dence interval and determine the sample size that achieves that goal; or a Bayesian approach can be used where we optimize some utility function—perhaps one that involves both precision of estimation and cost. One of the most popular approaches to sample-size determination involves studying the power of a test of hypothesis. It is the approach emphasized here, although much of the discussion is applicable in other contexts. The power approach involves these elements:
 
(a) Specify a hypothesis test on a parameter θ (along with the underlying probability model for the data).
 
(b ) Specify the signiï¬cance level α of the test.
 
(c ) Specify an effect size θ that reflects an alternative of scientiï¬c interest.
 
(d) Obtain historical values or estimates of other parameters needed to compute the power function of the test.
 
2.2 Determining the sample size is one of the early steps that must be taken in the planning of a survey. Unfortunately, there is no magic formula that will tell us what the perfect sample is since there are several factors we need to think about.
 
 
3. What Drives Our Needed Sample Size?
 
There are a few concerns that drive the sample size required for a meaningful test:
 
 
 
(a) We want to be reasonably sure that we don’t have a false positive—that there is no real difference, but we detect one anyway. Statisticians call this Type I error.
 
(b ) We want to be reasonably sure that we don’t miss a positive outcome (or get a false negative). This is called Type II error.
 
We want to know whether a variation is better, worse or the same as the original.
 
4. Factors for Right Sample Size
 
4.1 Analytical Plan The research objectives and planned analytical approach should be the first factor to consider when making the decision on sample size. For instance, there are statistical procedures (e.g. regression analysis) that require a certain number of observations per variable. Moreover, if comparative analysis between subgroups in the sample is expected, the sample size should be adjusted for it to be able to identify statistically significant differences between the groups.
 
4.2 Population Variability This refers to the target population's diversity. If the target population exhibits large variability in the behaviors and attitudes of interest being researched, a large sample is needed. If 20% or 80% of the population behaves in certain way, this indicates less variability than if 50% would do so. To be conservative, it is standard practice to use 50% (0.5) as the event probability in sample size calculations since it represents the highest variability that can be expected in the population.
 
4.2 Level of Confidence This is the level of risk we are willing to tolerate usually expressed as a percentage (e.g. 95% confidence level). Although survey results are reported as point estimates (e.g. 75% of respondents like this product), the fact is that since we are working with a sample of the target population, we can only be confident that the true value of the estimate in that population falls within a particular range or what is called confidence interval. The level of confidence indicates the probability that the true value of the estimate in fact will fall within the boundaries of the confidence interval. How confident can you be? As confident as your tolerance for risk allows you to, knowing that the confidence level is inversely proportional to estimate accuracy or margin of error. The more the level of confident, the larger the sample size.
 
4.3 Margin of Error Also known as sampling error, indicates the desired level of precision of the estimate. You have probably seen poll results quoted in the media, saying that the margin of error was plus or minus a particular percentage (e.g. +/-3%). This percentage defines the lower and upper bounds of the confidence interval likely to include the parameter estimate, and it is a measure of its reliability. The smaller the margin of error, the larger the sample size and the greater the estimate precision.
 
4.4 Cost Sample size cost is often one of the largest items in the budget for market research studies, especially if the target sample includes low-incidence segments or the response rates is low. Many times, we have to make a tradeoff between statistical accuracy and research cost.
 
4.5 Population Size Most of the time, the size of the total target population is unknown, and it is assumed to be large ( >100,000), but in studies where the sample is a large fraction of the population of interest, some adjustments may be needed.
 
5. Sample Size Calculation Check List As a summary, to determine the sample size needed in a survey, we need to answer the following questions:
(a) What type of data of data analysis will be conducted? Will subgroups be compared?
((b ) What is the probability of the event occurring? - If not previous data exists, use 50% for a conservative sample size estimate.
(c ) How much error is tolerable (confidence interval)? How much precision do we need?
(d) How confident do we need to be that the true population value falls within the confidence interval?
(e) What is the research budget? Can we afford the desired sample?
(f) What is the population size? Large? Small/Finite? If unknown, assume it to be large ( >100,000).
 
6. How to Choose Sample Size for a Simple Random Sample To choose the right sample size for a simple random sample, you need to define the following inputs.
(a) Specify the desired margin of error ME. This is your measure of precision.
(b ) Specify alpha (α).
(i) For a hypothesis test, alpha (α) is the significance level . (ii) For an estimation problem, alpha (α) is equal to 1 - Confidence level .  
(c ) Find the critical standard score   z . (i) For an estimation problem or for a two-tailed hypothesis test , the critical standard score (z) is the value for which the cumulative probability is 1 - alpha/2. (ii) For a one-tailed hypothesis test , the critical standard score (z) is the value for which the cumulative probability is 1 – alpha.  
(d) Unless the population size is very large, you need to specify the size of the population (N).  
(e) Given these inputs, the following formulas find the smallest sample size that provides the desired level of precision.  
   
Sample statistic   Population size   Sample size  
   
Mean Known n = { z 2 * σ 2 * [ N / (N - 1) ] } / { ME 2 + [ z 2 * σ 2 / (N - 1) ] }  
   
Mean Unknown n = ( z 2 * σ 2 ) / ME 2  
   
Proportion Known n = [ ( z 2 * p * q ) + ME 2 ] / [ ME 2 + z 2 * p * q / N ]  
   
Proportion Unknown n = [ ( z 2 * p * q ) + ME 2 ] / ( ME 2 )  
   
This approach works when the sample size is relatively large (greater than or equal to 30). Use the first or third formulas when the population size is known. When the population size is large but unknown, use the second or fourth formulas.  
   
For proportions, the sample size requirements vary, based on the value of the proportion. If you are unsure of the right value to use, set p equal to 0.5. This will produce a conservative sample size estimate; that is, the sample size will produce at least the precision called for and may produce better precision. A number of tools like Minitab also exist which can be used to calculate the sample size.  
   
7. Conclusion   Sample-size planning is often important, and almost always difï¬cult. It requires care in eliciting scientiï¬c objectives and in obtaining suitable quantitative information prior to the study. Successful resolution of the sample-size problem requires the close and honest collaboration of statisticians and subject-matter experts. Various types of changes to the study can be recommended if it turns out to be over or under-powered. Sample-size problems are context-dependent. Moreover, sample size is not always the main issue; it is only one aspect of the quality of a study design.  
   
References  
   
1. Odeh, R. E. and Fox, M. (1991), Sample Size Choice: Charts for Experiments with Linear Models, Marcel Dekker, New York, second edn.  
   
2. Mace, A. E. (1964), Sample-size determination, Reinhold, New York.  
   
About the Author  
   
1. Lt Col Hardeep Sandhu is a serving army officer. He has a certificate in advanced computing and Six Sigma and is an Electrical Engineer, MBA in HR & Marketing and M Phil in Management.  
   
2. The officer was third in merit at the national level for selection in the National Defence Academy, Khadakwasla and first in order of merit for commissioning into the Madras Sappers. He has been an Ambassador of the Country in a United Nations Mission in Africa. He has been commended for his role in saving over 100 people over a night in 1997 when the River Brahmaputra was in spate. In addition, he has been commended for his role in the Platinum Jubilee at Indian Military Academy, Dehradun in 2008 and for the Army and Republic Day Parade in 2009.  
   
3. The officer is a qualified Commando. He has represented the Army in a Sailing Expedition and is a keen sailor, rider and golfer. He is an avid traveler and painter and is a gold medalist in hockey, football, volleyball and sailing.  
   
   
 
hardeep.hr77

Using Hypothesis Testing in IT

 
 
How the use of hypothesis testing shall be a great asset in my company?
 
We do decision making everyday in our personal and professional life but most of the times it’s not backed by sufficient data to arrive at a confident decision. For example, we do choose different routes to reach a particular destination, but this decision most of the time is based on our likes of the route or experience with that route. This doesn’t provide us the sufficient confidence with objective evidence that with certain confidence level the route will result in saving travel time. Let me take another example from professional life, every time a software development project is won in an IT industry, we deploy various resources for the execution of project based on our estimate of size and effort required to complete the project. Here again our decision is based on the overall productivity and we get the number of resources to be deployed. We donot take into account what experience level of resources is going to have what impact on the execution effort and time for the project.
 
Hypothesis testing is the testing of any assumptions regarding a population with the sample data. There are broadly two types of hypothesis testing viz. Null & Alternate hypothesis.
 
The null hypothesis testing tests the claim of equality or of no difference, while alternate hypothesis tests inequality or of difference between two or more populations.
 
The assumptions made in the null hypothesis can be accepted / rejected on the basis of p-value comparison with significance level or alpha.
 
There can be two types of errors which can be committed while doing hypothesis evaluation primarily called Type I or alpha error and Type II or beta error.
 
Alpha error is committed when we fail to accept null hypothesis, while actually it is true.
 
Beta error is committed when we fail to accept alternate hypothesis, while actually it is true.
 
Hypothesis is a technique which can help us in arriving at a suitable mix of different experience level of resources to be deployed for the project with certain level of confidence of execution.
 
This technique offers us the opportunity to compare the productivity level of resources with different experience level and arrive at a conclusion if there is any significant difference in the productivity at different experience level. Knowing the productivity level of resources at various experience level, we can suitably deploy resources of different experience level in the project and thereby can reduce the cost in completing the project and improve project profitability.
 
There are other scenarios in IT industry where hypothesis test can be effectively and easily deployed to garner the success through this technique e.g. comparing the quality level of different modules, evaluating the quality of deliverable w.r.t. target etc.
 
One of the other opportunity I see in deploying hypothesis technique in IT industry is in the evaluation of various review techniques which organizations employs in review the work products e.g. Offline review, walkthrough review and inspections. Many a times these reviews are planned in the projects without considering their review yields and comparing the effectiveness data from various review techniques which unnecessarily results in spending more effort for certain types of work products where less stringent review could be have been deployed to capture the similar number of defects.
 
Thus hypothesis testing provides a very good technique in our quality arsenal which can be effectively and efficiently used in various decisions at different levels. Comments invited!
 
Sanjay Shah
 
sanjay.shah.

The Right Sample Size

The right sample size - How we address this dilemma?
 
Sample size calculation is the act of choosing the number of observations to include in a statistical sample. The sample size is an important feature of any empirical study in which the goal is to make inferences about a population from a sample. In practice, the sample size used in a study is determined based on the expense of data collection, and the need to have sufficient statistical power. In complicated studies there may be several different sample sizes involved in the study: for example, in as survey sampling involving stratified sampling there would be different sample sizes for each population. In a census, data are collected on the entire population; hence the sample size is equal to the population size. In experimental design, where a study may be divided into different treatment groups, there may be different sample sizes for each group.
 
Sample size is statistically determined by using the following:
 
1. Degree of variation - s (sigma) or Standard Deviation for Continuous data & p or Proportion Defective in case of Discrete data.
Higher the degree of variation, higher is the sample size required.
 
2. a (Alpha). Za = 1 - a = Confidence Level is the Probability of correctly accepting the null hypothesis (detecting no change when there is none). Typically a is set at .05 or 5%. Confidence Level of 95% means that if your audit on the sample shows a result of 70%, there is 95% chance that the audit results of the entire population will be 70%.
Higher the a, lesser the sample size as one is willing to take more risk.
 
3. D (Delta). Margin of Error or Confidence interval – it is set as per how close you want to be to the Target. For example, you have rolled out a survey and have used a confidence interval of 4. If 47% percent of your sample says “OK” , you can be "sure" that if you had asked the question of the entire relevant population, between 43% (47-4) and 51% (47+4) would have also said “OK”.
Higher the margin of error, lesser the sample size as one has settled to be less precise.
 
4. N or Population size (if known) – is used to “correct” the sample size for the known finite population.
The following are used for sampling for hypothesis testing only.
 
5. b (Beta). 1- b = Power is the Probability of correctly accepting the alternative hypothesis (detecting a change when there is one). Typically b is set at .1 or 10%.
Higher the b, lesser the sample size as one is willing to take more risk.
 
6. Whether One sided or 2-sided distribution; in other words, whether Alternate hypothesis is > or < instead of ≠ (does not equal).
Sample Size is higher for one sided or “≠” alternate hypothesis testing.
 
The formulae for calculating sample size are as follows:
 
Continuous data formula:
n = ((Za/2 * s)/D) ^2
 
Discrete data formula:
n = ((Za/2)/D) ^2 * p (1-p)
 
In case N (Population size) is known, multiply n (sample size) by (N-n) / (N-1) to determine corrected sample size.
 
Now, all of the above is great & nice to know, but here the question is how to determine the right sample size.
 
The Basic question:
How can I make sure that the data I get really represents the population?
• Can samples fool me? How will I know that I can trust my samples?
• How many samples do I need?
• Why so many? (How can we get away with so few?)
 
As a quality professional, I have faced the above questions several times and frankly, even asked myself the same sometimes. A honest manager of a service center once asked me “Why even determine sample size – can't I just plan it as per my team’s availability?”
 
The answer is “Absolutely not”. The right sampling strategy goes a long way in improving processes by reducing the cost of poor quality. Cost of poor quality is nothing but rework, callbacks, customer escalations, impact to downstream processing time & quality.
 
Firstly, the Four Basic Sampling Strategies
• Random Sample – Population Studies. Each unit has an equal probability of being selected in a sample.
• Stratified Random Sample – Population Studies. Randomly sample within a stratified category or group. Sample sizes for each group are generally proportional to the relative size of the group.
• Systematic Sampling – Population or Process Studies. Sample every nth unit. For example, collecting every 4th unit.
• Rational Subgroup – Process Studies. Each unit is collected at point “A” in a process every nth hour. Usually multiple sequential units are collected.
 
In a service center, we typically end up using the random sampling strategy, but it isn’t always the best one.
For example, there are 10 processors / call centre execs in a process & out of them, there are 3 who make the maximum errors, let’s say their contribution to overall errors is between 80-90%. Shouldn’t I be using stratified random sampling in this case rather than auditing 10% for everyone. Similarly, the team works on 15 activities/ worktypes/ call types of varying complexities, so shouldn’t the sampling strategy be adjusted to allow more auditing for the more complex activities/ worktypes/ call types which have higher chances of error.
 
Smaller sample sizes vs. Larger sample sizes:
• Less Cost / Higher costs
• Quicker data collection / Longer time to get data
• Wider confidence intervals and/or more risk of missing the population parameter / Tighter confidence intervals and/or less risk of missing the population parameter.
 
To sum it up,
Sample size for determining population mean or proportion depends on:
• What level of risk you’re willing to take? If your defects in your output have an impact on loss of human life or property, severely impacts the customer or has a compliance implication, the confidence level needs to be very high.
• What size difference you want to detect? If your accuracy target is 99%, you need to take a margin of error of less than 1%.
• How much variation is in the population? Consider that your team is performing at 99.5% accuracy vs. a target of 90%, in such a case you need lesser samples because you are almost sure that the team (population data) will meet the target.
• Before collecting data, you should think about the sampling strategy and sample size requirements to ensure that you have an appropriate amount of data for drawing conclusions.
lata.neogi.

Member Statistics

  • 57,289 Total Members
  • 1,305 Most Online
  • John _Paul A_dp0u Newest Member ·

Who's Online (See full list)

  • There are no registered users currently online

Account

Navigation

Search

Search

Configure browser push notifications
Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.