Jump to content

Aseem Shandilya

Sparks Nov'13
  • Content Count

  • Joined

  • Last visited

Community Reputation

2 Average

1 Follower

About Aseem Shandilya

  • Rank
    Active Member
  • Birthday 05/03/1989

Profile Information

  • Name
    Aseem Shandilya
  • Company
    Institute of Management Technology, Ghaziabad
  • Designation
  1. Thanks for your question Vipul. I really appreciate the passion with which you are following the article. It shows that I did something right to invoke such an enthusiastic response from a reader. I agree, I have indeed made the claim that “The benefit of D3M is that it gives you the ability to quantify your decision and you have absolute control over the decision making process”. Having said that, allow me to explain my intention in stating the above. By the above statement, I mean that the decision maker has absolute control over the decision making process. By the process, I mean setting up the model, deciding what kind of data you need, what kind of output you are looking for etc. Now, the success of said process depends on whether you have exercised your control correctly or not. Here, I would like to re-iterate what I had mentioned about Lehman brothers. Again, I am speculating but in all probability, Lehman brothers did not use their control properly (ignored the data warnings, as mentioned) and had to suffer for it. However, I do concede to your point that there is no guarantee of success using the D3M. So, would it be better if I stated the above as that “The benefit of D3M is that it gives you the ability to quantify your decision and you have absolute control over the decision making process. However, the success of the process depends on how it is defined by the decision maker in the first place”. I had missed mentioning this point in the article. Thank you for pointing it out. Coming to the second part of your question, which is indeed a rather unique perspective that you have given to the article. Suffice to say, I had to do some digging to come up with an answer for it. To start, I agree with you completely that numbers can be changed while abilities cannot be. Turns out, the world of data agrees with you too. As per this article in the New York Times; Google, one of the leading experts on data handling in the world has shifted the way that it looks for employees. Also, this article was written way back in 2007. Google too recognizes the importance of personality traits in making a hire. Now conventionally, data would be restricted to marks and past employment etc. just as you say. However, now Google has begun quantifying other aspects of the personality as well such as hobbies, interests, emotional quotient etc. Now, using this, I would like to make my point that “Data is not restricted to numbers alone. Not anymore”. Therefore, there is much more to this process than meets the eye. Therefore, the decision making can be done for all the scenarios that you have pointed out using the D3M. One only needs to change the model for decision making. Another point from you that I agree with is that the D3M is not to be trusted blindly. The reason for that is “Garbage in, garbage out”. The output of the D3M may be corrupted due to a corrupt input. Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. Regards, Aseem
  2. Thank you for your kind words Pranuj. I am glad I was able to clear up your doubts. Regarding data supporting D3M, here is an article that discusses the results of a survey conducted by Teradata. The findings of the article are shown below in the chart. This chart describes what respondents feel about the benefits of using data in making decisions. The article discusses the output of a survey conducted among 2200 volunteers Quoting verbatim from the article - “Separately in the study, Teradata asked a similar question about the benefits of using data in decision-making, providing a list of answers much broader in scope. According to those results, 58% believe the use of data can help them make more accurate decisions, 49% believe it helps them achieve better business results, 44% see it resulting in more efficient use of resources and reduced cost, and 43% agree that it helps identify new opportunities and new competitive advantages.” Regarding the second part of your question, the sudden increase in interest when it comes to D3M is primarily due to the huge amounts of information available to companies. Add to that the increased levels of competition; it is imperative for every company to implement D3M so that it does not lag behind other companies. Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. Regards, Aseem
  3. Thank you, Vipul. I am glad that I was able to make the case for D3M a little stronger. In order to answer your question, let me elaborate a little on the reasons that I feel are the most responsible for the Lehman brothers’ crash. I refer the most to this article for my analysis of the same. 1.) Ignored warnings – Lehman brothers saw a lucrative opportunity in providing the mortgages properties available as securities for sale to investors. This opportunity was a fickle one because it was born out of the manner in which the mortgage brokers were giving out housing loans to people. So much so, that many people were unable to pay back the mortgage. Consequently, the housing bubble burst and the real estate prices tanked. This led to the demand for mortgage securities vanishing. All the banks had been warned about the impending doom of the real estate industry and advised to let up on the practice of giving out mortgaged properties as securities. However, Lehman brothers chose to ignore this warning and kept investing heavily in this field. This went on to the point that they had taken too much debt. Quoting the article – “The bank held less than a dollar in reserves for every $30 of its liabilities” which shows how negligent the bank had become. 2.) Bailout refusal – The fed refused to bailout Lehman brothers in spite of the expectations. Everyone was expecting the fed to bailout Lehman like it did for AIG after the Lehman debacle. Add to the fact that most of the banks were in no position to help out Lehman for several reasons, it only made Lehman brothers’ grave deeper. Now with regards to why in spite of the data being there, did Lehman brothers fall? Here are the reasons: 1.) Confabulation – The D3M is designed to make a suggestion at the end of its processing. It is up to the user to ignore or accept it. While I cannot be too sure of what output did the D3M give for Lehman brothers, it depends on too many parameters like what data was it given, what was the model prescribed etc.; what I can say for sure is that ignoring the warning given by the D3M can often lead to disastrous results. As Lehman chose to ignore the warning about the housing bubble, one can only assume that it did the same with the warnings it got from the D3M. Again, nothing can be said conclusively until we know more details. 2.) D3M is not a guarantee – D3M is designed to be an assisting tool. Most of the important roles are still left to the decision makers. Even then, the D3M does work on a certifiable margin of error that changes subject to the parameters under which it is working. Even with the D3M, both the decision maker and the system go through a learning curve which helps them take better decisions with time. Having said that, I will not be completely one sided, I do admit that even the D3M makes a mistake at times. It is not a perfect system either. Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. Regards, Aseem
  4. Thanks Pranuj. I am really glad my article was able to interest you so much. When we say that a decision makers need to ask himself-"Whether I want the correct answers – quickly if possible or do I want the answers quickly – correct if possible?" we are talking in relative terms. I agree with you when you say that a firm is all about making profits and that they will be looking for an accuracy of 95% in all their results. Having said that let me try to explain the aforementioned phrase using an example. Suppose there is a company ABC which is into manufacturing light bulbs. It has to launch its product in a new market, say the city of Delhi. However, at the same time, its competitor is working on a similar product and is working towards launching its product too. So, it is natural that ABC will want to launch its product first in order to make the most of the market. At a time like this, ABC cannot afford to spend too much time trying to perfect its data for decision making. Therefore, it reaches a compromise. It literally says that this time we go for about 94.5% accuracy; in exchange, I speed up my decision making so that I can launch my product first. That is what is meant by the above phrase. The balance between time and accuracy always exists. In order to gain one, you will have to sacrifice the other. Equivalent exchange, so to speak. Even then, yes I will agree with you and say that there is a limit to how much accuracy can be sacrificed. Deciding that is up to the manager and varies from industry to industry. I am really glad you asked me to highlight the positives of the D3M. I shall try my best to explain it as per my understanding. 1.) Reduced effortàIncreased efficiency- As I have mentioned in my answer to Vipul’s question above, the D3M is a tool that goes through a sea of information on behalf of you and extracts the data that is relevant to your decision making. As a result, the efficiency of the decision making process as a whole goes up because the effort that you would give to sifting through data can now be diverted to analysing it. 2.) More accountabilityàLesser grey areas- Every firm literally has entire servers’ worth of data about things like employee activity. It would be very easy for an employee to hide some discrepancies or deceitful actions in so much data without letting anyone know. However, now with the D3M and its associated data mining tools, any discrepancy in the data, no matter how small is easily identifiable. As a result, there are lesser grey areas in your firm since everyone is accountable for his/her actions. 3.) Standardization & Customisation- With the D3M, the problem of decisions changing radically with the change in decision makers is reduced drastically. This is because with the D3M, as long as the model defining it remains the same, the output shall vary within some standards that have been defined. This ensures a consistent performance by the firm, irrespective of who comes in to make the decisions. This does not mean that the decisions will remain absolutely same. They will vary, but within a set limit. Another feature of the D3M is that it is easily customisable. It does not have a specialization or area of expertise on its own. Whichever model is laid onto the system, the D3M becomes an expert in that. Therefore, it can be used to take decisions across all departments in a firm by changing its model every time. Regarding best practices and implementation strategies, I think it is best explained by Mr. Avinash Kaushik in his article. He mentions seven common sense steps to best implement the D3M by establishing a culture for it. I choose to discuss the six most important ones. The article is from the point of view of a web analytics agency. All the below mentioned points are his own. I am just paraphrasing them: 1.) Identify needs – The Company needs to identify what is it that the customer hopes to attain using the D3M, that way you can send them focused bits and pieces of data instead of providing them access to the whole database. Similarly, when you are implementing D3M in your firm, you need to identify what is it that you want out of it? 2.) Differentiate between reporting and analysis – It is quite easy to generate several hundred reports chock full of data in the hopes that some of those reports will give you an insight. But, D3M is not about reporting, it’s about analysis. So, a culture has to be established to lean towards analysis and not reporting. 3.) Depersonalize decision making – Avinash mentions that it is never about you, but your customer. Therefore, a need arises to depersonalize the decision making process. In order to do so, you should go for tools such as benchmarking, the outside in approach etc. 4.) Be proactive, not reactive – You may not always have to make a decision, but that does not mean that during such times, the D3M is idle. Your firm should have a culture of ever on-going analysis. This way, when the time comes, the analysis done over a period of time can be fed into the D3M to get the best results. 5.) Hire critical thinkers as analysts – When you wish to make the best out of the D3M, it is advisable to hire analysts who are critical thinkers. Also, they should be given a set of parameters in which they work, so that data analysis is promoted. 6.) Establish processes – In order to make the best of D3M, it is of absolute importance that there are processes in place that are in sync with its goals. The reason for this is simple- “It is not so much about the tool itself, than it is about the accompanying processes”. Few of the processes that need to be in place are: a. Processes for data collection – These processes insure the cleanliness of data. b. Processes for data storage – These need to be done in a proper manner so that retrieval is not a hassle. c. Processes for data analysis Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. Regards, Aseem
  5. Thanks Apoorva, I am very grateful my article got selected. Regarding your question, there is no decision that can be taken without data. It is just that the form of data keeps changing. What we know as experience is nothing but past data. This is especially true in a business perspective. Therefore, every decision in a business, no matter how small, can be taken using the D3M. Having said that, there are a few situations in which using the D3M model will not be advisable: 1.) When the data cannot be obtained economically- The D3M is all about data. Without the right data, it cannot be used to obtain a usable result. Therefore, if ever there arises a situation when obtaining the data in order to use the D3M is simply too costly for the organisation, it is advisable not to go for D3M. 2.) When the data is not relevant/timely- As is true for life, no decision can be made if the data does not refer to the relevant time frame. For example, if a product has to be launched within the next seven days based on the data on last Diwali sales and the data for Diwali are available for a year before the one required, it will not be advisable to go for D3M. Simply because the output would not be accurate. 3.) When the data is not of high quality- As mentioned in the article, D3M follows the policy of “Garbage in, Garbage out”. If the data available for decision making is of low quality, the quality of the output will suffer too. So, if the data has too many lapses or incorrect values, it is not advisable to go for D3M. All in all, if the data is contaminated or compromised in any manner, it is not advisable to go for D3M. Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. Regards, Aseem
  6. Thanks Vipul, I am glad you find the article interesting. Regarding your question, in the short run, it may seem so that there is simply too much work involved. However,the intuition that we are talking of here is different from daily life intuition. In daily life, intuition is more or less synonymous with instinct. On the contrary, in a business perspective, intuition is a product derived from past experiences. It is quite easy to mistake one for the other. It happens primarily due to the similarities between the intuition based decision making and D3M. Allow me to explain: For any decision to be made, we need data. In the past, the volume of data was small hence it could be processed easily by the human brain. However, given the current status of technology, there is simply too much data for an individual to handle all by himself. D3M helps you by extracting the relevant data from the sea of data available. That way, your decisions are much more effective and the probability of overlooking something is reduced drastically. This has been put most beautifully by Mr. Avinash Kaushik in his article where he states the benefit of using the D3M as: Data : petabytes Reports : terabytes Excel : gigabytes PowerPoint : megabytes Insights : bytes One business decision based on actual data: Priceless The D3M is a filtering tool that saves you the effort of going through virtually millions of sheets trying to find data that is relevant to your decision making. Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. Regards, Aseem
  7. Thanks Angshuman, those words mean a lot to me. I agree with you completely that every individual using the D3M will have his/her own interpretation and use of it. It is true that everyone will have a different intuition which will affect the D3M system differently at each stage. The D3M is a mere tool in the hands of the decision maker. Therefore, it can be affected by whoever is using it at the following steps: 1.) Deciding data parameters – Every business will have different parameters for the input data that is required. Here, to avoid the problem of choosing excessive/low quality data it is imperative to use instinct honed over the years. 2.) Making preliminary assumptions – Intuition will guide the decision maker in making assumptions that fit the requirements. I would like to stress here that having the right kind of assumptions in the beginning can improve the quality of output drastically. 3.) Evaluating output – Once the output has been obtained after data-processing, it is necessary to evaluate it. Here, there are two options in front of the decision maker; consider the output given by the system and modify the initial assumptions as needed or ignore the output and implement the solution. Here again, intuition will guide a decision manager decide. Having said that, I would like to point out that the D3M will be affected by the change in people/decision makers but its operations will change very little owing to the fact that the decision makers will be constrained by the institution that they make decisions for. As long as their intuition is guided by the norms and culture of the organisation, the D3M will not be affected significantly in the long run. Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. Regards, Aseem
  8. Thanks Pranuj for appreciating my article. Glad you liked it. Moving on to your question, there are two basic categories that I would like to divide problems or errors into. 1.) Overthinking- a. Time inconsistency- This phenomenon is all about timeliness. Think of it this way, when a police officer has to shoot a criminal, if he were to try and account for every possibility (Bullet ricocheting off a surface, the criminal running away, the gun misfiring etc.); he would not be able to take the shot in time, thus endangering many more innocent lives. When you are making a business decision, you start by making a list of basics (What you need to know, what you hope to achieve etc.) and then you go on to add miscellaneous things into your list as and when you see fit. Now if you were to include everything and keep trying to achieve perfection, you will definitely miss your window of opportunity and your business will suffer. Hence, you need to choose between two options - “Whether you want the correct answers – quickly if possible or do you want the answers quickly – correct if possible?” Please observe how these two similar sounding phrases convey altogether different meanings. Choosing either of these options will depend on what kind of a decision are you going to make. b. Excess Volume- The data today has three basic parameters Volume, Velocity (Here, the rate at which data changes) and Variety. Owing to the high velocity and variety of data, coupled with the sheer size of the human race and the current level of technology the volume of data is too high. As a result, you have to look for only that much data which is the bare minimum for your needs. To quote experts- you need to be choose data that is satisficing (sufficient to be satisfying). Taking too much data into consideration will unnecessarily increase calculations and might even throw your out off. 2.) Data biases- Both of these biases are in the way we see data and are born out of basic human tendencies. In both these situations, we fail to utilize D3M properly. a. Pre-decision data bias (Psychologically called confirmation) - This happens when we justify our pre-conceived notions and assumptions using the data. This way we are ignoring what the processed data is trying to say. It would not be wrong to say that we are blinded by our own pre-conceived notions. b. Post-decision data bias (Psychologically called confabulation) – This happens when we refuse to believe what the data has concluded. We then proceed to modify the data or weed out the unwanted parts to arrive at the answer that we “wanted”. Here, it is important to point out that it is human nature to ignore errors if we feel that they are coming in the way of a favourable decision or solution. These are the basic situations that if not watched out for may cause problems to arise in the future. Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. Regards, Aseem
  9. You have been making decisions on the basis of your gut; it has never led you astray. Undoubtedly, you have been hearing this buzz about using data to make decisions around you. Can Data Driven Decision Making (D3M) be trusted? Is it worth all the hoopla surrounding it? Let us find out. As the name suggests, D3M is literally making decisions based on the analysis of data or information. Primarily, the information that must be analysed has to be relevant and there has to be a sufficient amount to help you make your decision. We shall discuss more about the nitty-gritties later as we dissect this term further. The concept of D3M was born out of necessity. Today, any action by a human causes data to be generated. According to an article on TechnologyReview, in the year 2012, 2.8 zettabytes of data was generated. That is almost equal to 3 billion hard disks of 1 terabyte each. A one terabyte hard disk can store 250,000 songs. Mind boggling, is it not? Even if you were to deal with the smallest fraction of the human race, there is still too much data for a mere mortal to handle. Enter the concepts of Big Data and D3M; these concepts have perhaps single-handedly revolutionized the way we make decisions. Surprisingly, the concept of D3M has been around for a really long time; organisations started using it as early as the 1980s. It is just the scale and corresponding technology that has evolved. It is best stated in an article by Pamela Wheaton Shorr back in 2003 where she states that making a decision without data is similar to flying a plane in the dark without any radar. She also states that using D3M makes sure that everyone is held accountable; in her words-“ there is nowhere to hide†because the analysis of collected data can find out the most minute of discrepancies ensuring you do not lose any money to such irregularities. Contrary to what many people believe, D3M is not that different from the methods that we have been using to make decisions. For starters, it is used all across the organization in different capacities as stated by Irving Wladawsky-Berger in this article on the Wall Street Journal. Henceforth, I aim to make it clear that D3M is an extension to the current intuition based decision making and not a replacement. As mentioned by several experts, every decision is made two times- a.) Intuition: We make a decision on the basis of our intuition and past knowledge b.) Data driven: Once data is available, we decide to continue with or forego the decision altogether. The benefit of D3M is that you can combine these steps into one and get better value for the time spent in making that decision. Even though it is not apparent, what we call intuition and belief are often a sum total of data. Our belief is born out of our past experiences; that too is data evaluated by our brain. So in essence when we talk about D3M, we have brought out the workings of our brain into the real world. Just like our brain, the D3M too has points which if not watched out for may cause incorrect decisions to be made. i.) Time consistency : Any decision that we make is of use to us only within a limited time frame. Similarly, for any decision made through the D3M system, every decision is good as long as it is made on time. What causes a delay in decision making? The answer is rather simple when you think about it. Just like in life, here too, you cannot have the perfect answer. It is important to draw the line somewhere. Before making any decision using the D3M, you have to ask yourself- “Whether you want the correct answers – quickly if possible or do you want the answers quickly – correct if possible?†Something we must do in life as well. ii.) How much data is enough data ? : While making decisions conventionally, it is quite possible for us to overthink things and in trying to account for everything, we fail to make an apt decision in time. The equivalent of this in terms of D3M is the above mentioned question. Here, experts advise us to use our instinct and decide the bare minimum amount of data that we need to make our decision. Most experts give a fairly simple mantra to highlight the importance of this concept- “ Garbage in, garbage out â€; this phrase highlights that the better the quality of data you put in for analysis, the better the results. It is here that your formidable intuitive skills will be useful the most. iii.) Confirmation : More often than not, we tend to pre-formulate an opinion or decision in our minds and instead of arriving at that decision through analysis; we use that analysis to verify our decision. Most of the times, we tend to ignore any discrepancies between the suggested decision and our pre-decided decision. This causes us to make the wrong decisions. An easy way out is to be ready to make changes to our initial assumptions. This is similar to the way we streamline our decisions in real life based upon feedback. In the discussion so far, I have tried to cover all the important details about D3M. The purpose of presenting even the possible issues above is to show you that this system is just an extension of what you have been doing till now. The benefit of D3M is that it gives you the ability to quantify your decision and you have absolute control over the decision making process. You do not have to take my word for it; just refer to the links provided within the article and make sure for yourself. References: 1. Decision-Driven Data Management: A Strategy for Better Decisions with Better Data (SAS White paper) 2. Other references mentioned inline Note - Visitors shall not be able to comment on this article until they are logged in.
  • Create New...