Data Driven Decision Making: Helping You Decide

Dear Aseem,

 

First of all a vote of appreciation for this nicely written article.

 

In the article you mentioned three issues which if not taken into care can lead to erroneous decision. Could you please throw some more light on these issues?

 

Plus are there any other important factors apart from the mentioned three which if not taken care of may lead to a bad decision making in the long run?

 

Regards,

Pranuj Singhal

Mr Aseem,

 

The way you have written this article is really commendable.

 

However, I have one doubt in my mind. You would definitely agree upon the fact that every person has a different instinct. You mention that even D3M is instinct driven. Don't you think the decision taken even using the D3M will change as decision makers change?

Regards,

Angshuman Sarkar

Hi Aseem,

 

I appreciate the way you write and have made things clear, but I have a a small doubt.

 

When you say deciding about source and quantity of data is based on intuition, don't you think there is too much extra effort involved? First one has to arrive at a consensus with the team regarding the data parameters, then again when the data is processed, the validity of assumptions and the answer is evaluated once more. I feel that this is simply too much work and taking decisions on the basis of intuition would have been an easier affair.

 

Thanks and regards,

Vipul Kumar Singh

Hi Aseem,

 

First of all congratulations for the article.

 

Just wanted to know that are there some decisions exempted from D3M? Knowing this would save someone the hassle of needlessly trying to implement D3M.

 

Regards

Apoorva Nangia

Dear Aseem,

 

First of all a vote of appreciation for this nicely written article.

 

In the article you mentioned three issues which if not taken into care can lead to erroneous decision. Could you please throw some more light on these issues?

 

Plus are there any other important factors apart from the mentioned three which if not taken care of may lead to a bad decision making in the long run?

 

Regards,

Pranuj Singhal

Thanks Pranuj for appreciating my article. Glad you liked it.

Moving on to your question, there are two basic categories that I would like to divide problems or errors into.

1.)    Overthinking-

a.       Time inconsistency- This phenomenon is all about timeliness. Think of it this way, when a police officer has to shoot a criminal, if he were to try and account for every possibility (Bullet ricocheting off a surface, the criminal running away, the gun misfiring etc.); he would not be able to take the shot in time, thus endangering many more innocent lives.

When you are making a business decision, you start by making a list of basics (What you need to know, what you hope to achieve etc.) and then you go on to add miscellaneous things into your list as and when you see fit. Now if you were to include everything and keep trying to achieve perfection, you will definitely miss your window of opportunity and your business will suffer. Hence, you need to choose between two options - “Whether you want the correct answers – quickly if possible or do you want the answers quickly – correct if possible?” Please observe how these two similar sounding phrases convey altogether different meanings. Choosing either of these options will depend on what kind of a decision are you going to make.

b.      Excess Volume- The data today has three basic parameters Volume, Velocity (Here, the rate at which data changes) and Variety. Owing to the high velocity and variety of data, coupled with the sheer size of the human race and the current level of technology the volume of data is too high. As a result, you have to look for only that much data which is the bare minimum for your needs. To quote experts- you need to be choose data that is satisficing (sufficient to be satisfying). Taking too much data into consideration will unnecessarily increase calculations and might even throw your out off.

2.)    Data biases- Both of these biases are in the way we see data and are born out of basic human tendencies. In both these situations, we fail to utilize D3M properly.

a.       Pre-decision data bias (Psychologically called confirmation) - This happens when we justify our pre-conceived notions and assumptions using the data. This way we are ignoring what the processed data is trying to say. It would not be wrong to say that we are blinded by our own pre-conceived notions.

b.      Post-decision data bias (Psychologically called confabulation) – This happens when we refuse to believe what the data has concluded. We then proceed to modify the data or weed out the unwanted parts to arrive at the answer that we “wanted”. Here, it is important to point out that it is human nature to ignore errors if we feel that they are coming in the way of a favourable decision or solution.

These are the basic situations that if not watched out for may cause problems to arise in the future.

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. 

Regards,

Aseem

Mr Aseem,

 

The way you have written this article is really commendable.

 

However, I have one doubt in my mind. You would definitely agree upon the fact that every person has a different instinct. You mention that even D3M is instinct driven. Don't you think the decision taken even using the D3M will change as decision makers change?

Regards,

Angshuman Sarkar

Thanks Angshuman, those words mean a lot to me.

I agree with you completely that every individual using the D3M will have his/her own interpretation and use of it. It is true that everyone will have a different intuition which will affect the D3M system differently at each stage.

The D3M is a mere tool in the hands of the decision maker. Therefore, it can be affected by whoever is using it at the following steps:

1.)    Deciding data parameters – Every business will have different parameters for the input data that is required. Here, to avoid the problem of choosing excessive/low quality data it is imperative to use instinct honed over the years.

2.)    Making preliminary assumptions – Intuition will guide the decision maker in making assumptions that fit the requirements. I would like to stress here that having the right kind of assumptions in the beginning can improve the quality of output drastically.

3.)    Evaluating output – Once the output has been obtained after data-processing, it is necessary to evaluate it. Here, there are two options in front of the decision maker; consider the output given by the system and modify the initial assumptions as needed or ignore the output and implement the solution. Here again, intuition will guide a decision manager decide.

Having said that, I would like to point out that the D3M will be affected by the change in people/decision makers but its operations will change very little owing to the fact that the decision makers will be constrained by the institution that they make decisions for. As long as their intuition is guided by the norms and culture of the organisation, the D3M will not be affected significantly in the long run.

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. 

Regards,

Aseem

Hi Aseem,

 

I appreciate the way you write and have made things clear, but I have a a small doubt.

 

When you say deciding about source and quantity of data is based on intuition, don't you think there is too much extra effort involved? First one has to arrive at a consensus with the team regarding the data parameters, then again when the data is processed, the validity of assumptions and the answer is evaluated once more. I feel that this is simply too much work and taking decisions on the basis of intuition would have been an easier affair.

 

Thanks and regards,

Vipul Kumar Singh

Thanks Vipul, I am glad you find the article interesting.

 

Regarding your question, in the short run, it may seem so that there is simply too much work involved. However,the intuition that we are talking of here is different from daily life intuition. In daily life, intuition is more or less synonymous with instinct. On the contrary, in a business perspective, intuition is a product derived from past experiences. It is quite easy to mistake one for the other. It happens primarily due to the similarities between the intuition based decision making and D3M. Allow me to explain:

For any decision to be made, we need data. In the past, the volume of data was small hence it could be processed easily by the human brain. However, given the current status of technology, there is simply too much data for an individual to handle all by himself. D3M helps you by extracting the relevant data from the sea of data available. That way, your decisions are much more effective and the probability of overlooking something is reduced drastically.

This has been put most beautifully by Mr. Avinash Kaushik in his article where he states the benefit of using the D3M as:

• Data : petabytes

• Reports : terabytes

Excel : gigabytes

PowerPoint : megabytes

Insights : bytes

One business decision based on actual data: Priceless

 

The D3M is a filtering tool that saves you the effort of going through virtually millions of sheets trying to find data that is relevant to your decision making.

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made.

Regards,

Aseem 

Hi Aseem,

 

First of all congratulations for the article.

 

Just wanted to know that are there some decisions exempted from D3M? Knowing this would save someone the hassle of needlessly trying to implement D3M.

 

Regards

Apoorva Nangia

Thanks Apoorva, I am very grateful my article got selected.

 

Regarding your question, there is no decision that can be taken without data. It is just that the form of data keeps changing. What we know as experience is nothing but past data. This is especially true in a business perspective. Therefore, every decision in a business, no matter how small, can be taken using the D3M.

Having said that, there are a few situations in which using the D3M model will not be advisable:

1.)    When the data cannot be obtained economically- The D3M is all about data. Without the right data, it cannot be used to obtain a usable result. Therefore, if ever there arises a situation when obtaining the data in order to use the D3M is simply too costly for the organisation, it is advisable not to go for D3M.

2.)    When the data is not relevant/timely- As is true for life, no decision can be made if the data does not refer to the relevant time frame. For example, if a product has to be launched within the next seven days based on the data on last Diwali sales and the data for Diwali are available for a year before the one required, it will not be advisable to go for D3M. Simply because the output would not be accurate.

3.)    When the data is not of high quality- As mentioned in the article, D3M follows the policy of “Garbage in, Garbage out”. If the data available for decision making is of low quality, the quality of the output will suffer too. So, if the data has too many lapses or incorrect values, it is not advisable to go for D3M.

All in all, if the data is contaminated or compromised in any manner, it is not advisable to go for D3M.

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made.

Regards,

Aseem

Thanks Pranuj for appreciating my article. Glad you liked it.

Moving on to your question, there are two basic categories that I would like to divide problems or errors into.

1.)    Overthinking-

a.       Time inconsistency- This phenomenon is all about timeliness. Think of it this way, when a police officer has to shoot a criminal, if he were to try and account for every possibility (Bullet ricocheting off a surface, the criminal running away, the gun misfiring etc.); he would not be able to take the shot in time, thus endangering many more innocent lives.

When you are making a business decision, you start by making a list of basics (What you need to know, what you hope to achieve etc.) and then you go on to add miscellaneous things into your list as and when you see fit. Now if you were to include everything and keep trying to achieve perfection, you will definitely miss your window of opportunity and your business will suffer. Hence, you need to choose between two options - “Whether you want the correct answers – quickly if possible or do you want the answers quickly – correct if possible?” Please observe how these two similar sounding phrases convey altogether different meanings. Choosing either of these options will depend on what kind of a decision are you going to make.

b.      Excess Volume- The data today has three basic parameters Volume, Velocity (Here, the rate at which data changes) and Variety. Owing to the high velocity and variety of data, coupled with the sheer size of the human race and the current level of technology the volume of data is too high. As a result, you have to look for only that much data which is the bare minimum for your needs. To quote experts- you need to be choose data that is satisficing (sufficient to be satisfying). Taking too much data into consideration will unnecessarily increase calculations and might even throw your out off.

2.)    Data biases- Both of these biases are in the way we see data and are born out of basic human tendencies. In both these situations, we fail to utilize D3M properly.

a.       Pre-decision data bias (Psychologically called confirmation) - This happens when we justify our pre-conceived notions and assumptions using the data. This way we are ignoring what the processed data is trying to say. It would not be wrong to say that we are blinded by our own pre-conceived notions.

b.      Post-decision data bias (Psychologically called confabulation) – This happens when we refuse to believe what the data has concluded. We then proceed to modify the data or weed out the unwanted parts to arrive at the answer that we “wanted”. Here, it is important to point out that it is human nature to ignore errors if we feel that they are coming in the way of a favourable decision or solution.

These are the basic situations that if not watched out for may cause problems to arise in the future.

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made. 

Regards,

Aseem

 

Dear Aseem,

 

Could you please throw some more light on "Whether you want the correct answers – quickly if possible or do you want the answers quickly – correct if possible?" How this quote is related to D3M model. Any decision maker will always like to go for a minimum of 95% positive result oriented decision. And not everybody will prefer to take the risk of taking quick decision - correct if possible. In this business world, all the firm wants is profit. And to get the maximum profit, the firm believes in taking the best possible decision taking there best available time. And we all know, taking decisions without taking time into consideration may lead to a blunder.

 

Plus I would also like you to highlight few positives of this D3M model. In the entire article you highlighted the need of D3M insisting on factors which if not taken care of can lead to erroneous decision. Please provide some insights on the positives of the model you proposed. I would also appreciate if you please provide some best practices and strategies for adopting this model.

 

Thanks & Regards,

Pranuj Singhal

Dear Aseem,

 

Could you please throw some more light on "Whether you want the correct answers – quickly if possible or do you want the answers quickly – correct if possible?" How this quote is related to D3M model. Any decision maker will always like to go for a minimum of 95% positive result oriented decision. And not everybody will prefer to take the risk of taking quick decision - correct if possible. In this business world, all the firm wants is profit. And to get the maximum profit, the firm believes in taking the best possible decision taking there best available time. And we all know, taking decisions without taking time into consideration may lead to a blunder.

 

Plus I would also like you to highlight few positives of this D3M model. In the entire article you highlighted the need of D3M insisting on factors which if not taken care of can lead to erroneous decision. Please provide some insights on the positives of the model you proposed. I would also appreciate if you please provide some best practices and strategies for adopting this model.

 

Thanks & Regards,

Pranuj Singhal

Thanks Pranuj. I am really glad my article was able to interest you so much.

 

When we say that a decision makers need to ask himself-"Whether I want the correct answers – quickly if possible or do I want the answers quickly – correct if possible?" we are talking in relative terms. I agree with you when you say that a firm is all about making profits and that they will be looking for an accuracy of 95% in all their results. Having said that let me try to explain the aforementioned phrase using an example. Suppose there is a company ABC which is into manufacturing light bulbs. It has to launch its product in a new market, say the city of Delhi. However, at the same time, its competitor is working on a similar product and is working towards launching its product too. So, it is natural that ABC will want to launch its product first in order to make the most of the market. At a time like this, ABC cannot afford to spend too much time trying to perfect its data for decision making. Therefore, it reaches a compromise. It literally says that this time we go for about 94.5% accuracy; in exchange, I speed up my decision making so that I can launch my product first.

That is what is meant by the above phrase. The balance between time and accuracy always exists. In order to gain one, you will have to sacrifice the other. Equivalent exchange, so to speak. Even then, yes I will agree with you and say that there is a limit to how much accuracy can be sacrificed. Deciding that is up to the manager and varies from industry to industry.

 

I am really glad you asked me to highlight the positives of the D3M. I shall try my best to explain it as per my understanding.

1.)    Reduced effortàIncreased efficiency- As I have mentioned in my answer to Vipul’s question above, the D3M is a tool that goes through a sea of information on behalf of you and extracts the data that is relevant to your decision making. As a result, the efficiency of the decision making process as a whole goes up because the effort that you would give to sifting through data can now be diverted to analysing it.

2.)    More accountabilityàLesser grey areas- Every firm literally has entire servers’ worth of data about things like employee activity. It would be very easy for an employee to hide some discrepancies or deceitful actions in so much data without letting anyone know. However, now with the D3M and its associated data mining tools, any discrepancy in the data, no matter how small is easily identifiable. As a result, there are lesser grey areas in your firm since everyone is accountable for his/her actions.

3.)    Standardization & Customisation- With the D3M, the problem of decisions changing radically with the change in decision makers is reduced drastically. This is because with the D3M, as long as the model defining it remains the same, the output shall vary within some standards that have been defined. This ensures a consistent performance by the firm, irrespective of who comes in to make the decisions. This does not mean that the decisions will remain absolutely same. They will vary, but within a set limit.

Another feature of the D3M is that it is easily customisable. It does not have a specialization or area of expertise on its own. Whichever model is laid onto the system, the D3M becomes an expert in that. Therefore, it can be used to take decisions across all departments in a firm by changing its model every time.

 

Regarding best practices and implementation strategies, I think it is best explained by Mr. Avinash Kaushik in his article. He mentions seven common sense steps to best implement the D3M by establishing a culture for it. I choose to discuss the six most important ones. The article is from the point of view of a web analytics agency. All the below mentioned points are his own. I am just paraphrasing them:

1.)    Identify needs – The Company needs to identify what is it that the customer hopes to attain using the D3M, that way you can send them focused bits and pieces of data instead of providing them access to the whole database. Similarly, when you are implementing D3M in your firm, you need to identify what is it that you want out of it?

2.)    Differentiate between reporting and analysis – It is quite easy to generate several hundred reports chock full of data in the hopes that some of those reports will give you an insight. But, D3M is not about reporting, it’s about analysis. So, a culture has to be established to lean towards analysis and not reporting.

3.)    Depersonalize decision making – Avinash mentions that it is never about you, but your customer. Therefore, a need arises to depersonalize the decision making process. In order to do so, you should go for tools such as benchmarking, the outside in approach etc.

4.)    Be proactive, not reactive – You may not always have to make a decision, but that does not mean that during such times, the D3M is idle. Your firm should have a culture of ever on-going analysis. This way, when the time comes, the analysis done over a period of time can be fed into the D3M to get the best results.

5.)    Hire critical thinkers as analysts – When you wish to make the best out of the D3M, it is advisable to hire analysts who are critical thinkers. Also, they should be given a set of parameters in which they work, so that data analysis is promoted.

6.)    Establish processes – In order to make the best of D3M, it is of absolute importance that there are processes in place that are in sync with its goals. The reason for this is simple- “It is not so much about the tool itself, than it is about the accompanying processes”. Few of the processes that need to be in place are:

a.       Processes for data collection – These processes insure the cleanliness of data.

b.      Processes for data storage – These need to be done in a proper manner so that retrieval is not a hassle.

c.       Processes for data analysis

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made.

Regards,

Aseem 

Thanks Vipul, I am glad you find the article interesting.

 

Regarding your question, in the short run, it may seem so that there is simply too much work involved. However,the intuition that we are talking of here is different from daily life intuition. In daily life, intuition is more or less synonymous with instinct. On the contrary, in a business perspective, intuition is a product derived from past experiences. It is quite easy to mistake one for the other. It happens primarily due to the similarities between the intuition based decision making and D3M. Allow me to explain:

For any decision to be made, we need data. In the past, the volume of data was small hence it could be processed easily by the human brain. However, given the current status of technology, there is simply too much data for an individual to handle all by himself. D3M helps you by extracting the relevant data from the sea of data available. That way, your decisions are much more effective and the probability of overlooking something is reduced drastically.

This has been put most beautifully by Mr. Avinash Kaushik in his article where he states the benefit of using the D3M as:

• Data : petabytes

• Reports : terabytes

Excel : gigabytes

PowerPoint : megabytes

Insights : bytes

One business decision based on actual data: Priceless

 

The D3M is a filtering tool that saves you the effort of going through virtually millions of sheets trying to find data that is relevant to your decision making.

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made.

Regards,

Aseem 

Hi Aseem,

 

You sound very convincing regarding your thoughts on D3M and why it is so helpful for the business to grow. As you said numbers help us a lot in taking decision from past experiences. If we have data we can analyse the data and take our decision. Well i would like you to convince me a little bit more.

 

In 2008 Lehman brothers saw its downfall due to subprime crisis, should I assume that they didn't had the data? or that they didn't had the correct data? or that they expected this to happen and they were ready for bankruptcy?

 

Well this was not the case to be, after being set up in 1850 Lehman brothers had seen many problems such as the railroad bankruptcies of the 1800s, the Great Depression of the 1930s, two world wars, a capital shortage when it was spun off by American Express in 1994, and the Long Term Capital Management collapse and Russian debt default of 1998. But they survived it all without falling. It means that they had the belief in them that they have avoided these situations, so they can do it again. Now continuing with the facts Lehman Brothers boomed in the year 2006 and its  capital markets surged 56% being the highest in any investment banking or asset management industry.They acquired 5 mortgage firms including the big ones like BNC mortgage. Lehman also reported record profits every year from 2005 to 2007. In 2007 they reported net income of a record $4.2 billion on revenue of $19.3 billion. In Feb 2007, the stock reached a record $86.16, giving Lehman a market cap of close to $60 billion. Lehman brother had this data with them to support their actions, analyzing this available data and previous achievements Chief Financial Officer of Lehman brother said "the risks posed by rising home delinquencies were well contained and would have little impact on the firm's earning". The impact was "so small" that the 4th largest Investment bank lost all its wealth and went into bankruptcy. In 2008 they had all the required data at their side and they couldn't handle it while at the earlier situation which comprised of the biggest crisis in 1930s they came out easily without proper data and data management services . Why the data didn't help them in 2008 when they lost a fight against crisis?

 

I would appreciate is you could clarify my doubts.

 

 

Thanks and Regards,

Vipul Kumar Singh

 

Thanks Pranuj. I am really glad my article was able to interest you so much.

 

When we say that a decision makers need to ask himself-"Whether I want the correct answers – quickly if possible or do I want the answers quickly – correct if possible?" we are talking in relative terms. I agree with you when you say that a firm is all about making profits and that they will be looking for an accuracy of 95% in all their results. Having said that let me try to explain the aforementioned phrase using an example. Suppose there is a company ABC which is into manufacturing light bulbs. It has to launch its product in a new market, say the city of Delhi. However, at the same time, its competitor is working on a similar product and is working towards launching its product too. So, it is natural that ABC will want to launch its product first in order to make the most of the market. At a time like this, ABC cannot afford to spend too much time trying to perfect its data for decision making. Therefore, it reaches a compromise. It literally says that this time we go for about 94.5% accuracy; in exchange, I speed up my decision making so that I can launch my product first.

That is what is meant by the above phrase. The balance between time and accuracy always exists. In order to gain one, you will have to sacrifice the other. Equivalent exchange, so to speak. Even then, yes I will agree with you and say that there is a limit to how much accuracy can be sacrificed. Deciding that is up to the manager and varies from industry to industry.

 

I am really glad you asked me to highlight the positives of the D3M. I shall try my best to explain it as per my understanding.

1.)    Reduced effortàIncreased efficiency- As I have mentioned in my answer to Vipul’s question above, the D3M is a tool that goes through a sea of information on behalf of you and extracts the data that is relevant to your decision making. As a result, the efficiency of the decision making process as a whole goes up because the effort that you would give to sifting through data can now be diverted to analysing it.

2.)    More accountabilityàLesser grey areas- Every firm literally has entire servers’ worth of data about things like employee activity. It would be very easy for an employee to hide some discrepancies or deceitful actions in so much data without letting anyone know. However, now with the D3M and its associated data mining tools, any discrepancy in the data, no matter how small is easily identifiable. As a result, there are lesser grey areas in your firm since everyone is accountable for his/her actions.

3.)    Standardization & Customisation- With the D3M, the problem of decisions changing radically with the change in decision makers is reduced drastically. This is because with the D3M, as long as the model defining it remains the same, the output shall vary within some standards that have been defined. This ensures a consistent performance by the firm, irrespective of who comes in to make the decisions. This does not mean that the decisions will remain absolutely same. They will vary, but within a set limit.

Another feature of the D3M is that it is easily customisable. It does not have a specialization or area of expertise on its own. Whichever model is laid onto the system, the D3M becomes an expert in that. Therefore, it can be used to take decisions across all departments in a firm by changing its model every time.

 

Regarding best practices and implementation strategies, I think it is best explained by Mr. Avinash Kaushik in his article. He mentions seven common sense steps to best implement the D3M by establishing a culture for it. I choose to discuss the six most important ones. The article is from the point of view of a web analytics agency. All the below mentioned points are his own. I am just paraphrasing them:

1.)    Identify needs – The Company needs to identify what is it that the customer hopes to attain using the D3M, that way you can send them focused bits and pieces of data instead of providing them access to the whole database. Similarly, when you are implementing D3M in your firm, you need to identify what is it that you want out of it?

2.)    Differentiate between reporting and analysis – It is quite easy to generate several hundred reports chock full of data in the hopes that some of those reports will give you an insight. But, D3M is not about reporting, it’s about analysis. So, a culture has to be established to lean towards analysis and not reporting.

3.)    Depersonalize decision making – Avinash mentions that it is never about you, but your customer. Therefore, a need arises to depersonalize the decision making process. In order to do so, you should go for tools such as benchmarking, the outside in approach etc.

4.)    Be proactive, not reactive – You may not always have to make a decision, but that does not mean that during such times, the D3M is idle. Your firm should have a culture of ever on-going analysis. This way, when the time comes, the analysis done over a period of time can be fed into the D3M to get the best results.

5.)    Hire critical thinkers as analysts – When you wish to make the best out of the D3M, it is advisable to hire analysts who are critical thinkers. Also, they should be given a set of parameters in which they work, so that data analysis is promoted.

6.)    Establish processes – In order to make the best of D3M, it is of absolute importance that there are processes in place that are in sync with its goals. The reason for this is simple- “It is not so much about the tool itself, than it is about the accompanying processes”. Few of the processes that need to be in place are:

a.       Processes for data collection – These processes insure the cleanliness of data.

b.      Processes for data storage – These need to be done in a proper manner so that retrieval is not a hassle.

c.       Processes for data analysis

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made.

Regards,

Aseem 

 

Dear Aseem,

 

I appreciate your understanding of the model and the way you answered my query. The positives and best practices mentioned are satisfactory. Thank you!

 

But as I was going through the article, what strikes me is that the article is based more on conjecture. In today's world what we want is data to back the theory. So I would appreciate if you could present your D3M model with some statistical data.

 

Along with this I have one more view which I would like you to address. We "The Humans" are always good at decision making because we take many factors into consideration. Our mind has this beautiful habit of analyzing the given data at any point in time. Then why there is a sudden burst of interest in D3M model? What is so peculiar about this model that without which decision making will be like "feeding a lion"?

 

Thanks & Regards,

Pranuj Singhal

 

Hi Aseem,

 

You sound very convincing regarding your thoughts on D3M and why it is so helpful for the business to grow. As you said numbers help us a lot in taking decision from past experiences. If we have data we can analyse the data and take our decision. Well i would like you to convince me a little bit more.

 

In 2008 Lehman brothers saw its downfall due to subprime crisis, should I assume that they didn't had the data? or that they didn't had the correct data? or that they expected this to happen and they were ready for bankruptcy?

 

Well this was not the case to be, after being set up in 1850 Lehman brothers had seen many problems such as the railroad bankruptcies of the 1800s, the Great Depression of the 1930s, two world wars, a capital shortage when it was spun off by American Express in 1994, and the Long Term Capital Management collapse and Russian debt default of 1998. But they survived it all without falling. It means that they had the belief in them that they have avoided these situations, so they can do it again. Now continuing with the facts Lehman Brothers boomed in the year 2006 and its  capital markets surged 56% being the highest in any investment banking or asset management industry.They acquired 5 mortgage firms including the big ones like BNC mortgage. Lehman also reported record profits every year from 2005 to 2007. In 2007 they reported net income of a record $4.2 billion on revenue of $19.3 billion. In Feb 2007, the stock reached a record $86.16, giving Lehman a market cap of close to $60 billion. Lehman brother had this data with them to support their actions, analyzing this available data and previous achievements Chief Financial Officer of Lehman brother said "the risks posed by rising home delinquencies were well contained and would have little impact on the firm's earning". The impact was "so small" that the 4th largest Investment bank lost all its wealth and went into bankruptcy. In 2008 they had all the required data at their side and they couldn't handle it while at the earlier situation which comprised of the biggest crisis in 1930s they came out easily without proper data and data management services . Why the data didn't help them in 2008 when they lost a fight against crisis?

 

I would appreciate is you could clarify my doubts.

 

 

Thanks and Regards,

Vipul Kumar Singh

 

 

Thank you, Vipul. I am glad that I was able to make the case for D3M a little stronger.

In order to answer your question, let me elaborate a little on the reasons that I feel are the most responsible for the Lehman brothers’ crash. I refer the most to this article for my analysis of the same.

1.)    Ignored warnings – Lehman brothers saw a lucrative opportunity in providing the mortgages properties available as securities for sale to investors. This opportunity was a fickle one because it was born out of the manner in which the mortgage brokers were giving out housing loans to people. So much so, that many people were unable to pay back the mortgage. Consequently, the housing bubble burst and the real estate prices tanked. This led to the demand for mortgage securities vanishing.

All the banks had been warned about the impending doom of the real estate industry and advised to let up on the practice of giving out mortgaged properties as securities. However, Lehman brothers chose to ignore this warning and kept investing heavily in this field. This went on to the point that they had taken too much debt. Quoting the article – “The bank held less than a dollar in reserves for every $30 of its liabilities” which shows how negligent the bank had become.

2.)    Bailout refusal – The fed refused to bailout Lehman brothers in spite of the expectations. Everyone was expecting the fed to bailout Lehman like it did for AIG after the Lehman debacle. Add to the fact that most of the banks were in no position to help out Lehman for several reasons, it only made Lehman brothers’ grave deeper.

Now with regards to why in spite of the data being there, did Lehman brothers fall? Here are the reasons:

1.)    Confabulation – The D3M is designed to make a suggestion at the end of its processing. It is up to the user to ignore or accept it. While I cannot be too sure of what output did the D3M give for Lehman brothers, it depends on too many parameters like what data was it given, what was the model prescribed etc.; what I can say for sure is that ignoring the warning given by the D3M can often lead to disastrous results. As Lehman chose to ignore the warning about the housing bubble, one can only assume that it did the same with the warnings it got from the D3M. Again, nothing can be said conclusively until we know more details.

2.)    D3M is not a guarantee – D3M is designed to be an assisting tool. Most of the important roles are still left to the decision makers. Even then, the D3M does work on a certifiable margin of error that changes subject to the parameters under which it is working. Even with the D3M, both the decision maker and the system go through a learning curve which helps them take better decisions with time. Having said that, I will not be completely one sided, I do admit that even the D3M makes a mistake at times. It is not a perfect system either.

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made.

Regards,

Aseem

Dear Aseem,

 

I appreciate your understanding of the model and the way you answered my query. The positives and best practices mentioned are satisfactory. Thank you!

 

But as I was going through the article, what strikes me is that the article is based more on conjecture. In today's world what we want is data to back the theory. So I would appreciate if you could present your D3M model with some statistical data.

 

Along with this I have one more view which I would like you to address. We "The Humans" are always good at decision making because we take many factors into consideration. Our mind has this beautiful habit of analyzing the given data at any point in time. Then why there is a sudden burst of interest in D3M model? What is so peculiar about this model that without which decision making will be like "feeding a lion"?

 

Thanks & Regards,

Pranuj Singhal

Thank you for your kind words Pranuj. I am glad I was able to clear up your doubts.

Regarding data supporting D3M, here is an article  that discusses the results of a survey conducted by Teradata. The findings of the article are shown below in the chart. This chart describes what respondents feel about the benefits of using data in making decisions. The article discusses the output of a survey conducted among 2200 volunteers

Quoting verbatim from the article - “Separately in the study, Teradata asked a similar question about the benefits of using data in decision-making, providing a list of answers much broader in scope. According to those results, 58% believe the use of data can help them make more accurate decisions, 49% believe it helps them achieve better business results, 44% see it resulting in more efficient use of resources and reduced cost, and 43% agree that it helps identify new opportunities and new competitive advantages.”

Regarding the second part of your question, the sudden increase in interest when it comes to D3M is primarily due to the huge amounts of information available to companies. Add to that the increased levels of competition; it is imperative for every company to implement D3M so that it does not lag behind other companies.

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made.

Regards,

Aseem

Hi Aseem,

 

Referring to the line "The benefit of D3M is that it gives you the ability to quantify your decision and you have absolute control over the decision making process" from your article I assume you wanted to say that that data will give you absolute control over decision and that refers not to make any lapses. This didn't proved right in Lehman Brothers case and you agreed that it makes mistake. If you agree that this system also makes than I believe this line needs to be rephrased.

 

Carrying on to the other queries I had, I agree Data shows you a way to take decision but it does not mean that its the right way. Data can be manipulated but not instinct . Numbers can be changed but not abilities . What will you say when you see the average package of some college and then make a decision of joining them, can you be sure of making the right decision. What will you say for a teacher who has multiple articles written on his name, can you bet on his ability to be perfect and a master in his profession. How will you judge an author, by the number of volumes that comes out every year or by his/her writing style. Most companies choose the CEO based on their instincts, vision and not on the basis of their data judgment. If Mr. Raghuram Rajan had followed the data then he wouldn't have changed the monetary policies and wouldn't have followed his instincts. Wars are predicted and avoided without data. If data is to be blindly believed then our government is doing a good job which says that the people who are below poverty line has decreased but on the other hand we see the ever increasing cases of poverty and hunger. 

 

I believe I have made my case stronger and reduced the importance of data to 50 percent . So the "control" that was "absolute" in your article needs to be "loosened a bit".

 

Thanks and regards

Vipul Kumar Singh

Hi Aseem,

 

Referring to the line "The benefit of D3M is that it gives you the ability to quantify your decision and you have absolute control over the decision making process" from your article I assume you wanted to say that that data will give you absolute control over decision and that refers not to make any lapses. This didn't proved right in Lehman Brothers case and you agreed that it makes mistake. If you agree that this system also makes than I believe this line needs to be rephrased.

 

Carrying on to the other queries I had, I agree Data shows you a way to take decision but it does not mean that its the right way. Data can be manipulated but not instinct . Numbers can be changed but not abilities . What will you say when you see the average package of some college and then make a decision of joining them, can you be sure of making the right decision. What will you say for a teacher who has multiple articles written on his name, can you bet on his ability to be perfect and a master in his profession. How will you judge an author, by the number of volumes that comes out every year or by his/her writing style. Most companies choose the CEO based on their instincts, vision and not on the basis of their data judgment. If Mr. Raghuram Rajan had followed the data then he wouldn't have changed the monetary policies and wouldn't have followed his instincts. Wars are predicted and avoided without data. If data is to be blindly believed then our government is doing a good job which says that the people who are below poverty line has decreased but on the other hand we see the ever increasing cases of poverty and hunger. 

 

I believe I have made my case stronger and reduced the importance of data to 50 percent . So the "control" that was "absolute" in your article needs to be "loosened a bit".

 

Thanks and regards

Vipul Kumar Singh

 

Thanks for your question Vipul. I really appreciate the passion with which you are following the article. It shows that I did something right to invoke such an enthusiastic response from a reader.

I agree, I have indeed made the claim that “The benefit of D3M is that it gives you the ability to quantify your decision and you have absolute control over the decision making process”.

Having said that, allow me to explain my intention in stating the above. By the above statement, I mean that the decision maker has absolute control over the decision making process. By the process, I mean setting up the model, deciding what kind of data you need, what kind of output you are looking for etc. Now, the success of said process depends on whether you have exercised your control correctly or not. Here, I would like to re-iterate what I had mentioned about Lehman brothers. Again, I am speculating but in all probability, Lehman brothers did not use their control properly (ignored the data warnings, as mentioned) and had to suffer for it. However, I do concede to your point that there is no guarantee of success using the D3M. So, would it be better if I stated the above as that “The benefit of D3M is that it gives you the ability to quantify your decision and you have absolute control over the decision making process. However, the success of the process depends on how it is defined by the decision maker in the first place”. I had missed mentioning this point in the article. Thank you for pointing it out.

Coming to the second part of your question, which is indeed a rather unique perspective that you have given to the article. Suffice to say, I had to do some digging to come up with an answer for it. To start, I agree with you completely that numbers can be changed while abilities cannot be. Turns out, the world of data agrees with you too. As per this article in the New York Times; Google, one of the leading experts on data handling in the world has shifted the way that it looks for employees. Also, this article was written way back in 2007. Google too recognizes the importance of personality traits in making a hire. Now conventionally, data would be restricted to marks and past employment etc. just as you say. However, now Google has begun quantifying other aspects of the personality as well such as hobbies, interests, emotional quotient etc.

Now, using this, I would like to make my point that “Data is not restricted to numbers alone. Not anymore”. Therefore, there is much more to this process than meets the eye. Therefore, the decision making can be done for all the scenarios that you have pointed out using the D3M. One only needs to change the model for decision making. Another point from you that I agree with is that the D3M is not to be trusted blindly. The reason for that is “Garbage in, garbage out”. The output of the D3M may be corrupted due to a corrupt input.

Hope that answers your question. Please feel free to seek further clarification or point out any mistakes I may have made.

Regards,

Aseem