Nice insight into the intricacies of data driven decision making.  Kudos!!!

Gaurav Agarwal : Really Appreciate ur comments ...

a very good article- nice flow of thoughts, relevant examples and very specific and focused pattern. Also very smooth to read.

Anirban Sharma : Thank You so much for the appreciation ... Do feel free to ask any queries that you may have ...

Hi Partha,

 

I have a question. Don't you feel that putting the things in the perspective of banking sector (a quantitative based sector) steals the show away from the importance of data based decision making in every sector and every possible role of a manager? 

I understand that Know your Customer makes the banking sector as much data driven as does the referral system of e-Commerce or market research tools in a Marketing firm or the SEO methods used by a digital marketing consultant. What is your take on it? 

Hari Shankar Shasvat : Hii... First of of all I would like to thank you for showing interest in my article. To answer your question, Yes my article is centred around banking sector or it could have been any sector for that matter. The purpose of the article was to influence a Non-Data driven decision maker with substantial years of work experience in the sector (10 years to be precise). Now I thought that to influence a person with that amount of domain knowledge something generic could not be written. Inorder to influence him to adopt the data driven approach one needs to give him instances from his relevant area of knowledge & expertise, then only he might follow your direction else there is high probability of his rejecting your idea.

Hence I assumed that the manager would be from banking industry and written the article accordingly. This was a mere assumption and it could well have been Marketing firm or IT Consulting in which case the article would have been centered on those sectors. Hope this clarifies your query...

Mr Mitra,

 

First of all let me tell you the article you have written is very informative and interesting.

But, I do have a question. What I believe is when we talk about big data, and collection of data from social media web site etc. But, don't you think this can lead to erroneous data collection? The sample size would be really small and that too will vary from survey to survey or among social media websites. Moreover, it is being said that Customer do not know many of the times what they want. So, again collecting data based on it may lead to a decision which can be said as data driven, but won't that lead to a decision which is right partially?

Regards,

Angshuman Sarkar

Anghsuman Sarkar: Hii ... Thank you for your interest in my article and really appreciate your comments. You have asked 2 questions which I will take separately.

 

1. Issue of small Sample size : Please note that the primary reasons for smaller sample size are : 1. Data Aquisition bottleneck caused due to huge amount of data and limited capability of data analysis API(s) which forces analysts to use a small amount of data & 2. Data Complexity and the corresponding cost of analysis which is also needed to be kept in mind. Hence due to these 2 limitations we select a smaller sample size. But as further technological advancements happen, the capability to handle larget datasets would also increase, allowing us to use larger samples and make more acccurate predictions. As of now we try to make as accurate predictions as possible with our limited sample size.

 

Hence the responsibility lies with the analysts who needs to judiciously choose samples from a diverse set of users representative of the topology of a certain social graph (groups of related users) independent of other factors. The process is known as Graph sampling which has limitations of not considering the similarity of shared content and other contextual information like frequency of status changes, location etc... which are useful in the study of social diffusion (process of transfer of ideas & influences from more to less advanced people when they come in contact, which can be tapped for innovations). Hence to address this we need to do attribute based sampling with Activity (Status Updates), location & randomness as attributes and carry out analysis individually and ultimately merging them to come to a concrete conclusion.

Thus utilizing topology and user contextual attributes makes the analysis more accurate but this is again helping us do predictive analysis which cannot be 100% accurate. However with technological advancements and more data analysis our prediction only gets better and better.

 

2. Unawareness of Customers of what they really need and corresponding erroneous data collection: Predictive analysis of Customer data is done to guess what he/she might need in the future but he/she is not currently aware of. This is precisely the role of a marketer to anticipate the future needs and convert them to Wants whose accuracy is improved by data analysis. However before actually launching the product, customer needs to be educated and promotions done, after which customer reactions needs to be studied and then only the product launch is done. This improves the probability of success. However even then instances of failure are there which cannot be avoided and we can only improve our success rate through data analysis and related marketing efforts.

Dear Mr. Mitra,

 

Now you have cleared my doubt completely. Thanks a lot for enlightening me with the information. You have absolutely bridged the gap between my knowledge in Data Driven Decision making and the real life scenarios! Thanks once again. Hope you make a great future and career ahead.

Regards,

Angshuman Sarkar

HI Partha,

 

A very well thought and insightful article I must say... Well so far as I have witnessed in professional life most people complain about data privacy as the major issue which they are afraid of in their digital presence. Don't you think that tapping such data for marketing banking products will aggravate their fear ??

Thanks Sebanti for appreciating my work. To answer your question, Yes decision making systems does required access to huge personal data which might lead to concerns about privacy, ownership & regulation. You would agree that people have this tendency to be less apprehensive about data fetching if it is aimed at improving services to them. In this scenario they would be glad if the banking or any other sector does access their data to improve services delivered to them. Such data collection of course would involve their consent and hence minimize conflicts.

The problem however arises when we access the same data for marketing new products which they feel as inappropriate usage, for company’s own interests. Exhaustive research needs to be done to strike a balance between the 2 scenarios discussed above so that people see the data usage as predominantly beneficial with some instances of promotion as well. Besides, the customers should also be convinced that there can be situations where the promotions address their own concerns. In this way slowly and gradually we can develop a mutual trust situation where data access conflicts can be minimized. It must be noted that D3M is the future and we eventually have to accept it as a way of life. The only question is When?     

Dear Mr. Mitra,

 

Thank you for what is indeed a very insightful and interesting read. However, I have a small query . Implementing Data driven decision making would transfer the decision making authority from human to software. What is your take on it ?

 

regards,

Govind Agarwal

Implementing Data driven decision making will reduce the importance or status of managers in decision making. What is you opinion on this ?

Hi Govind,

 

Thank you so much for showing interest in my article. Well as far as your query is concerned let me answer it like this, we are talking about Data Driven Decision Making. Though the name implies many things it does hide a few, to be comprehended by the reader - A way to keep him/her interested and curious, as they say "Less is More".

 

To come to the point let me clarify that whenever we talk about Data Driven Decision making please be advised that we mean Decision Support Systems(DSS). That is the system which is going to support or facilitate the decision making process to be done by a manager or any professional as such by providing him/her a predictive anaysis of the various situation outcomes. Hence the ultimate authority lies with the human itself and not computer. However there are situations where software does have some decision making authority as in the case of artificial intelligence say in some space systems, which will be referred to as Decision automation systems (DAS). DSS is in the current scope on discussion and not DAS which can be adressed separately in a separate article or thread.

Hi Gaurav,

 

Thank you for your much asked and very relevant question. Most managers do feel to be downgraded because of D3M. Well this is selective perception I must say. Like a collection or compilation of recipes cannot replace the importance or taste that a Mother's hand can impart, similarly a D3M cannot substitute the human element that is crucial in business decision making. The D3M is only a predictive tool which lets you know various outcomes of a decision situation through rigorous data analysis and industry best practices which comes in handy so that the decision maker does not miss them during decision making. They can then use their expertise, domain knowledge to customize the decisions as per the situation which would involve some amount of non linear thinking at times and not algorithm driven thinking as most situations would differ from each other.

 

Data only assists in decision making.How to use data effectively & efficiently to arrive at a decision is where the role of manager gets demonstrated. Hope this answers your query...

Partha

Nice Read!!!

One query: People may use D3M to deflect responsibility of any wrong decision on the software. The managers may ultimately blame the system and get away with it.What do you think about this ?

insightful....

Hi Prabhat,

 

Thanks for showing interest in the article. As far your query is concerned my response will be in affirmative. People may try to avoid the responsibility of any wrong decision and hence the importance of appropriate training before introducing D3M (as you call it) is critical. Managers must learn that Decision support systems are provided to them to make them more effective and efficient at work and reduce mundane tasks of data analysis through software, but at the end of the day the decision authority and responsibility lies with him because of the non linear nature of situations where decision is to be made. Hence there is a need for continuous reminding of the manager regarding the effectiveness and limitations of decision support systems which is like a weapon in his baggage to deal with situations but the warrior has to be him.

Thanks Rahul for your appreciation ...

Hi Partha,

 

Well answered... could be an effective approach to tackle the customer reactions to data accessibility. Kudos .....

Thanks Partha. Your answer indicates you have a profound knowledge of Data Driven Decision Making . Hope to see many more such articles from you in future .

@ Partha  well gr8 to hear this!