Leaderboard

Multicollinearity is a statistical phenomenon. It happens when several independent variables are highly correlated, However not perfectly correlated and In this situation we get regression results to be unreliable. In the above example, we could see how and to what extend does Consumer Price Index and Inflation Index can predict the rates. There is a considerable overlap between Consumer Price Index and Borrow Rate and Substantial overlap between Inflation Index and Borrow rate. Now, because there is a significant overlap between Consumer Price Index and Inflation Index themselves. It would be possible to predict with the unique non-overlapping contribution. Unique non-overlapping contribution of Consumer Price Index is Area c and Unique non-overlapping contribution of Inflation Index is Area b and Area a will be lost to standard error. Why Multicollinearity is considered as a problem? We would not be able to discriminate the individual effects of the independent variables on the dependent variable Further Correlated independent variables make it hard to make inference about individual regression coefficients and their effects on dependent variable. As a result, it is difficult to disprove Null Hypothesis, wherein actually the same should be rejected. Multicollinearity might not affect the accuracy of the model by a lot. But we might lose reliability in determining the effects of individual features of the model and that can be a problem when it comes to interpretability. How do we detect Multicollinearity? By using scatter plot or by using correlation matrix it would be possible to detect multicollinearity with regards to bivariate relationship between variables It can be detected based on Variance Inflation Factor or as popularly referred as VIF. VIF score of independent variable represents how well the variable is explained by other independent variables. When R2 value is close to 1, higher the value of VIF and higher the multicollinearity with the independent variable. VIF = 1 implies No correlation between independent variables and other variables VIF > 5, indicates high multicollinearity Diagnosis and Fix: Dropping one of the correlated features can bring down multicollinearity significantly Priority of dropping variable is based on the high VIF value Combining correlated variables into one and drop the others Points to remember before fixing: Removing multicollinearity will be a good option when more preference is given to individual features relatively to the group features that impact the focus variable Efficient corrective action to remove multicollinearity requires selectivity and selectivity in turn requires specifics about the nature of the problem.