Graham Harrison is a hands-on data scientist with 20+ years of programming experience. He’s a Technology and Data Executive, an Institute of Directors Digital Ambassador, and Founder of the Data Science Consultancy. Graham’s qualifications include an Executive MBA in Leadership and Management from the University of Nottingham, and an Emeritus certificate in Applied Data Science from Columbia.
In this post, Graham explores the concept of causal inference, and the ways it can be used in machine learning. Causal inference, Graham argues, can have important applications for organisations in a number of ways:
Causal inference is the application of the combination of statistics, probability, machine learning and computer programming in understanding the answer to the question “why?”.
In my work as a data scientist I have developed and implemented
many machine learning algorithms that produced accurate predictions that have
added significant value to organisational outcomes.
For example, accurate predictions of staff churn allow proactive intervention to support and encourage likely churners to stay and that insight can increase staff productivity and decrease recruitment costs.
However, that may not be enough. Following one successful machine learning prediction project one of the business domain experts approached me and asked, “why are the staff members identified as churners leaving the organisation?” Dipping into my Data Science tool bag I used SHAP (SHapley Additive exPlanations) to show what features were contributing the greatest weights to the overall prediction and to individual cases.
This helped the customer to understand more about the way the algorithm worked and prompted their next question – “What do I need to change
to stop churn happening in the first place rather than just intervening for staff that might leave?” This prompted me to do some research which led to some revelatory conclusions.
To find out more about causal inference in machine learning, read the article in full over on our magazine: https://issuu.com/datasciencetalent/docs/the_data_scientist_mag_issue_3_digital/s/46505602
