What is Explainable AI?

By 2022, 75 percent of new end user solutions leveraging AI and ML techniques will be built with commercial instead of open-source platforms. AI and ML solutions will be applied in every business irrespective of their product. So, it’s important to know what the ML algorithm is doing behind the scenes and it’s the duty of the ML engineer to explain the model to the person who is using it. For example, if a model suggests the user the best route then user should be aware of the features and constraints the model is considering to show the best route, so that user has an overview of the whole process and this what explainable AI is.

Explainable AI (XAI) is artificial intelligence which is programmed to describe its rationale, purpose and decision-making process in a way that can be easily understood by the normal person. XAI is often discussed in relation to deep learning and plays an important role in the FAT ML model (fairness, accountability and transparency in machine learning).

An important goal of explainable AI is to provide algorithmic accountability. Until recently, Artificial Intelligent systems have essentially been black boxes. Even if the inputs and outputs are known, the algorithms used to arrive at a decision are not easily understood or often proprietary, despite when the inner-workings of the programming is open-source and made freely available. As artificial intelligence becomes increasingly prevalent, it is becoming more important than ever to disclose how bias and the question of trust are being addressed.

So with all the information we have , let us try to relate the concepts with the above figure. Every machine learning model begins with cleaning the data and then segregate the data into training and test data sets and then different algorithms like clustering , regression are applied on the training data .Machine learning model then predicts the output for the given input and this model is given to the clients to use it in their business. Here is where problems might arise like clients are not aware of when the model will succeed or when will it fail. Why is it taking this feature into consideration ? How is it related to the output? .

Here is where XAI comes into picture , here client will be interacting with engineers who had developed the model and get to know all the details of the model. So the client has clear understanding about why the particular error is occuring , why is that constraint is taken into consideration. So if anything goes wrong the user can easily check the error and correct it. With both the domain knowledge and machine learning model knowledge user will be more intuitive in taking business decisions.

As advances in AI make these models so sophisticated that their logic becomes increasingly subtle and incomprehensible to humans, the need for auditable, accountable, and understandable AI becomes inevitable and might be hurried along by regulators’ (and consumers’) justified concerns. If your organization is using or looking to use AI—and by now, this should be a universal driver—you’re going to have to make sure you understand how your algorithms are working. If you don’t, you’ll leave your organization open to legal action, regulatory fines, loss of customer trust, security risks through lack of effective oversight, reputational damage, and, most fundamentally, losing full control of how your business operates.

The success of AI models is because of the machines own internal representations which are even more difficult than manually generated features leading to its inexplicable nature. There are a lot of on-going research on the ante-hoc and post-hoc methods to make the AI more answerable and awareness to inculcate these methods in existing programs. Good initiatives by leading organizations like DARPA, Google, DeepMind, etc. are leading the necessary change. Despite this, there will always be a tradeoff between explainability and accuracy whose balance depends on various factors like end-user, legal liability, technicality of the concerned parties and the field of application. Artificial intelligence should not become a powerful deity which we simply follow blindly without understanding its reasoning but we shouldn’t forget about its beneficial insight it can have. Ideally, we should build flexible and interpretable models that can work in collaboration with experts and their domain knowledge.

Related terms that are important to understand

In order to grasp these concepts better we must understand the terms given here:

Machine Learning: It is the part of Artificial Intelligence in which the system learns and improves by experience without implicitly programming it to do so.

Data cleaning: Data cleaning is the process of detecting and modifying corrupt or inaccurate records from a record table or set or database and refers to identifying incorrect,incomplete, inaccurate or irrelevant elements of the information and then substituting, modifying, or deleting the dirty or coarse information.