Research on explainability of artificial intelligence algorithms advances

December 2020.- For Leopoldo Bertossi, Full Professor at the Faculty of Engineering and Sciences of Universidad Adolfo Ibáñez, we are living fast and fruitful times in informatics and computer science. “It is a very interesting time to work in artificial intelligence, machine learning and data science: there is much to be done and the impact of the advances achieved can be seen quickly”.

Bertossi, senior researcher at the Millennium Institute for Foundational Research on Data (IMFD), notes that “one of the most important challenges in artificial intelligence today is being able to scientifically explain the decisions made by algorithms, and enabling these algorithms to provide these explanations. This line of research is known as explainable artificial intelligence or explainable AI”.

Imagine, for example, that you need to apply for a loan at a bank: “Before, the analysis was done by an executive with paper and pencil; today it is most common for the institution to have an artificial intelligence (AI) system that uses our information to make a decision”. This system is an algorithm that has been fed and trained over time with data from many loans, learning to say yes or no to applicants based on the experience it has acquired.

What happens if the loan is not granted? The applicant would want to know why their loan was not approved, or what they could do to improve their credit profile. These are the explanations, at an individual level, that today are not easy to obtain, since algorithms can be, internally, very complex mathematical functions that are difficult to interpret (the topic of interpretability of algorithms and AI solutions goes hand in hand with that of explainability). It becomes complex to identify exactly which elements an algorithm took into account. Various solutions to this problem have recently been proposed, but there is still much to be done.

“In addition, those of us who work in this field are trying to develop methods to know whether the AI system, which is delivering results in an automated way, is doing so fairly or whether it is falling into biases,” the expert explains. That is why, today, explainability in AI goes hand in hand with concepts such as fairness and transparency, that is, fairness and transparency of artificial intelligence algorithms. Many countries are developing legislation on this, and Chile will not be excluded from that debate.

Now, obtaining answers is no trivial challenge: explainability in AI is one of the most complex and challenging lines of research today, being one of IMFD’s areas of focus, and also the subject of a recent scientific publication by Bertossi, together with Marcelo Arenas, Director of IMFD and Full Professor at the School of Engineering of Universidad Católica; Pablo Barceló, Deputy Director of IMFD and Director of the Institute for Mathematical and Computational Engineering IMC-UC, and Mikaël Monet, IMFD postdoctoral researcher.

In the scientific paper The Tractability of SHAP-Score-Based Explanations for Classification over Deterministic and Decomposable Boolean Circuits, which appears in the proceedings of the AAAI 2021 conference, the leading general international conference in AI, the researchers analyzed different methods that could help make algorithms’ decision-making processes more transparent.

There is a model, called SHAP-value, used to determine the value assigned to different variables within a classification algorithm. If we apply this to the bank loan example, the SHAP-value would reveal the weight an algorithm gives to factors such as financial history, a particular person’s employment situation, and whether it is considering elements such as skin color. “If skin color had the highest ranking on a relevance scale for the classification, that should make us very nervous about what is happening inside the algorithm, since there would be a clear, undesired bias,” the researcher points out.

In the publication, the researchers managed to define, in precise mathematical terms, in which classification models the SHAP-value can be obtained quickly, also creating a categorization of selection algorithms according to how feasible it is to obtain the SHAP-value, and identifying those algorithms for which obtaining that value is computationally complex.

And this is what explainability aims to do: to reveal an algorithm’s decision-making process, which could bring to light biases that do not align with the concepts of fairness and transparency.

For Bertossi, this is a first step: “Considering that there are already many algorithms trained today to make decisions about our lives, the impact of obtaining explanations in artificial intelligence is very high. While our work can be applied to a broad class of classification models, the work does not end there: there is still much to research in this area, in order to advance and contribute to making these systems, effectively, more transparent and fair”.