Explainable AI

A human hand with tattoos reaching out to a robotic hand on a white background.

A very traditional problem solving method is the following: given a set of features or variables, can we understand the features to form a conclusion. This could be something like a treatment strategy wherein the strategy is built on a series of data and then ingesting the data helps make a conclusion. However, an equally complicated question is that given that there are many factors that affect the decision – is there one or few factors that play a key role and changing those factors will affect the decision. Which means that if there are many factors that would affect the decision, can you change/alter that one factor and that would make a change to that decision.

Using AI to ingest a variable factors and then come up with an answer is usually opaque. And therefore finding the key factor is the basis of xAI -explainable AI. It is also called feature attribution. The method to figure this out is SHAP (Shapley addition explanations). One good website for a book chapter is here : https://christophm.github.io/interpretable-ml-book/shap.html

Similar Posts

  • |

    Knowledge graphs

    Knowledge graph is a way of representing information where entities/nodes (people, places, products, concepts) are linked by relationships/edges (works, creates, has ). It is a semantic network that captures facts and context. There is also an ontology that defines different types of node/edges and defines what types exist and how they relate. These are used…

  • Detection of AI created content

    As AI has progressed generating code, writing, music or other intelligent language based skills through an LLM there is a parallel growth in detection of AI generated content.  Like everything AI, it is a probability game. Trying to estimate the combination of words/tokens and comparing with what is in the model. Using a standard model…

  • DeepSpot

    Kalin Klonchev – the winner of a competition for AI based data analysis from Broad in 2024 had also created a tool called DeepSpot. Worth looking at for spot analysis of H&E sections by converting a full H&E slide pictures to “spots” which are analyzed. Some good links: DeepSpot paper: https://www.medrxiv.org/content/10.1101/2025.02.09.25321567v1 DeepSpot GitHub repository: https://github.com/ratschlab/DeepSpot…

  • Observability

    Observability is important for AI and AI tools. It is the ability to monitor them for token usage, response quality and model drift. Typically, an AI system is monitored through logs, traces and metrics but an AI system on AI agent may need other metrics. Troubleshooting a complex AI system that produces its output probabilistically…