Understand why a model made a prediction: feature importance, SHAP, LIME, and explainer dashboards.
Black-box models (e.g. complex trees, neural nets) give a prediction but not the reason. Explainable AI (XAI) answers: “Which inputs (features) drove this output?” That builds trust, helps debug, and meets regulatory or fairness needs. Common tools: feature importance, SHAP (Shapley values), LIME (local approximations), and explainer dashboards (e.g. Plotly Dash).
Tree-based models (Random Forest, XGBoost) expose feature_importances_: which features the model used most to split. For linear models, coefficients show how each feature pushes the prediction up or down.
from sklearn.ensemble import RandomForestClassifier import pandas as pd # Train a model (e.g. on diabetes.csv – download below) df = pd.read_csv("datasets/diabetes.csv") X = df.drop("Outcome", axis=1) y = df["Outcome"] model = RandomForestClassifier() model.fit(X, y) # Feature importance for name, imp in sorted(zip(X.columns, model.feature_importances_), key=lambda x: -x[1]): print(f"{name}: {imp:.3f}")
datasets/diabetes.csv.
SHAP assigns each feature a contribution (Shapley value) to the prediction for a single sample. LIME fits a simple model (e.g. linear) around that sample to approximate the black box locally. Both help answer “why did the model say 1 for this patient?” Install: pip install shap lime and use their APIs on your trained model and a single row.
Tools like ExplainerDashboard (Python) or Plotly Dash can wrap your model and SHAP/LIME to build an interactive dashboard: change inputs, see prediction and feature contributions. Great for presenting to stakeholders.
In one sentence: why would a doctor or regulator prefer a model that is explainable (e.g. feature importance, SHAP) over a pure black box?