+- The Lumin Archive (https://theluminarchive.co.uk)
+-- Forum: The Lumin Archive — Core Forums (https://theluminarchive.co.uk/forumdisplay.php?fid=3)
+--- Forum: Computer Science (https://theluminarchive.co.uk/forumdisplay.php?fid=8)
+---- Forum: Artificial Intelligence & Machine Learning (https://theluminarchive.co.uk/forumdisplay.php?fid=25)
+---- Thread: Explainable AI — Opening the Black Box of Machine Learning (/showthread.php?tid=344)
Explainable AI — Opening the Black Box of Machine Learning - Leejohnston - 11-17-2025
Thread 7 — Explainable AI (XAI): Opening the Black Box of Machine Learning
Understanding Why AI Makes Its Decisions
As AI becomes more powerful, transparency becomes essential.
Explainable AI (XAI) aims to reveal why models behave the way they do.
1. Why We Need XAI
Without explanation, AI can be:
• untrustworthy
• biased
• opaque
• difficult to debug
This is dangerous in:
• medicine
• law
• finance
• scientific research
2. Local vs Global Explanations
Global — how the entire model behaves.
Local — why a specific decision was made.
Example:
Why did the model reject this loan application?
3. Key XAI Techniques
• SHAP values
Shows how each feature contributed to the output.
• LIME
Perturbs input slightly to measure influence.
• Saliency maps
Visual highlights of what influenced an image prediction.
• Integrated gradients
Measures contributions along the path from baseline to input.
4. Interpreting Neural Networks
Tools analyse:
• neuron activations
• attention patterns
• network internal structure
• feature embeddings
Helps uncover how models “think.”
5. Challenges in XAI
• Complex models resist simple explanations
• Explanations can mislead
• Interpretability is subjective
• Some systems (like deep LLMs) are massively high-dimensional
6. The Future of XAI
Research focuses on:
• mechanistic interpretability
• transparent architectures
• self-explaining models
• safety-critical auditing
Final Thoughts
Explainable AI bridges the gap between raw model power and human understanding.
It ensures that AI remains safe, fair, and transparent — essential for the future of intelligent systems.