Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
"Why did you do that?": Explaining black box models with Inductive Synthesis
Uppsala University, Disciplinary Domain of Humanities and Social Sciences, Faculty of Social Sciences, Department of Informatics and Media.ORCID iD: 0000-0001-5213-8253
Uppsala University, Disciplinary Domain of Humanities and Social Sciences, Faculty of Social Sciences, Department of Informatics and Media.ORCID iD: 0000-0002-2323-6847
Uppsala University, Disciplinary Domain of Humanities and Social Sciences, Faculty of Social Sciences, Department of Informatics and Media, Information Systems.
Uppsala University, Disciplinary Domain of Humanities and Social Sciences, Faculty of Social Sciences, Department of Informatics and Media, Information Systems.
2019 (English)In: International Conference on Computational Science (ICCS), 2019Conference paper, Published paper (Refereed)
Abstract [en]

By their nature, the composition of black box models is opaque. This makes the ability to generate explanations for the response to stimuli challenging. The importance of explaining black box models has become increasingly important given the prevalence of AI and ML systems and the need to build legal and regulatory frameworks around them. Such explanations can also increase trust in these uncertain systems. In our paper we present RICE, a method for generating explanations of the behaviour of black box models by (1) probing a model to extract model output examples using sensitivity analysis; (2) applying CNPInduce, a method for inductive logic program synthesis, to generate logic programs based on critical input-output pairs; and (3) interpreting the target program as a human-readable explanation. We demonstrate the application of our method by generating explanations of an artificial neural network trained to follow simple traffic rules in a hypothetical self-driving car simulation. We conclude with a discussion on the scalability and usability of our approach and its potential applications to explanation-critical scenarios.

Place, publisher, year, edition, pages
2019.
Keywords [en]
artificial intelligence, machine learning, black box models, explanation, inductive logic, program synthesis
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:uu:diva-382400OAI: oai:DiVA.org:uu-382400DiVA, id: diva2:1306760
Conference
International Conference on Computational Science (ICCS), Faro, Algarve, Portugal, 12–14 June 2019
Available from: 2019-04-24 Created: 2019-04-24 Last updated: 2019-10-14Bibliographically approved

Open Access in DiVA

fulltext(312 kB)31 downloads
File information
File name FULLTEXT01.pdfFile size 312 kBChecksum SHA-512
4e901bb10c544d0b56b7acfd0edcd44cc7b40f3b59d58757c02f2eb0a9962caf8a71c169c2a53530ffb9191883ff6559223a5d35e65a979eebd6e7228d8c36ed
Type fulltextMimetype application/pdf

Other links

Pre-print on arXiv

Search in DiVA

By author/editor
Paçacı, GörkemJohnson, DavidMcKeever, SteveHamfelt, Andreas
By organisation
Department of Informatics and MediaInformation Systems
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 31 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 51 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf