An evidence-based neuro-symbolic framework for ambiguous image scene classification
2025 (English)In: Proceedings of Machine Learning Research / [ed] Gilpin L.H., Giunchiglia E., Hitzler P., van Krieken E., ML Research Press , 2025, Vol. 284Conference paper, Published paper (Refereed)
Abstract [en]
In this study, we propose a novel neuro-symbolic approach to deal with the inherent ambiguity in image scene classification, combining the usage of pre-trained deep learning (DL) models with concepts from modal logic and evidence theory. The DL models are used to detect objects and estimate their depth in a set of labeled images. The obtained outputs are employed to form a dataset of instances characterizing the possible classes. Subsequently, a multi-valued mapping is defined between the data instances and the considered images resulting into each image being represented by the set of instances associated with it. The obtained mapping is utilized to infer necessity and possibility conditions of each class, or equivalently its upper (plausibility) and lower (belief) probabilities. Based on these interval evaluations, a rule-based and a score-based classifiers are built. The overall method is explainable and directly interpretable, robust to data scarcity and data imbalance. The presented framework is studied and evaluated on an abandoned bag detection use case.
Place, publisher, year, edition, pages
ML Research Press , 2025. Vol. 284
Series
Proceedings of Machine Learning Research (PMLR), E-ISSN 2640-3498
Keywords [en]
Computer Vision, Evidence Theory, Image Scene Classification, Modal Logic, Multi-valued Mapping, Classification (of information), Computer circuits, Deep learning, Image classification, Mapping, Condition, Evidence theories, Evidence-based, Labeled images, Learning models, Logic theory, Multivalued mappings, Rule based
National Category
Computer graphics and computer vision
Identifiers
URN: urn:nbn:se:bth-28865Scopus ID: 2-s2.0-105020237323ISBN: 9781713845065 (print)OAI: oai:DiVA.org:bth-28865DiVA, id: diva2:2012151
Conference
19th Conference on Neurosymbolic Learning and Reasoning, NeSy 2025, Santa Cruz, Sept 8-10, 2025
Part of project
HINTS - Human-Centered Intelligent Realities
Funder
Knowledge Foundation, 202200682025-11-072025-11-072025-11-07Bibliographically approved