Digitala Vetenskapliga Arkivet

Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Hybrid quantum–classical multimodal fusion under weak cross-modal alignment for bird species recognition
Department of Biomedical and Mechatronics Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Kajang, 43000, Selangor, Malaysia.
Department of Electrical and Electronic Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Kajang, 43000, Selangor, Malaysia.
Department of Electronic Engineering, Faculty of Engineering and Green Technology, Universiti Tunku Abdul Rahman (UTAR) Kampar Campus, Jalan Universiti, Bandar Barat, Kampar, 31900, Perak Darul Ridzuan, Malaysia.
Department of Biomedical Engineering & Health Sciences, Faculty of Electrical Engineering, Universiti Teknologi Malaysia, UTM Johor Bahru, Johor Bahru, 81310, Johor, Malaysia.
Show others and affiliations
2027 (English)In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 333, article id 133774Article in journal (Refereed) Published
Abstract [en]

Multimodal learning systems typically assume strong alignment between data modalities, an assumption frequently violated in real-world scenarios such as ecological monitoring, where visual and acoustic observations are weakly or inconsistently paired. This paper proposes a hybrid quantum-classical fusion framework that integrates independently learned modality-specific representations through a Parameterized Quantum Circuit (PQC). The proposed Quantum Fusion Layer (QFL) is designed to capture high-order cross-modal interactions under weak alignment conditions. Experimental evaluation on a multimodal bird species classification task shows that the proposed framework outperforms classical fusion baselines, achieving Accuracy and Macro F1-score of 0.9875 on a held-out test set. Additional analyses comprising stratified cross-validation, ablation studies, robustness evaluation under controlled noise conditions, a parameter-matched classical MLP control, and cross-domain generalization on a geographically out-of-distribution benchmark (Bird-SEA10) of 238 multimodal pairs from Singapore and Thailand indicate stable performance across data partitions, resilience to moderate input degradation, and meaningful generalization under geographic domain shift (cross-domain Accuracy = 0.8445, Macro F1 = 0.8382, ΔF1 = −0.1493 relative to in-domain performance). The parameter-matched MLP control achieves Accuracy of 0.2125 and Macro F1 of 0.0897, suggesting that the observed performance advantage originates from the structural properties of the quantum circuit rather than from reduced parameter capacity. These findings indicate that quantum-enhanced feature transformations may provide an effective mechanism for multimodal fusion under weak alignment, with potential applicability to broader multimodal learning problems involving heterogeneous and imperfectly aligned data sources.

Place, publisher, year, edition, pages
Elsevier Ltd , 2027. Vol. 333, article id 133774
Keywords [en]
Quantum machine learning, Multimodal learning, Multimodal fusion, Weak cross-modal alignment, Ecological monitoring, Bird species recognition
National Category
Computer Sciences
Research subject
Machine Learning
Identifiers
URN: urn:nbn:se:ltu:diva-119406DOI: 10.1016/j.eswa.2026.133774ISI: 001843978000001Scopus ID: 2-s2.0-105046477847OAI: oai:DiVA.org:ltu-119406DiVA, id: diva2:2093020
Available from: 2026-08-18 Created: 2026-08-18 Last updated: 2026-08-18Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Mokayed, Hamam
By organisation
Embedded Internet Systems Lab
In the same journal
Expert systems with applications
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 4 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf