Digitala Vetenskapliga Arkivet

Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Deep Evidential Doctor
Halmstad University, School of Information Technology.ORCID iD: 0000-0001-5688-0156
2022 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Recent years have witnessed an unparalleled surge in deep neural networks (DNNs) research, surpassing traditional machine learning (ML) and statistical methods on benchmark datasets in computer vision, audio processing and natural language processing (NLP). Much of this success can be attributed to the availability of numerous open-source datasets, advanced computational resources and algorithms. These algorithms learn multiple levels of simple to complex abstractions (or representations) of data resulting in superior performances on downstream applications. This has led to an increasing interest in reaping the potential of DNNs in real-life safety-critical domains such as autonomous driving, security systems and healthcare. Each of them comes with their own set of complexities and requirements, thereby necessitating the development of new approaches to address domain-specific problems, even if building on common foundations.

In this thesis, we address data science related challenges involved in learning effective prediction models from structured electronic health records (EHRs). In particular, questions related to numerical representation of complex and heterogeneous clinical concepts, modelling the sequential structure of EHRs and quantifying prediction uncertainties are studied. From a clinical perspective, the question of predicting onset of adverse outcomes for individual patients is considered to enable early interventions, improve patient outcomes, curb unnecessary expenditures and expand clinical knowledge.

This is a compilation thesis including five articles. It begins by describing a healthcare information platform that encapsulates clinical, operational and financial data of patients across all public care delivery units in Halland, Sweden. Thus, the platform overcomes the technical and legislative data-related challenges inherent to the modern era's complex and fragmented healthcare sector. The thesis presents evidence that expert clinical features are powerful predictors of adverse patient outcomes. However, they are well complemented by clinical concept embeddings; gleaned via NLP inspired language models. In particular, a novel representation learning framework (KAFE: Knowledge And Frequency adapted Embeddings) has been proposed that leverages medical knowledge schema and adversarial principles to learn high quality embeddings of both frequent and rare clinical concepts. In the context of sequential EHR modelling, benchmark experiments on cost-sensitive recurrent nets have shown significant improvements compared to non-sequential networks. In particular, an attention based hierarchical recurrent net is proposed that represents individual patients as weighted sums of ordered visits, where visits are, in turn, represented as weighted sums of unordered clinical concepts. In the context of uncertainty quantification and building trust in models, the field of deep evidential learning has been extended. In particular for multi-label tasks, simple extensions to current neural network architecture are proposed, coupled with a novel loss criterion to infer prediction uncertainties without compromising on accuracy. Moreover, a qualitative assessment of the model behaviour has also been an important part of the research articles, to analyse the correlations learned by the model in relation to established clinical science.

Put together, we develop DEep Evidential Doctor (DEED). DEED is a generic predictive model that learns efficient representations of patients and clinical concepts from EHRs and quantifies its confidence in individual predictions. It is also equipped to infer unseen labels.

Overall, this thesis presents a few small steps towards solving the bigger goal of artificial intelligence (AI) in healthcare. The research has consistently shown impressive prediction performance for multiple adverse outcomes. However, we believe that there are numerous emerging challenges to be addressed in order to reap the full benefits of data and AI in healthcare. For future works, we aim to extend the DEED framework to incorporate wider data modalities such as clinical notes, signals and daily lifestyle information. We will also work to equip DEED with explainability features.

Place, publisher, year, edition, pages
Halmstad: Halmstad University Press, 2022. , p. 21
Series
Halmstad University Dissertations ; 88
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:hh:diva-46347ISBN: 978-91-88749-85-7 (print)ISBN: 978-91-88749-86-4 (electronic)OAI: oai:DiVA.org:hh-46347DiVA, id: diva2:1637806
Public defence
2022-03-15, J102 (Wigforss), Visionen, Kristian IV:s väg 3, Halmstad, 13:00 (English)
Opponent
Supervisors
Available from: 2022-02-15 Created: 2022-02-15 Last updated: 2022-05-12Bibliographically approved
List of papers
1. Data resource profile: Regional healthcare information platform in Halland, Sweden
Open this publication in new window or tab >>Data resource profile: Regional healthcare information platform in Halland, Sweden
Show others...
2020 (English)In: International Journal of Epidemiology, ISSN 0300-5771, E-ISSN 1464-3685, Vol. 49, no 3, p. 738-739fArticle in journal (Refereed) Published
Abstract [en]

Accurate and comprehensive healthcare data coupled with modern analytical tools can play a vital role in enabling care providers to make better-informed decisions, leading to effective and cost-efficient care delivery. This paper describes a novel strategic healthcare analysis and research platform that encapsulates 360-degree pseudo-anonymized data covering clinical, operational capacity and financial data on over 500,000 patients treated since 2009 across all care delivery units in the county of Halland, Sweden. The over-arching goal is to develop a comprehensive healthcare data infrastructure that captures complete care processes at individual, organizational and population levels. These longitudinal linked healthcare data are a valuable tool for research in a broad range of areas including health economy and process development using real world evidence.

Key messages

Structured and standardized variables have been linked from different regional healthcare sources into a research information platform including all healthcare visits in the county of Halland in Sweden, from 2009 to date.

Since 2015, the regional information platform integrates a cost component to each healthcare visit: thus being able to quantify patient level value, safety and cost efficiency across the continuum of care. © The Author(s) 2020; all rights reserved. Published by Oxford University Press on behalf of the International Epidemiological Association

Place, publisher, year, edition, pages
Oxford: Oxford University Press, 2020
Keywords
ambulances, delivery of health care, demography, emergency treatment, id, iduronate sulfatase, inpatients, outpatients, pharmacies, primary health care, guidelines, gender, health care systems, cost-effectiveness analysis, data security, semantic web
National Category
Health Care Service and Management, Health Policy and Services and Health Economy
Identifiers
urn:nbn:se:hh:diva-39308 (URN)10.1093/ije/dyz262 (DOI)000593364900005 ()31930310 (PubMedID)2-s2.0-85079289898 (Scopus ID)
Funder
Swedish Research Council, 2019–00198
Available from: 2019-05-02 Created: 2019-05-02 Last updated: 2022-07-06Bibliographically approved
2. Training machine learning models to predict 30-day mortality in patients discharged from the emergency department: a retrospective, population based registry study
Open this publication in new window or tab >>Training machine learning models to predict 30-day mortality in patients discharged from the emergency department: a retrospective, population based registry study
Show others...
2019 (English)In: BMJ Open, E-ISSN 2044-6055, Vol. 9, no 8, article id e028015Article in journal (Refereed) Published
Abstract [en]

Background: Aggressive treatment at end-of-life (EOL) can be traumatic to patients and may not add clinical benefit. Absent an accurate prognosis of death, individual level biases may prevent timely discussions about the scope of EOL care and patients are at risk of being subject to care against their desire. The aim of this work is to develop predictive algorithms for identifying patients at EOL, with clinically meaningful discriminatory power.

Methods: Retrospective, population-based study of patients utilizing emergency departments (EDs) in Sweden, Europe. Electronic health records (EHRs) were used to train supervised learning algorithms to predict all-cause mortality within 30 days following ED discharge. Algorithm performance was validated out of sample on EHRs from a separate hospital, to which the algorithms were previously unexposed.

Results: Of 65,776 visits in the development set, 136 (0.21%) experienced the outcome. The algorithm with highest discrimination attained ROC-AUC 0.945 (95% CI 0.933 - 0.956), with sensitivity 0.869 (95% CI 0.802, 0.931) and specificity 0.858 (0.855, 0.860) on the validation set.

Conclusions: Multiple algorithms displayed excellent discrimination and outperformed available indexes for short-term mortality prediction. The practical utility of the algorithms increases as the required data were captured electronically and did not require de novo data collection.

Trial registration number: Not applicable.

Place, publisher, year, edition, pages
London: BMJ Publishing Group Ltd, 2019
National Category
Social and Clinical Pharmacy
Identifiers
urn:nbn:se:hh:diva-39307 (URN)10.1136/bmjopen-2018-028015 (DOI)000502537200142 ()31401594 (PubMedID)2-s2.0-85070687111 (Scopus ID)
Note

Funding: This work was partly funded by Region Halland, Sweden.The initial stage of MCBs involvement in the work was funded by a grant for post-doctoral research from the Tegger Foundation.

Available from: 2019-05-02 Created: 2019-05-02 Last updated: 2023-08-28Bibliographically approved
3. Readmission prediction using deep learning on electronic health records
Open this publication in new window or tab >>Readmission prediction using deep learning on electronic health records
2019 (English)In: Journal of Biomedical Informatics, ISSN 1532-0464, E-ISSN 1532-0480, Vol. 97, article id 103256Article in journal (Refereed) Published
Abstract [en]

Unscheduled 30-day readmissions are a hallmark of Congestive Heart Failure (CHF) patients that pose significant health risks and escalate care cost. In order to reduce readmissions and curb the cost of care, it is important to initiate targeted intervention programs for patients at risk of readmission. This requires identifying high-risk patients at the time of discharge from hospital. Here, using real data from over 7,500 CHF patients hospitalized between 2012 and 2016 in Sweden, we built and tested a deep learning framework to predict 30-day unscheduled readmission. We present a cost-sensitive formulation of Long Short-Term Memory (LSTM) neural network using expert features and contextual embedding of clinical concepts. This study targets key elements of an Electronic Health Record (EHR) driven prediction model in a single framework: using both expert and machine derived features, incorporating sequential patterns and addressing the class imbalance problem. We show that the model with all key elements achieves a higher discrimination ability (AUC 0.77) compared to the rest. Additionally, we present a simple financial analysis to estimate annual savings if targeted interventions are offered to high risk patients. © 2019 The Authors

Place, publisher, year, edition, pages
Maryland Heights, MO: Academic Press, 2019
Keywords
Electronic health records, Readmission prediction, Long short-term memory networks, Contextual embeddings
National Category
Health Care Service and Management, Health Policy and Services and Health Economy
Identifiers
urn:nbn:se:hh:diva-39297 (URN)10.1016/j.jbi.2019.103256 (DOI)000525699100005 ()31351136 (PubMedID)2-s2.0-85069858722 (Scopus ID)
Projects
HiCube - behovsmotiverad hälsoinnovation
Funder
European Regional Development Fund (ERDF)
Note

Funding: The authors thank the European Regional Development Fund (ERDF), Health Technology Center and CAISR at Halmstad University and Hallands Hospital for financing the research work under the project HiCube - behovsmotiverad hälsoinnovation.

Available from: 2019-04-30 Created: 2019-04-30 Last updated: 2022-02-15Bibliographically approved
4. KAFE: Knowledge and Frequency Adapted Embeddings
Open this publication in new window or tab >>KAFE: Knowledge and Frequency Adapted Embeddings
2022 (English)In: Machine Learning, Optimization, and Data Science: 7th International Conference, LOD 2021, Grasmere, UK, October 4–8, 2021, Revised Selected Papers, Part II / [ed] Giuseppe Nicosia; Varun Ojha; Emanuele La Malfa; Gabriele La Malfa; Giorgio Jansen; Panos M. Pardalos; Giovanni Giuffrida; Renato Umeton, Cham: Springer, 2022, Vol. 13164, p. 132-146Conference paper, Published paper (Refereed) [Artistic work]
Abstract [en]

Word embeddings are widely used in several Natural Language Processing (NLP) applications. The training process typically involves iterative gradient updates of each word vector. This makes word frequency a major factor in the quality of embedding, and in general the embedding of words with few training occurrences end up being of poor quality. This is problematic since rare and frequent words, albeit semantically similar, might end up far from each other in the embedding space.

In this study, we develop KAFE (Knowledge And Frequency adapted Embeddings) which combines adversarial principles and knowledge graph to efficiently represent both frequent and rare words. The goal of adversarial training in KAFE is to minimize the spatial distinguishability (separability) of frequent and rare words in the embedding space. The knowledge graph encourages the embedding to follow the structure of the domain-specific hierarchy, providing an informative prior that is particularly important for words with low amount of training data. We demonstrate the performance of KAFE in representing clinical diagnoses using real-world Electronic Health Records (EHR) data coupled with a knowledge graph. EHRs are notorious for including ever-increasing numbers of rare concepts that are important to consider when defining the state of the patient for various downstream applications. Our experiments demonstrate better intelligibility through visualisation, as well as higher prediction and stability scores of KAFE over state-of-the-art. © Springer Nature Switzerland AG 2022

Place, publisher, year, edition, pages
Cham: Springer, 2022
Series
Forskning i Halmstad, ISSN 1400-5409
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 13164
Keywords
Word embeddings, Knowledge graphs, Adversarial learning
National Category
Natural Language Processing
Identifiers
urn:nbn:se:hh:diva-46333 (URN)10.1007/978-3-030-95470-3_10 (DOI)000772650800010 ()2-s2.0-85125483733 (Scopus ID)978-3-030-95469-7 (ISBN)978-3-030-95470-3 (ISBN)
Conference
The 7th International Online & Onsite Conference on Machine Learning, Optimization, and Data Science, Grasmere, Lake District, England, United Kingdom, October 4 – 8, 2021
Available from: 2022-02-11 Created: 2022-02-11 Last updated: 2025-02-07Bibliographically approved
5. DEED: DEep Evidential Doctor
Open this publication in new window or tab >>DEED: DEep Evidential Doctor
2023 (English)In: Artificial Intelligence, ISSN 0004-3702, E-ISSN 1872-7921, Vol. 325, article id 104019Article in journal (Refereed) Published
Abstract [en]

As Deep Neural Networks (DNN) make their way into safety-critical decision processes, it becomes imperative to have robust and reliable uncertainty estimates for their predictions for both in-distribution and out-of-distribution (OOD) examples. This is particularly important in real-life high-risk settings such as healthcare, where OOD examples (e.g., patients with previously unseen or rare labels, i.e., diagnoses) are frequent, and an incorrect clinical decision might put human life in danger, in addition to having severe ethical and financial costs. While evidential uncertainty estimates for deep learning have been studied for multi-class problems, research in multi-label settings remains untapped. In this paper, we propose a DEep Evidential Doctor (DEED), which is a novel deterministic approach to estimate multi-label targets along with uncertainty. We achieve this by placing evidential priors over the original likelihood functions and directly estimating the parameters of the evidential distribution using a novel loss function. Additionally, we build a redundancy layer (particularly for high uncertainty and OOD examples) to minimize the risk associated with erroneous decisions based on dubious predictions. We achieve this by learning the mapping between the evidential space and a continuous semantic label embedding space via a recurrent decoder. Thereby inferring, even in the case of OOD examples, reasonably close predictions to avoid catastrophic consequences. We demonstrate the effectiveness of DEED on a digit classification task based on a modified multi-label MNIST dataset, and further evaluate it on a diagnosis prediction task from a real-life electronic health record dataset. We highlight that in terms of prediction scores, our approach is on par with the existing state-of-the-art having a clear advantage of generating reliable, memory and time-efficient uncertainty estimates with minimal changes to any multi-label DNN classifier. © 2023 The Author(s)

Place, publisher, year, edition, pages
Amsterdam: Elsevier, 2023
Keywords
Deep neural networks, Electronic health records, Multi-label classification, Risk minimization, Uncertainty quantification
National Category
Computer Sciences
Research subject
Health Innovation, IDC
Identifiers
urn:nbn:se:hh:diva-46348 (URN)10.1016/j.artint.2023.104019 (DOI)001093432000001 ()2-s2.0-85174183630 (Scopus ID)
Funder
VinnovaSwedish Research Council, 2019-00198
Note

Som manuskript i avhandling / As manuscript in thesis

Available from: 2022-02-15 Created: 2022-02-15 Last updated: 2023-11-21Bibliographically approved

Open Access in DiVA

fulltext(1405 kB)854 downloads
File information
File name FULLTEXT02.pdfFile size 1405 kBChecksum SHA-512
966d92ef644af5ba43ca767748b089ea77c37b5644cc8a8805f79a7de7f514e71ebf4477b3d7df78ea9952a6703b57f66e5df1702abb102cee2b2bef787f9346
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Ashfaq, Awais
By organisation
School of Information Technology
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 854 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 2232 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf