Digitala Vetenskapliga Arkivet

CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Deep Learning for Digital Pathology in Limited Data Scenarios
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0003-1066-3070
2022 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The impressive technical advances seen for machine learning algorithms in combination with the digitalization of medical images in the radiology and pathology departments show great promise in introducing powerful image analysis tools for image diagnostics. In particular, deep learning, a subfield within machine learning, has shown great success, advancing fields such as image classification and detection. However, these types of algorithms are only used to a very small extent in clinical practice. 

One reason is that the unique nature of radiology and pathology images and the clinical setting in which they are acquired poses challenges not seen in other image domains. Differences relate to capturing methods, as well as the image contents. In addition, these datasets are not only unique on a per-image basis but as a collective dataset. Characteristics such as size, class balance, and availability of annotated labels make creating robust and generalizable deep learning methods a challenge. 

This thesis investigates how deep learning models can be trained for applications in this domain, with particular focus on histopathology data. We investigate how domain shift between different scanners causes performance drop, and present ways of mitigating this. We also present a method to detect when domain shift occurs between different datasets. Another hurdle is the shortage of labeled data for medical applications, and this thesis looks at two different approaches to solving this problem. The first approach investigates how labeled data from one organ and cancer type can boost cancer classification in another organ where labeled data is scarce. The second approach looks at a specific type of unsupervised learning method, self-supervised learning, where the model is trained on unlabeled data. For both of these approaches, we present strategies to handle low-data regimes that may greatly increase the availability to build deep learning models for a wider range of applications. 

Furthermore, deep learning technology enables us to go beyond traditional medical domains, and combine the data from both radiology and pathology. This thesis presents a method for improved cancer characterization on contrast-enhanced CT by incorporating corresponding pathology data during training. The method shows the potential of im-proving future healthcare by intergraded diagnostics made possible by machine-learning technology. 

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2022. , p. 60
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 2253
Keywords [en]
Medical imaging, Digital pathology, Radiology, Machine learning, Deep learning.
National Category
Cancer and Oncology
Identifiers
URN: urn:nbn:se:liu:diva-189009DOI: 10.3384/9789179294748ISBN: 9789179294731 (print)ISBN: 9789179294748 (electronic)OAI: oai:DiVA.org:liu-189009DiVA, id: diva2:1701722
Public defence
2022-11-14, Kåkenhus, K3, Campus Norrköping, Norrköping, 09:15 (English)
Opponent
Supervisors
Available from: 2022-10-07 Created: 2022-10-07 Last updated: 2023-04-03Bibliographically approved
List of papers
1. A Closer Look at Domain Shift for Deep Learning in Histopathology
Open this publication in new window or tab >>A Closer Look at Domain Shift for Deep Learning in Histopathology
2019 (English)Conference paper, Poster (with or without abstract) (Refereed)
Abstract [en]

Domain shift is a significant problem in histopathology. There can be large differences in data characteristics of whole-slide images between medical centers and scanners, making generalization of deep learning to unseen data difficult. To gain a better understanding of the problem, we present a study on convolutional neural networks trained for tumor classification of H&E stained whole-slide images. We analyze how augmentation and normalization strategies affect performance and learned representations, and what features a trained model respond to. Most centrally, we present a novel measure for evaluating the distance between domains in the context of the learned representation of a particular model. This measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. The results show how learning is heavily influenced by the preparation of training data, and that the latent representation used to do classification is sensitive to changes in data distribution, especially when training without augmentation or normalization.

Series
arXiv.org
Keywords
deep learning, histopathology, domain shift
National Category
Medical Image Processing
Identifiers
urn:nbn:se:liu:diva-169071 (URN)
Conference
COMPAY19: 2nd MICCAI workshop on Computational Pathology, Shenzhen, China, October 13 2019
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Available from: 2020-09-08 Created: 2020-09-08 Last updated: 2023-04-03Bibliographically approved
2. A Study of Deep Learning Colon Cancer Detection in Limited Data Access Scenarios
Open this publication in new window or tab >>A Study of Deep Learning Colon Cancer Detection in Limited Data Access Scenarios
Show others...
2020 (English)Conference paper, Poster (with or without abstract) (Refereed)
National Category
Medical Image Processing
Identifiers
urn:nbn:se:liu:diva-169838 (URN)
Conference
International Conference on Learning Representations (ICLR) Workshop on AI for Overcoming Global Disparities in Cancer Care (AI4CC)
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Available from: 2020-09-20 Created: 2020-09-20 Last updated: 2023-04-03
3. Evaluation of Contrastive Predictive Coding for Histopathology Applications
Open this publication in new window or tab >>Evaluation of Contrastive Predictive Coding for Histopathology Applications
2020 (English)Conference paper, Published paper (Refereed)
National Category
Computer Sciences
Identifiers
urn:nbn:se:liu:diva-171370 (URN)
Conference
NeurIPS Workshop on Machine Learning for Health (ML4H)
Available from: 2020-11-13 Created: 2020-11-13 Last updated: 2023-04-03
4. Combatting out-of-distribution errors using model-agnostic meta-learning for digital pathology
Open this publication in new window or tab >>Combatting out-of-distribution errors using model-agnostic meta-learning for digital pathology
2021 (English)In: Proceedings of SPIE Medical Imaging: Digital Pathology / [ed] John E. Tomaszewski; Aaron D. Ward, SPIE - International Society for Optical Engineering, 2021, Vol. 11603, article id 116030SConference paper, Published paper (Refereed)
Abstract [en]

Clinical deployment of systems based on deep neural networks is hampered by sensitivity to domain shift, caused by e.g. new scanners or rare events, factors usually overcome by human supervision. We suggest a correct-then-predict approach, where the user labels a few samples of the new data for each slide, which is used to update the network. This few-shot meta-learning method is based on Model-Agnostic Meta-Learning (MAML), with the goal of training to adapt quickly to new tasks. Here we adapt and apply the method to the histopathological setting by identifying a task as a whole-slide image with its corresponding classification problem. We evaluated the method on three datasets, while purposefully leaving out-of-distribution data out from the training data, such as whole-slide images from other centers, scanners or with different tumor classes. Our results show that MAML outperforms conventionally trained baseline networks on all our datasets in average accuracy per slide. Furthermore, MAML is useful as a robustness mechanism to out-of-distribution data. The model becomes less sensitive to differences between whole-slide images and is viable for clinical implementation when used with the correct-then-predict workflow. This offers a reduced need for data annotation when training networks, and a reduced risk of performance loss when domain shift data occurs after deployment.

Place, publisher, year, edition, pages
SPIE - International Society for Optical Engineering, 2021
Series
Progress in biomedical optics and imaging, ISSN 1605-7422, E-ISSN 2410-9045
Keywords
digital pathology, deep learning, meta learning, domain shift, MAML
National Category
Computer Sciences Medical Image Processing Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-174913 (URN)10.1117/12.2579796 (DOI)000671008800023 ()2-s2.0-85103271794 (Scopus ID)9781510640351 (ISBN)9781510640368 (ISBN)
Conference
Medical Imaging 2021: Digital Pathology, 15 February 2021 - 19 February 2021,online only, United States
Note

Copyright 2021 Society of Photo-Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited. 

Available from: 2021-04-08 Created: 2021-04-08 Last updated: 2022-10-07
5. Measuring Domain Shift for Deep Learning in Histopathology
Open this publication in new window or tab >>Measuring Domain Shift for Deep Learning in Histopathology
2021 (English)In: IEEE journal of biomedical and health informatics, ISSN 2168-2194, E-ISSN 2168-2208, Vol. 25, no 2, p. 325-336Article in journal (Refereed) Published
Abstract [en]

The high capacity of neural networks allows fitting models to data with high precision, but makes generalization to unseen data a challenge. If a domain shift exists, i.e. differences in image statistics between training and test data, care needs to be taken to ensure reliable deployment in real-world scenarios. In digital pathology, domain shift can be manifested in differences between whole-slide images, introduced by for example differences in acquisition pipeline - between medical centers or over time. In order to harness the great potential presented by deep learning in histopathology, and ensure consistent model behavior, we need a deeper understanding of domain shift and its consequences, such that a model's predictions on new data can be trusted. This work focuses on the internal representation learned by trained convolutional neural networks, and shows how this can be used to formulate a novel measure - the representation shift - for quantifying the magnitude of model specific domain shift. We perform a study on domain shift in tumor classification of hematoxylin and eosin stained images, by considering different datasets, models, and techniques for preparing data in order to reduce the domain shift. The results show how the proposed measure has a high correlation with drop in performance when testing a model across a large number of different types of domain shifts, and how it improves on existing techniques for measuring data shift and uncertainty. The proposed measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. We see techniques for measuring, understanding and overcoming the domain shift as a crucial step towards reliable use of deep learning in the future clinical pathology applications.

Place, publisher, year, edition, pages
IEEE, 2021
Keywords
deep learning, machine learning, domain shift, histopathology
National Category
Computer Sciences
Identifiers
urn:nbn:se:liu:diva-170816 (URN)10.1109/JBHI.2020.3032060 (DOI)000616310200003 ()
Note

Funding:  Wallenberg AI and Autonomous Systems and Software Program (WASP-AI); research environment ELLIIT; AIDA VinnovaVinnova [2017-02447]

Available from: 2020-10-23 Created: 2020-10-23 Last updated: 2023-04-03
6. Primary Tumor and Inter-Organ Augmentations for Supervised Lymph Node Colon Adenocarcinoma Metastasis Detection
Open this publication in new window or tab >>Primary Tumor and Inter-Organ Augmentations for Supervised Lymph Node Colon Adenocarcinoma Metastasis Detection
2021 (English)In: MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION - MICCAI 2021, PT V, SPRINGER INTERNATIONAL PUBLISHING AG , 2021, Vol. 12905, p. 624-633Conference paper, Published paper (Refereed)
Abstract [en]

The scarcity of labeled data is a major bottleneck for developing accurate and robust deep learning-based models for histopathology applications. The problem is notably prominent for the task of metastasis detection in lymph nodes, due to the tissues low tumor-to-non-tumor ratio, resulting in labor- and time-intensive annotation processes for the pathologists. This work explores alternatives on how to augment the training data for colon carcinoma metastasis detection when there is limited or no representation of the target domain. Through an exhaustive study of cross-validated experiments with limited training data availability, we evaluate both an inter-organ approach utilizing already available data for other tissues, and an intra-organ approach, utilizing the primary tumor. Both these approaches result in little to no extra annotation effort. Our results show that these data augmentation strategies can be an efficient way of increasing accuracy on metastasis detection, but fore-most increase robustness.

Place, publisher, year, edition, pages
SPRINGER INTERNATIONAL PUBLISHING AG, 2021
Series
Lecture Notes in Computer Science, ISSN 0302-9743
Keywords
Computer aided diagnosis; Computational pathology; Domain adaptation; Inter-organ; Colon cancer metastasis
National Category
Language Technology (Computational Linguistics)
Identifiers
urn:nbn:se:liu:diva-181214 (URN)10.1007/978-3-030-87240-3_60 (DOI)000712025900060 ()9783030872403 (ISBN)9783030872397 (ISBN)
Conference
International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), ELECTR NETWORK, sep 27-oct 01, 2021
Note

Funding Agencies|Wallenberg AI, Autonomous Systems and Software Program (WASP) - Knut and AliceWallenberg Foundation; strategic research environment ELLIIT; VINNOVAVinnova [2017-02447]

Available from: 2021-11-23 Created: 2021-11-23 Last updated: 2023-04-03

Open Access in DiVA

fulltext(5013 kB)391 downloads
File information
File name FULLTEXT02.pdfFile size 5013 kBChecksum SHA-512
495ef5bccc8daae8af0521e6a5488a9469cde9efec69c61c7062bcad3859b98ae7fec07cc2a59235d8b24f9d7454df7d74704a283947a6455edaf112c9bb5a26
Type fulltextMimetype application/pdf
Order online >>

Other links

Publisher's full text

Search in DiVA

By author/editor
Stacke, Karin
By organisation
Media and Information TechnologyFaculty of Science & Engineering
Cancer and Oncology

Search outside of DiVA

GoogleGoogle Scholar
Total: 425 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 2602 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf