Change search
ReferencesLink to record
Permanent link

Direct link
Testing of Heterogeneous Systems
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
2014 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Context: A system of systems often exhibits heterogeneity, for instance in implementation, hardware, process and verification. We define a heterogeneous system, as a system comprised of multiple systems (system of systems) where at least one subsystem exhibits heterogeneity with respect to the other systems. The system of systems approach taken in development of heterogeneous systems give rise to various challenges due to continuous change in configurations and multiple interactions between the functionally independent subsystems. The challenges posed to testing of heterogeneous systems are mainly related to interoperability, conformance and large regression test suites. Furthermore, the inherent complexities of heterogeneous systems also pose challenge to the specification, selection and execution of tests. Objective: The main objective of this licentiate thesis is to provide an insight on the state of the art in testing heterogeneous systems. Moreover, we also aimed to investigate different test techniques used to test heterogeneous systems in industrial settings and their usefulness as well as to identify and prioritize different information sources that can help practitioners to define a generic search space for test case selection process. Method: The findings presented in this thesis are obtained through a controlled experiment, a systematic literature review (SLR), a case study and an exploratory survey. The purpose of systematic literature review was to investigate the existing state of art in testing heterogeneous systems and identification of research gaps. The results from the SLR further laid down the foundation of action research conducted through an exploratory survey to compare different test techniques. We also conducted an industrial case study to investigate the relevant data sources for search space initiation to prioritize and specify test cases in context of heterogeneous systems. Results: Based on our literature review, we found that testing of heterogeneous systems is considered a problem of integration and system testing. It has been observed that multiple interactions between the system and subsystems results into a testing challenge, especially when the configurations change continuously. It is also observed that current literature targets the problem of testing heterogeneous systems with multiple test objectives resulting in employing different test methods to reach a domain specific testing challenge. Using the exploratory survey, we found three test techniques to be most relevant in context of testing heterogeneous systems. However, the most frequently used technique mentioned by the practitioners is manual exploratory testing which is not a much researched topic in the context of heterogeneous systems. Moreover, multiple information sources for test selection process are identified through the case study and the survey. Conclusion: Companies engaged in development of heterogeneous systems encounter huge challenges due to multiple interactions between the system and subsystems. However, the conclusions we draw from the research studies included herein show a gap between literature and industry. Search-based testing is widely discussed in the literature but is the least used test technique in industrial practice. Moreover, for test selection process there are no frameworks that take in account all the information sources that we investigated. Therefore, to fill this gap there is a need for an optimized test selection process based on the information sources. There is also a need to study different test techniques identified through our SLR and survey and compare these techniques on real heterogeneous systems.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Institute of Technology , 2014.
Blekinge Institute of Technology Licentiate Dissertation Series, ISSN 1650-2140 ; 3
Keyword [en]
Testing, Heterogeneous systems, Information sources, Test selection
National Category
Software Engineering
URN: urn:nbn:se:bth-00591Local ID: 978-91-7295-282-9OAI: diva2:834219
Available from: 2014-06-24 Created: 2014-05-12 Last updated: 2015-06-30Bibliographically approved

Open Access in DiVA

fulltext(1666 kB)120 downloads
File information
File name FULLTEXT01.pdfFile size 1666 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Ghazi, Nauman
By organisation
Department of Software Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 120 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 134 hits
ReferencesLink to record
Permanent link

Direct link