Digitala Vetenskapliga Arkivet

Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Decision Making and Visualizations Based on Test Results
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Westermo R & D AB, Sweden.ORCID iD: 0000-0003-1688-6937
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.ORCID iD: 0000-0003-0611-2655
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.ORCID iD: 0000-0002-5032-2310
2018 (English)In: Empirical Software Engineering and Measurement, 12th International Symposium on ESEM18, 2018, article id 34Conference paper, Published paper (Refereed)
Abstract [en]

Background: Testing is one of the main methods for quality assurance in the development of embedded software, as well as in software engineering in general. Consequently, test results (and how they are reported and visualized) may substantially influence business decisions in software-intensive organizations. Aims: This case study examines the role of test results from automated nightly software testing and the visualizations for decision making they enable at an embedded systems company in Sweden. In particular, we want to identify the use of the visualizations for supporting decisions from three aspects: in daily work, at feature branch merge, and at release time. Method: We conducted an embedded case study with multiple units of analysis by conducting interviews, questionnaires, using archival data and participant observations. Results: Several visualizations and reports built on top of the test results database are utilized in supporting daily work, merging a feature branch to the master and at release time. Some important visualizations are: lists of failing test cases, easy access to log files, and heatmap trend plots. The industrial practitioners perceived the visualizations and reporting as valuable, however they also mentioned several areas of improvement such as better ways of visualizing test coverage in a functional area as well as better navigation between different views. Conclusions: We conclude that visualizations of test results are a vital decision making tool for a variety of roles and tasks in embedded software development, however the visualizations need to be continuously improved to keep their value for its stakeholders.

Place, publisher, year, edition, pages
2018. article id 34
Keywords [en]
Software Testing, Visualizations, Decision Making
National Category
Engineering and Technology Computer Systems
Identifiers
URN: urn:nbn:se:mdh:diva-40902DOI: 10.1145/3239235.3268921ISI: 000469776800034Scopus ID: 2-s2.0-85053207166ISBN: 978-1-4503-5823-1 (print)OAI: oai:DiVA.org:mdh-40902DiVA, id: diva2:1247828
Conference
Empirical Software Engineering and Measurement, 12th International Symposium on ESEM18, 11 Oct 2018, Oulu, Finland
Projects
TOCSYC - Testing of Critical System Characteristics (KKS)The Volvo chair of vehicular electronics and software architectureITS ESS-H Industrial Graduate School in Reliable Embedded Sensor SystemsTESTMINE - Mining Test Evolution for Improved Software Regression Test Selection (KKS)Available from: 2018-09-13 Created: 2018-09-13 Last updated: 2022-11-09Bibliographically approved
In thesis
1. Automated System Level Software Testing of Networked Embedded Systems
Open this publication in new window or tab >>Automated System Level Software Testing of Networked Embedded Systems
2018 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Embedded systems are ubiquitous and play critical roles in management systems for industry and transport. Software failures in these domains may lead to loss of production or even loss of life, so the software in these systems needs to be reliable. Software testing is a standard approach for quality assurance of embedded software, and many software development processes strive for test automation. However, important challenges for successful software test automation are: lack of time for testing, lack of test environment availability, and an excess of test results information that renders decision-making hard.

In this thesis these challenges are tackled in three ways. First, in order to combat lack of time for testing, a method for automated system level regression test selection was implemented and evaluated using data from several years of nightly testing in a real world industrial setting. It was shown that automated test selection can be integrated into system level nightly testing and solve problems such as nightly testing not finishing on time. Second, in order to improve the hardware coverage of devices in the test environment and how test cases map to hardware, an algorithm based on the subgraph isomorphism problem was implemented and evaluated using industrial data. This implementation was significantly faster than the previous implementation, and the mapping process was done in such a way that hardware coverage increased over iterations. Third, to better understand decision-making in critical steps of the software development process in an industrial setting, two empirical studies were conducted. The results showed how visualizations and a test results database support decision-making. Results also describe the overall flow of information in software testing: from developers to hardware, and back to developers via the test results database.

Automated system level software testing of networked embedded systems can be difficult to achieve. This thesis addresses several important challenges and provides results that are of interest both to industrial practitioners and researchers.

Place, publisher, year, edition, pages
Västerås: Mälardalen University, 2018
Series
Mälardalen University Press Licentiate Theses, ISSN 1651-9256 ; 275
National Category
Software Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-41109 (URN)978-91-7485-408-4 (ISBN)
Presentation
2018-11-13, Room Zeta, Mälardalens högskola, Västerås, 13:15 (English)
Opponent
Supervisors
Available from: 2018-10-02 Created: 2018-10-02 Last updated: 2022-11-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Strandberg, Per ErikAfzal, WasifSundmark, Daniel
By organisation
Embedded SystemsEmbedded Systems
Engineering and TechnologyComputer Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 150 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf