Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Testing Safety-Critical Systems using Fault Injection and Property-Based Testing
Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), Centre for Research on Embedded Systems (CERES).ORCID iD: 0000-0003-1713-3726
2015 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Testing software-intensive systems can be challenging, especially when safety requirements are involved. Property-Based Testing (PBT) is a software testing technique where properties about software are specified and thousands of test cases with a wide range of inputs are automatically generated based on these properties. PBT does not formally prove that the software fulfils its specification, but it is an efficient way to identify deviations from the specification. Safety-critical systems that must be able to deal with faults, without causing damage or injuries, are often tested using Fault Injection (FI) at several abstraction levels. The purpose of FI is to inject faults into a system in order to exercise and evaluate fault handling mechanisms. The aim of this thesis is to investigate how knowledge and techniques from the areas of FI and PBT can be used together to test functional and safety requirements simultaneously.

We have developed a FI tool named FaultCheck that enables PBT tools to use common FI-techniques directly on source code. In order to evaluate and demonstrate our approach, we have applied our tool FaultCheck together with the commercially available PBT tool QuickCheck on a simple and on a complex system. The simple system is the AUTOSAR End-to-End (E2E) library and the complex system is a quadcopter simulator that we developed ourselves. The quadcopter simulator is based on a hardware quadcopter platform that we also developed, and the fault models that we inject into the simulator using FaultCheck are derived from the hardware quadcopter platform. We were able to efficiently apply FaultCheck together with QuickCheck on both the E2E library and the quadcopter simulator, which gives us confidence that FI together with PBT can be used to test and evaluate a wide range of simple and complex safety-critical software.

Place, publisher, year, edition, pages
Halmstad: Halmstad University Press , 2015. , 85 p.
Series
Halmstad University Dissertations, 13
Keyword [en]
Fault Injection, FaultCheck, Property-Based Testing, Simulation, Fault Model, Quadcopter, E2E
National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:hh:diva-28173ISBN: 978-91-87045-29-5 ISBN: 978-91-87045-28-8 OAI: oai:DiVA.org:hh-28173DiVA: diva2:808260
Presentation
2015-05-26, Wigforssalen, House Visionen, Kristian IV:s väg 3, Halmstad, 10:15
Opponent
Supervisors
Projects
PROWESSKARYON
Funder
EU, FP7, Seventh Framework ProgrammeKnowledge Foundation
Note

This research has been funded through the PROWESS EU project (Grant agreement no: 317820), the KARYON EU project (Grant agreement no: 288195) and through EISIGS (grants from the Knowledge Foundation).

Available from: 2015-05-04 Created: 2015-04-27 Last updated: 2015-05-04Bibliographically approved
List of papers
1. Combining Fault-Injection with Property-Based Testing
Open this publication in new window or tab >>Combining Fault-Injection with Property-Based Testing
2014 (English)In: ES4CPS: Workshop Proceedings of Engineering Simulations for Cyber Physical Systems, New York: ACM Press, 2014Conference paper, Published paper (Refereed)
Abstract [en]

In this paper we present a methodology and a platform using Fault Injection (FI) and Property-Based Testing (PBT). PBT is a technique in which test cases are automatically generated from a specification of a system property. The generated test cases vary input stimuli as well as the sequence in which commands are executed. FI is used to accelerate the occurrences of faults in a system to exercise and evaluate fault handling mechanisms and e.g. calculate error detection coverage. By combining the two we have achieved a way of randomly injecting different faults at arbitrary moments in the execution sequence while checking whether certain properties still hold. We use the commercially available tool QuickCheck for generating the test cases and developed FaultCheck for FI. FaultCheck enables the user to utilize fault models, commonly used during FI, from PBT tools like QuickCheck. We demonstrate our method and tools on a simplified example of two Airbag systems that should meet safety requirements. We can easily find a safety violation in one of the examples, whereas by using the AUTOSAR E2E-library implementation, exhaustive testing cannot reveal any such safety violation. This demonstrates that our approach on testing can reveal certain safety violations in a cost-effective way. © 2014 ACM.

Place, publisher, year, edition, pages
New York: ACM Press, 2014
Series
ACM International Conference Proceedings Series
Keyword
Fault model, QuickCheck, FaultCheck, Fault Injection, Property-Based Testing
National Category
Telecommunications
Identifiers
urn:nbn:se:hh:diva-27414 (URN)10.1145/2559627.2559629 (DOI)2-s2.0-84904571627 (Scopus ID)978-1-4503-2614-8 (ISBN)
Conference
Workshop on Engineering Simulations for Cyber Physical Systems, ES4CPS 2014 - Held in Conjunction with the Conference DATE 2014 – Design, Automation and Test in Europe, Dresden, Germany, 28 March, 2014
Available from: 2015-01-06 Created: 2015-01-06 Last updated: 2017-03-24Bibliographically approved
2. Towards Collision Avoidance for Commodity Hardware Quadcopters with Ultrasound Localization
Open this publication in new window or tab >>Towards Collision Avoidance for Commodity Hardware Quadcopters with Ultrasound Localization
Show others...
2015 (English)In: 2015 International Conference on Unmanned Aircraft Systems (ICUAS), 2015, 193-203 p.Conference paper, Published paper (Refereed)
Abstract [en]

We present a quadcopter platform built with commodity hardware that is able to do localization in GNSS-denied areas and avoid collisions by using a novel easy-to-setup and inexpensive ultrasound-localization system. We address the challenge to accurately estimate the copter's position and not hit any obstacles, including other, moving, quadcopters. The quadcopters avoid collisions by placing contours that represent risk around static and dynamic objects and acting if the risk contours overlap with ones own comfort zone. Position and velocity information is communicated between the copters to make them aware of each other. The shape and size of the risk contours are continuously updated based on the relative speed and distance to the obstacles and the current estimated localization accuracy. Thus, the collision-avoidance system is autonomous and only interferes with human or machine control of the quadcopter if the situation is hazardous. In the development of this platform we used our own simulation system using fault-injection (sensor faults, communication faults) together with automatically-generated tests to identify problematic scenarios for which the localization and risk contour parameters had to be adjusted. In the end, we were able to run thousands of simulations without any collisions, giving us confidence that also many real quadcopters can manoeuvre collision free in space-constrained GNSS-denied areas. ©2015 IEEE.

National Category
Engineering and Technology
Identifiers
urn:nbn:se:hh:diva-28169 (URN)10.1109/ICUAS.2015.7152291 (DOI)2-s2.0-84941030880 (Scopus ID)978-1-4799-6009-5 (ISBN)978-1-4799-6009-15 (ISBN)
Conference
The 2015 International Conference on Unmanned Aircraft Systems (ICUAS), Denver, Colorado, USA, June 9-12, 2015
Projects
KARYONPROWESS
Funder
EU, FP7, Seventh Framework ProgrammeKnowledge Foundation
Note

This research has been funded through the KARYON EU project (Grant agreement no: 288195), the PROWESS EU project (Grant agreement no: 317820) and through EISIGS (grants from the Knowledge Foundation).

Available from: 2015-04-27 Created: 2015-04-27 Last updated: 2016-11-30Bibliographically approved
3. Using Simulation, Fault Injection and Property-Based Testing to Evaluate Collision Avoidance of a Quadcopter System
Open this publication in new window or tab >>Using Simulation, Fault Injection and Property-Based Testing to Evaluate Collision Avoidance of a Quadcopter System
2015 (English)In: 2015 IEEE International Conference on Dependable Systems and Networks Workshops (DSN-W) / [ed] Juan E. Guerrero, Los Alamitos, CA: IEEE Computer Society, 2015, 104-111 p.Conference paper, Published paper (Refereed)
Abstract [en]

In this work we use our testing platform based on FaultCheck and QuickCheck that we apply on a quadcopter simulator. We have used a hardware platform as the basis for the simulator and for deriving realistic fault models for our simulations. The quadcopters have a collision-avoidance mechanism that shall take over control when the situation becomes hazardous, steer away from the potential danger and then give control back to the pilot, thereby preventing collisions regardless of what the pilot does. We use our testing platform to randomly generate thousands of simulations with different input stimuli (using QuickCheck) for hundreds of quadcopters, while injecting faults simultaneously (using FaultCheck). This way, we can effectively adjust system parameters and enhance the collision-avoidance mechanism. © 2015 IEEE

Place, publisher, year, edition, pages
Los Alamitos, CA: IEEE Computer Society, 2015
Keyword
Fault Injection, FaultCheck, Property-Based Testing, Simulation, QuickCheck, Fault Model, Quadcopter
National Category
Engineering and Technology
Identifiers
urn:nbn:se:hh:diva-28172 (URN)10.1109/DSN-W.2015.28 (DOI)2-s2.0-84957653792 (Scopus ID)978-1-4673-8044-7 (ISBN)
Conference
1st International Workshop on Safety and Security of Intelligent Vehicles (SSIV), Rio de Janeiro, Brazil, June 22, 2015
Projects
PROWESSKARYON
Funder
EU, FP7, Seventh Framework ProgrammeKnowledge Foundation
Note

This research has been funded through the PROWESS EU project (Grant agreement no: 317820), the KARYON EU project (Grant agreement no: 288195) and through EISIGS (grants from the Knowledge Foundation).

Available from: 2015-04-27 Created: 2015-04-27 Last updated: 2016-11-30Bibliographically approved

Open Access in DiVA

fulltext(3366 kB)624 downloads
File information
File name FULLTEXT01.pdfFile size 3366 kBChecksum SHA-512
77da4e53f4f544bdb6bb25b3bb008beacdff6c19a31fd87d2786fbc47be3c1ea07a6e5756bc4fcd67e8d537df03990dc6e1d64d62ce4e5dc8aea9e6dd4b76364
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Vedder, Benjamin
By organisation
Centre for Research on Embedded Systems (CERES)
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar
Total: 624 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 502 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf