Change search
Refine search result
1 - 28 of 28
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. 7809198067.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering.
    A Runtime Verification based Concurrency Bug Detector for FreeRTOS Embedded SoftwareIn: Article in journal (Other academic)
    The full text will be freely available from 2019-03-01 17:14
  • 2.
    Abbaspour Asadollah, Sara
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eldh, Sigrid
    Ericsson AB, Kista, Sweden.
    Hansson, Hans
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Paul Enoiu, Eduard
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Study on Concurrency Bugs in an Open Source Software2016In: IFIP Advances in Information and Communication Technology, vol. 472, 2016, Vol. 472, p. 16-31Conference paper (Refereed)
    Abstract [en]

    Concurrent programming puts demands on software debugging and testing, as concurrent software may exhibit problems not present in sequential software, e.g., deadlocks and race conditions. In aiming to increase efficiency and effectiveness of debugging and bug-fixing for concurrent software, a deep understanding of concurrency bugs, their frequency and fixingtimes would be helpful. Similarly, to design effective tools and techniques for testing and debugging concurrent software understanding the differences between non-concurrency and concurrency bugs in real-word software would be useful.

  • 3.
    Bergström, Henning
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Using Timed Base-Choice Coverage Criterion for Testing Industrial Control Software2017In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2017, Institute of Electrical and Electronics Engineers Inc. , 2017, p. 216-219Conference paper (Refereed)
    Abstract [en]

    The base-choice criterion was proposed as a suitable technique for testing software based on its nominal choice of input parameters. Test cases are created based on this strategy by varying the values of one input parameter at a time while keeping the values of the other parameters fixed on the base choice. However, this strategy might not be as effective when used on industrial control software for testing timed behavior. We propose to incorporate time as another parameter when generating and executing tests by defining the timed base-choice coverage criterion. We performed an empirical evaluation using 11 industrial programs written in the IEC 61131-3 programming language. We found that tests generated for timed base-choice criterion show better code coverage (7% improvement) and fault detection (27% improvement) in terms of mutation score than tests satisfying base-choice coverage criterion. The results demonstrate the feasibility of applying timed base-choice criterion for testing industrial control software.

  • 4.
    Charbachi, Peter
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Eklund, Linus
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Can Pairwise Testing Perform Comparably to Manually Handcrafted Testing Carried Out by Industrial Engineers?2017In: Proceedings - 2017 IEEE International Conference on Software Quality, Reliability and Security Companion, QRS-C 2017, Institute of Electrical and Electronics Engineers Inc. , 2017, p. 92-99Conference paper (Refereed)
    Abstract [en]

    Testing is an important activity in engineering of industrial software. For such software, testing is usually performed manually by handcrafting test suites based on specific design techniques and domain-specific experience. To support developers in testing, different approaches for producing good test suites have been proposed. In the last couple of years combinatorial testing has been explored with the goal of automatically combining the input values of the software based on a certain strategy. Pairwise testing is a combinatorial technique used to generate test suites by varying the values of each pair of input parameters to a system until all possible combinations of those parameters are created. There is some evidence suggesting that these kinds of techniques are efficient and relatively good at detecting software faults. Unfortunately, there is little experimental evidence on the comparison of these combinatorial testing techniques with, what is perceived as, rigorous manually handcrafted testing. In this study we compare pairwise test suites with test suites created manually by engineers for 45 industrial programs. The test suites were evaluated in terms of fault detection, code coverage and number of tests. The results of this study show that pairwise testing, while useful for achieving high code coverage and fault detection for the majority of the programs, is almost as effective in terms of fault detection as manual testing. The results also suggest that pairwise testing is just as good as manual testing at fault detection for 64% of the programs.

  • 5.
    de Oliveira Neto, Francisco Gomes
    et al.
    Chalmers-University of Gothenburg, Gothenburg, Sweden.
    Ahmad, Azeem
    Chalmers-University of Gothenburg, Gothenburg, Sweden.
    Leifler, Ola
    Linköping University, Linköping, Sweden.
    Sandhal, Kristian
    Linköping University, Linköping, Sweden.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Improving Continuous Integration with Similarity-based Test Case Selection2018In: 13th IEEE/ACM International Workshop on Automation of Software Test AST'18, 2018, Vol. F137727, p. 39-45Conference paper (Refereed)
    Abstract [en]

    Automated testing is an essential component of Continuous Integration (CI) and Delivery (CD), such as scheduling automated test sessions on overnight builds. That allows stakeholders to execute entire test suites and achieve exhaustive test coverage, since running all tests is often infeasible during work hours, i.e., in parallel to development activities. On the other hand, developers also need test feedback from CI servers when pushing changes, even if not all test cases are executed. In this paper we evaluate similarity-based test case selection (SBTCS) on integration-level tests executed on continuous integration pipelines of two companies. We select test cases that maximise diversity of test coverage and reduce feedback time to developers. Our results confirm existing evidence that SBTCS is a strong candidate for test optimisation, by reducing feedback time (up to 92% faster in our case studies) while achieving full test coverage using only information from test artefacts themselves.

  • 6.
    Enoiu, Eduard
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Automatic test generation for industrial control software2016Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Since the early days of software testing, automatic test generation has been suggested as a way of allowing tests to be created at a lower cost. However, industrially useful and applicable tools for automatic test generation are still scarce. As a consequence, the evidence regarding the applicability or feasibility of automatic test generation in industrial practice is limited. This is especially problematic if we consider the use of automatic test generation for industrial safety-critical control systems, such as are found in power plants, airplanes, or trains.

    In this thesis, we improve the current state of automatic test generation by developing a technique based on model-checking that works with IEC 61131-3 industrial control software. We show how automatic test generation for IEC 61131-3 programs, containing both functional and timing information, can be solved as a model checking problem for both code and mutation coverage criteria. 

    The developed technique has been implemented in the CompleteTest tool. To evaluate the potential application of our technique, we present several studies where the tool is applied to industrial control software. Results show that CompleteTest is viable for use in industrial practice; it is efficient in terms of the time required to generate tests that satisfy both code and mutation coverage and scales well for most of the industrial programs considered.

    However, our results also show that there are still challenges associated with the use of automatic test generation. In particular, we found that while automatically generated tests, based on code coverage, can exercise the logic of the software as well as tests written manually, and can do so in a fraction of the time, they do not show better fault detection compared to manually created tests. Specifically, it seems that manually created tests are able to detect more faults of certain types (i.e, logical replacement, negation insertion and timer replacement) than automatically generated tests. To tackle this issue, we propose an approach for improving fault detection by using mutation coverage as a test criterion. We implemented this approach in the CompleteTest tool and used it to evaluate automatic test generation based on mutation testing. While the resulting tests were more effective than automatic tests generated based on code coverage, in terms of fault detection, they still were not better than manually created tests.

    In summary, our results highlight the need for improving the goals used by automatic test generation tools. Specifically, fault detection scores could be increased by considering some new mutation operators as well as higher-order mutations. Our thesis suggests that automatically generated test suites are significantly less costly in terms of testing time than manually created test suites. One conclusion, strongly supported by the results of this thesis, is that automatic test generation is efficient but currently not quite as effective as manual testing. This is a significant progress that needs to be further studied; we need to consider the implications and the extent to which automatic test generation can be used in the development of reliable safety-critical systems.

  • 7.
    Enoiu, Eduard
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Model Checking-Based Software Testing for Function-Block Diagrams2014Licentiate thesis, comprehensive summary (Other academic)
    Abstract [en]

    Software testing becomes more complex, more time-consuming, and more expensive. The risk that software errors remain undetected and cause critical failures increases. Consequently, in safety-critical development, testing software is standardized and it requires an engineer to show that tests fully exercise, or cover, the logic of the software. This method often requires a trained engineer to perform manual test generation, is prone to human error, and is expensive or impractical to use frequently in production. To overcome these issues, software testing needs to be performed earlier in the development process, more frequently, and aided by automated tools.

    We devised an automated test generation tool called COMPLETETEST that avoids many of those problems. The method implemented in the tool and described in this thesis, works with software written in Function Block Diagram language, and can provide tests in just a few seconds. In addition, it does not rely on the expertise of a researcherspecialized in automated test generation and model checking. Although COMPLETETEST itself uses a model checker, a complex technique requiring a high level of expertise to generate tests, it provides a straightforward tabular interface to the intended users. In this way, its users do not need to learn the intricacies of using this approach such as how coverage criteria can be formalized and used by a model checker to automatically generate tests. If the technique can be demonstrated to work in production, it could detect and aid in the detection of errors in safety-critical software development, where conventional testing is not always applicable and efficient.

    We conducted studies based on industrial use-case scenarios from Bombardier Transportation AB, showing how the approach can be applied to generate tests in software systems used in the safety-critical domain. To evaluate the approach, it was applied on real-world programs. The results indicate that it is efficient in terms of time required to generate tests and scales well for most of the software. There are still issues to resolve before the technique can be applied to more complex software, but we are already working on ways to overcome them. In particular, we need to understand how its usage in practice can vary depending on human and software process factors.

  • 8.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enablers and impediments for collaborative research in software testing: An empirical exploration2014In: WISE 2014 - Proceedings of the 2014 ACM International Workshop on Long-Term Industrial Collaboration on Software Engineering, Co-located with ASE 2014, 2014, p. 49-54Conference paper (Refereed)
    Abstract [en]

    When it comes to industrial organizations, current collaboration efforts in software engineering research are very often kept in-house, depriving these organizations off the skills necessary to build independent collaborative research. The current trend, towards empirical software engineering research, requires certain standards to be established which would guide these collaborative efforts in creating a strong partnership that promote independent, evidence-based, software engineering research. This paper examines key enabling factors for an effcient and effective industry-academia collaboration in the software testing domain. A major finding of the research was that while technology is a strong enabler to better collaboration, it must be complemented with industrial openness to disclose research results and the use of a dedicated tooling platform. We use as an example an automated test generation approach that has been developed in the last two years collaboratively with Bombardier Transportation AB in Sweden.

  • 9.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Ostrand, Thomas
    Software Engineering Research Consultant, Västerås, Sweden.
    Weyuker, Elaine
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Swedish Institute of Computer Science, Stockholm, Sweden.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Automated Test Generation using Model-Checking: An Industrial Evaluation2016In: International Journal on Software Tools for Technology Transfer (STTT), ISSN 1433-2779, E-ISSN 1433-2787, Vol. 18, no 3, p. 335-353Article in journal (Refereed)
    Abstract [en]

    In software development, testers often focus on functional testing to validate implemented programs against their specifications. In safety critical software development, testers are also required to show that tests exercise, or cover, the structure and logic of the implementation. To achieve different types of logic coverage, various program artifacts such as decisions and conditions are required to be exercised during testing. Use of model-checking for structural test generation has been proposed by several researchers. The limited application to models used in practice and the state-space explosion can, however, impact model-checking and hence the process of deriving tests for logic coverage. Thus, there is a need to validate these approaches against relevant industrial systems such that more knowledge is built on how to efficiently use them in practice. In this paper, we present a tool-supported approach to handle software written in the Function Block Diagram language such that logic coverage criteria can be formalized and used by a model-checker to automatically generate tests. To this end, we conducted a study based on industrial use-case scenarios from Bombardier Transportation AB, showing how our toolbox COMPLETETEST can be applied to generate tests in software systems used in the safety-critical domain. To evaluate the approach, we applied the toolbox to 157 programs and found that it is efficient in terms of time required to generate tests that satisfy logic coverage and scales well for most of the programs.

  • 10.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Swedish Institute of Computer Science, Kista, Sweden.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Controlled Experiment in Testing of Safety-Critical Embedded Software2016In: Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation, ICST 2016, 2016, p. 1-11Conference paper (Refereed)
    Abstract [en]

    In engineering of safety critical systems, regulatory standards often put requirements on both traceable specification-based testing, and structural coverage on program units. Automated test generation techniques can be used to generate inputs to cover the structural aspects of a program. However, there is no conclusive evidence on how automated test generation compares to manual test design, or how testing based on the program implementation relates to specification-based testing. In this paper, we investigate specification-and implementation-based testing of embedded software written in the IEC 61131-3 language, a programming standard used in many embedded safety critical software systems. Further, we measure the efficiency and effectiveness in terms of fault detection. For this purpose, a controlled experiment was conducted, comparing tests created by a total of twenty-three software engineering master students. The participants worked individually on manually designing and automatically generating tests for two IEC 61131-3 programs. Tests created by the participants in the experiment were collected and analyzed in terms of mutation score, decision coverage, number of tests, and testing duration. We found that, when compared to implementation-based testing, specification-based testing yields significantly more effective tests in terms of the number of faults detected. Specifically, specification-based tests more effectively detect comparison and value replacement type of faults, compared to implementation-based tests. On the other hand, implementation-based automated test generation leads to fewer tests (up to 85% improvement) created in shorter time than the ones manually created based on the specification.

  • 11.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Comparative Study of Manual and Automated Testing for Industrial Control Software2017In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, 2017, p. 412-417, article id 7927994Conference paper (Refereed)
    Abstract [en]

    Automated test generation has been suggested as a way of creating tests at a lower cost. Nonetheless, it is not very well studied how such tests compare to manually written ones in terms of cost and effectiveness. This is particularly true for industrial control software, where strict requirements on both specification-based testing and code coverage typically are met with rigorous manual testing. To address this issue, we conducted a case study in which we compared manually and automatically created tests. We used recently developed real-world industrial programs written in the IEC 61131-3, a popular programming language for developing industrial control systems using programmable logic controllers. The results show that automatically generated tests achieve similar code coverage as manually created tests, but in a fraction of the time (an average improvement of roughly 90%). We also found that the use of an automated test generation tool does not result in better fault detection in terms of mutation score compared to manual testing. Specifically, manual tests more effectively detect logical, timer and negation type of faults, compared to automatically generated tests. The results underscore the need to further study how manual testing is performed in industrial practice and the extent to which automated test generation can be used in the development of reliable systems.

  • 12.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Doganay, Kivanc
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Bohlin, Markus
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    MOS: An Integrated Model-based and Search-based Testing Tool for Function Block Diagrams2013In: 2013 1st International Workshop on Combining Modelling and Search-Based Software Engineering, CMSBSE 2013 - Proceedings, 2013, p. 55-60Conference paper (Refereed)
    Abstract [en]

    In this paper we present a new testing tool for safety critical applications described in Function Block Diagram (FBD) language aimed to support both a model and a search-based approach. Many benefits emerge from this tool, including the ability to automatically generate test suites from an FBD program in order to comply to quality requirements such as component testing and specific coverage measurements. Search-based testing methods are used to generate test data based on executable code rather than the FBD program, alleviating any problems that may arise from the ambiguities that occur while creating FBD programs. Test cases generated by both approaches are executed and used as a way of cross validation. In the current work, we describe the architecture of the tool, its workflow process, and a case study in which the tool has been applied in a real industrial setting to test a train control management system.

  • 13.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Marinescu, Raluca
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Aida
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Seceleanu, Cristina
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Design Tool for Service-oriented Systems2013In: Electronical Notes in Theoretical Computer Science, ISSN 1571-0661, E-ISSN 1571-0661, Vol. 295, p. 95-100Article in journal (Other academic)
    Abstract [en]

    In this paper we present a modeling and analysis tool for service-oriented systems. The tool enables graphical modeling of service-based systems, within the resource-aware timed behavioral language Remes, as well as a textual system description. We have developed a graphical environment where services can be composed as desired by the user, together with a textual service composition interface in which compositions can also be checked for correctness. We also provide automated traceability between the two design interfaces, which results in a tool that enhances the potential of system design by intuitive service manipulation. The paper presents the design principles, infrastructure, and the user interface of our tool.

  • 14.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University.
    Marinescu, Raluca
    Mälardalen University, School of Innovation, Design and Engineering.
    Seceleanu, Cristina
    Mälardalen University, School of Innovation, Design and Engineering.
    Paul, Pettersson
    Mälardalen University, School of Innovation, Design and Engineering.
    ViTAL : A Verification Tool for EAST-ADL Models using UPPAAL PORT2012In: Proceedings of the 17th IEEE International Conference on Engineering of Complex Computer Systems, Paris, France, 2012, p. 328-337Conference paper (Refereed)
    Abstract [en]

    A system’s architecture influence on the functions and other properties of embedded systems makes its high level analysis and verification very desirable. EAST-ADL is an architecture description language dedicated to automotive embedded system design with focus on structural and functional modeling. The behavioral description is not integrated within the execution semantics, which makes it harder to transform, analyze, and verify EAST-ADL models. Model-based techniques help address this issue by enabling automated transformation between different design models, and providing means for simulation and verification. We present a verification tool, called ViTAL, which provides the possibility to express the functional EAST-ADL behavior as timed automata models, which have precise semantics and can be formally verified. The ViTAL tool enables the transformation of EAST-ADL functional models to the UPPAAL PORT tool for model checking. This method improves the verification of functional and timing requirements in EAST-ADL, and makes it possible to identify dependencies and potential conflicts between different vehicle functions before the actual AUTOSAR implementation.

  • 15.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Marinescu, Raluca
    Mälardalen University, School of Innovation, Design and Engineering.
    Seceleanu, Cristina
    Mälardalen University, School of Innovation, Design and Engineering.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering.
    Towards the Analysis and Verification of EAST-ADL Models using UPPAAL PORT2012Report (Other academic)
    Abstract [en]

    A system’s architecture influence on the functions and other properties of embedded systems makes its high level analysis and verification very desirable. EAST-ADL is an architecture description language dedicated to automotive embedded system design with focus on structural and functional modeling. The behavioral description is not integrated within the execution semantics, which makes it harder to transform, analyze, and verify EAST-ADL models. Model-based techniques help address this issue by enabling automated transformation between different design models, and providing means for simulation and verification. We present a verification tool, called ViTAL, which provides the possibility to express the functional EAST-ADL behavior as timed automata models, which have precise semantics and can be formally verified. The ViTAL tool enables the transformation of EAST-ADL functional models to the UPPAAL PORT tool for model checking. This method improves the verification of functional and timing requirements in EAST-ADL, and makes it possible to identify dependencies and potential conflicts between different vehicle functions before the actual AUTOSAR implementation.

  • 16.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Feldt, Robert
    Blekinge Institute of Technology, Sweden.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Mutation-Based Test Generation for PLC Embedded Software using Model Checking2016In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016, Vol. 9976, p. 155-171Conference paper (Refereed)
    Abstract [en]

    Testing is an important activity in engineering of industrial embedded software. In certain application domains (e.g., railway industry) engineering software is certified according to safety standards that require extensive software testing procedures to be applied for the development of reliable systems. Mutation analysis is a technique for creating faulty versions of a software for the purpose of examining the fault detection ability of a test suite. Mutation analysis has been used for evaluating existing test suites, but also for generating test suites that detect injected faults (i.e., mutation testing). To support developers in software testing, we propose a technique for producing test cases using an automated test generation approach that operates using mutation testing for software written in IEC 61131-3 language, a programming standard for safety-critical embedded software, commonly used for Programmable Logic Controllers (PLCs). This approach uses the Uppaal model checker and is based on a combined model that contains all the mutants and the original program. We applied this approach in a tool for testing industrial PLC programs and evaluated it in terms of cost and fault detection. For realistic validation we collected industrial experimental evidence on how mutation testing compares with manual testing as well as automated decision-coverage adequate test generation. In the evaluation, we used manually seeded faults provided by four industrial engineers. The results show that even if mutation-based test generation achieves better fault detection than automated decision coverage-based test generation, these mutation-adequate test suites are not better at detecting faults than manual test suites. However, the mutation-based test suites are significantly less costly to create, in terms of testing time, than manually created test suites. Our results suggest that the fault detection scores could be improved by considering some new and improved mutation operators (e.g., Feedback Loop Insertion Operator (FIO)) for PLC programs as well as higher-order mutations.

  • 17.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Causevic, Adnan
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Comparative Study of Manual and Automated Testing in Industrial Embedded SoftwareManuscript (preprint) (Other (popular science, discussion, etc.))
    Abstract [en]

    Testing is an important activity in engineering of industrial embedded software. In certain application domains (e.g., railway industry) engineering software is certified according to safety standards that require extensive software testing procedures to be applied for the development of reliable systems. Mutation analysis is a technique for creating faulty versions of a software for the purpose of examining the fault detection ability of a test suite. Mutation analysis has been used for evaluating existing test suites, but also for generating test suites that detect injected faults (i.e., mutation testing). To support developers in software testing, we propose a technique for producing test cases using an automated test generation approach that operates using mutation testing for software written in IEC 61131-3 language, a programming standard for safety-critical embedded software, commonly used for Programmable Logic Controllers (PLCs). This approach uses the Uppaal model checker and is based on a combined model that contains all the mutants and the original program. We applied this approach in a tool for testing industrial PLC programs and evaluated it in terms of cost and fault detection. For realistic validation we collected industrial experimental evidence on how mutation testing compares with manual testing as well as automated decision-coverage adequate test generation. In the evaluation, we used manually seeded faults provided by four industrial engineers. The results show that even if mutation-based test generation achieves better fault detection than automated decision coverage-based test generation, these mutation-adequate test suites are not better at detecting faults than manual test suites. However, the mutation-based test suites are significantly less costly to create, in terms of testing time, than manually created test suites. Our results suggest that the fault detection scores could be improved by considering some new and improved mutation operators (e.g., Feedback Loop Insertion Operator (FIO)) for PLC programs as well as higher-order mutations.

  • 18.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Model-based test suite generation for function block diagrams using the UPPAAL model checker2013In: Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2013, 2013, p. 158-167Conference paper (Refereed)
    Abstract [en]

    A method for model-based test generation of safety-critical embedded applications using Programmable Logic Controllers and implemented in a programming language such as Function Block Diagram (FBD) is described. The FBD component model is based on the IEC 1131 standard and it is used primarily for embedded systems, in which timeliness is an important property to be tested. Our method involves the transformation of FBD programs with timed annotations into timed automata models which are used to automatically generate test suites. Specifically we demonstrate how to use model transformation for formalization and model checking of FBD programs using the UPPAAL tool. Many benefits emerge from this method, including the ability to automatically generate test suites from a formal model in order to ensure compliance to strict quality requirements including unit testing and specific coverage measurements. The approach is experimentally assessed on a train control system in terms of consumed resources.

  • 19.
    Enoiu, Eduard Paul
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Using Logic Coverage to Improve Testing Function Block Diagrams2013In: Testing Software and Systems: Lecture Notes in Computer Science, Volume 8254, Springer Berlin Heidelberg , 2013, p. 1-16Chapter in book (Refereed)
    Abstract [en]

    In model-driven development, testers are often focusing on functional model-level testing, enabling verification of design models against their specifications. In addition, in safety-critical software development, testers are required to show that tests cover the structure of the implementation. Testing cost and time savings could be achieved if the process of deriving test cases for logic coverage is automated and provided test cases are ready to be executed. The logic coverage artifacts, i.e., predicates and clauses, are required for different logic coverage, e.g., MC/DC. One way of dealing with test case generation for ensuring logic coverage is to approach it as a model-checking problem, such that model-checking tools automatically create test cases. We show how logic coverage criteria can be formalized and used by a model-checker to provide test cases for ensuring this coverage on safety-critical software described in the Function Block Diagram programming language. Based on our experiments, this approach, supported by a tool chain, is an applicable and useful way of generating test cases for covering Function Block Diagrams.

  • 20.
    Ericsson, Sara
    et al.
    Mälardalen University.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Combinatorial modeling and test case generation for industrial control software using ACTS2018In: Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 414-425Conference paper (Refereed)
    Abstract [en]

    Combinatorial testing has been suggested as an effective method of creating test cases at a lower cost. However, industrially applicable tools for modeling and combinatorial test generation are still scarce. As a direct effect, combinatorial testing has only seen a limited uptake in industry that calls into question its practical usefulness. This lack of evidence is especially troublesome if we consider the use of combinatorial test generation for industrial safety-critical control software, such as are found in trains, airplanes, and power plants. To study the industrial application of combinatorial testing, we evaluated ACTS, a popular tool for combinatorial modeling and test generation, in terms of applicability and test efficiency on industrial-sized IEC 61131-3 industrial control software running on Programmable Logic Controllers (PLC). We assessed ACTS in terms of its direct applicability in combinatorial modeling of IEC 61131-3 industrial software and the efficiency of ACTS in terms of generation time and test suite size. We used 17 industrial control programs provided by Bombardier Transportation Sweden AB and used in a train control management system. Our results show that not all combinations of algorithms and interaction strengths could generate a test suite within a realistic cut-off time. The results of the modeling process and the efficiency evaluation of ACTS are useful for practitioners considering to use combinatorial testing for industrial control software as well as for researchers trying to improve the use of such combinatorial testing techniques.

  • 21.
    Flemström, Daniel
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Gustafsson, T.
    Scania CV AB, Sweden.
    Kobetski, A.
    RISE SICS AB, Sweden.
    From natural language requirements to passive test cases using guarded assertions2018In: Proceedings - 2018 IEEE 18th International Conference on Software Quality, Reliability, and Security, QRS 2018, Institute of Electrical and Electronics Engineers Inc. , 2018, p. 470-481Conference paper (Refereed)
    Abstract [en]

    In large-scale embedded system development, requirements are often expressed in natural language. Translating these requirements to executable test cases, while keeping the test cases and requirements aligned, is a challenging task. While such a transformation typically requires extensive domain knowledge, we show that a systematic process in combination with passive testing would facilitate the translation as well as linking the requirements to tests. Passive testing approaches observe the behavior of the system and test their correctness without interfering with the normal behavior. We use a specific approach to passive testing: guarded assertions (G/A). This paper presents a method for transforming system requirements expressed in natural language into G/As. We further present a proof of concept evaluation, performed at Bombardier Transportation Sweden AB, in which we show how the process would be used, together with practical advice of the reasoning behind the translation steps.

  • 22.
    Hussain, Aliya
    et al.
    Mälardalen University.
    Tiwari, Saurabh
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Suryadevara, Jagadish
    Volvo Construction Equipment AB, Sweden.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    From Modeling to Test Case Generation in the Industrial Embedded System Domain2018In: First International Workshop on Model-Driven Engineering for Design-Runtime Interaction in Complex Systems MDE@DeRun, 2018Conference paper (Refereed)
    Abstract [en]

    Model-based testing (MBT) is the process of generating test cases from specification models representing system requirements and the desired functionality. The generated test cases are then executed on the system under test in an attempt to obtain a pass or fail verdict. While different MBT techniques have been developed, only a few target the real-world industrial embedded system domain and show evidence on its applicability. As a consequence, there is a serious need to investigate the use of MBT and the evidence on how modeling and test generation can improve the current way of manually creating test cases based on natural language requirements. In this paper, we describe an on-going investigation being carried out to improve the current testing processes by using the MBT approach within an industrial context. Our results suggest that activity and structure diagrams, developed under MBT, are useful for describing the test specification of an accelerator pedal control function. The use of MBT results in less number of test cases compared to manual testing performed by industrial engineers.

  • 23.
    Kang, Eun-Young
    et al.
    University of Namur, Belgium .
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Marinescu, Raluca
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Seceleanu, Cristina
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Schnobbens, Pierre Yves
    University of Namur, Belgium .
    Pettersson, Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    A Methodology for Formal Analysis and Verification of EAST-ADL Models2013In: Reliability Engineering & System Safety, ISSN 0951-8320, E-ISSN 1879-0836, Vol. 120, no Special Issue, p. 127-138Article in journal (Refereed)
    Abstract [en]

    The architectural design of embedded software has a direct impact on the final implementation, with respect to performance and other quality attributes. Therefore, guaranteeing that an architectural model meets the specified requirements is beneficial for detecting software flaws early in the development process. In this paper, we present a formal modeling and verification methodology for safety-critical automotive products that are originally described in the domain-specific architectural language East-adl. We propose a model-based approach that integrates the architectural models with component-aware model checking, and describe its tool support called ViTAL. The functional and timing behavior of each function block in the East-adl model, as well as the interactions between function blocks are formally captured and expressed as Timed Automata models, which have precise semantics and can be formally verified with ViTAL. Furthermore, we show how our approach, supported by ViTAL, can be used to formally prove that the East-adl system model fulfills the specified real-time requirements and behavioral constraints. We demonstrate that the approach improves the modeling and verification capability of East-adl and identifies dependencies, as well as potential conflicts between different automotive functions before implementation. The method is substantiated by verifying an automotive braking system model, with respect to particular functional and timing requirements.

  • 24.
    Marinescu, Raluca
    et al.
    Mälardalen University, School of Innovation, Design and Engineering.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering.
    Extending EAST-ADL for modeling and analysis of system's resource-usage2012In: Proceedings - International Computer Software and Applications Conference, 2012, p. 532-537Conference paper (Refereed)
    Abstract [en]

    EAST-ADL is an architectural description language dedicated to automotive embedded systems design, with focus on structural and functional modeling. The current architectural notations lack support for modeling and analysis of resource-usage, and therefore it is not possible to reason about resource requirements. In this paper, we describe our work towards filling the gap between EAST-ADL language and formal modeling and analysis of system’s resource usage, by extending the EAST-ADL language with embedded resources, such as storage, energy, communication and computation. To formalize this approach and provide a basis for rigorous analysis, we show how to analyze EAST-ADL models using the framework of priced timed automata and weighted CTL. We report our experiences from applying this approach for integrating resource-wise analysis into EAST-ADL.

  • 25.
    Marinescu, Raluca
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Seceleanu, Cristina
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Statistical Analysis of Resource Usage of Embedded Systems Modeled in EAST-ADL2015In: IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 2015, p. 380-385Conference paper (Refereed)
    Abstract [en]

    The growing complexity of modern automotive embedded systems requires new techniques for model-based design that takes under consideration both software and hardware constraints and enables verification at early stages of development. In this context, EAST-ADL was developed as a domain specific language dedicated to modeling of functional-, software-, and hardware- architecture of automotive embedded systems. This language represents a convenient abstraction when reasoning about the system functionality and supports modeling of relevant extra-functional properties, like timing and resource usage. By providing formal semantics to the EAST-ADL language, as a network of priced timed automata, it becomes possible to reason about feasibility and worst-case resource consumption of the embedded components. In this paper, we show how to analyze such embedded systems modeled in EAST-ADL by using statistical model-checking. We report our experiences from applying this approach to an industrial Brake-by-Wire system.

  • 26.
    Marinescu, Raluca
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Seceleanu, Cristina
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Sundmark, Daniel
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Automatic Test Generation for Energy Consumption of Embedded Systems Modeled in EAST-ADL2017In: Proceedings - 10th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2017, 2017, p. 69-76Conference paper (Refereed)
    Abstract [en]

    Testing using architectural design models is intended to determine if the realized system meets its specification, and works as a whole in terms of computational components and their interactions. The growing complexity of embedded systems requires new techniques that are able to support testing of extra-functional requirements, like energy usage of components and systems, which is very necessary in order to obtain valid implementations. In this paper, we show how architectural models described in the EAST-ADL architectural language can also be used for testing the energy consumption of embedded systems, after transforming them into networks of formal models called priced timed automata. Assuming an EAST-ADL model annotated with energy consumption information, we show how to automatically generate energy-aware test cases based on statistical model checking (SMC) of the resulting network of priced timed automata. We automate the generation of executable test cases with UPPAAL SMC, using a test strategy based on several random simulation runs of the system. By seeding the original formal model with a set of energy-consumption related faults, we are able to carry out fault detection analysis. We apply this technique on a Brake-by-Wire system from the automotive domain, and evaluate it in terms of efficiency and model fault detection. 

  • 27.
    Marinescu, Raluca
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Filipovikj, Predrag
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Larsson, Jonatan
    Mälardalen University.
    Seceleanu, Cristina
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    An Energy-aware Mutation Testing Framework for EAST-ADL Architectural Models2017In: 29th Nordic Workshop on Programming Theory NWPT'17, Turku, Finland , Finland: TUCS Lecture Notes , 2017, p. 40-43Conference paper (Refereed)
    Abstract [en]

    Early design artifacts of embedded systems, such as architectural models, represent convenient abstractions for reasoning about a system’s structure and functionality. One such example is the Electronic Architecture and Software Tools-Architecture Description Language (EAST-ADL), a domain-specific architectural language that targets the automotive industry. EAST-ADL is used to represent both hardware and software elements, as well as related extra-functional information (e.g., timing properties, triggering information, resource consumption). Testing architectural models is an important activity in engineering large-scale industrial systems, which sparks a growing research interest. Modern embedded systems, such as autonomous vehicles and robots, have low-energy computing demands, making testing for energy usage increasingly important. Nevertheless, testing resource-aware properties of architectural models has received less attention than the functional testing of such models. In our previous work, we have outlined a method for testing energy consumption in embedded systems using manually created faults based on statistical model checking of a priced formal system model. In this paper, we extend our previous work by showing how mutation testing] can be used to generate and select test cases based on the concept of energy-aware mutants– small syntactic modifications in the architectural model, intended to mimic real energy faults. Test cases that can distinguish a certain behavior from its mutations are sensitive to changes in the model, and hence considered to be good at detecting faults. The main contributions of this paper are: (i) an approach for creating energy-related mutants for EAST-ADL architectural models, (ii) a method for overcoming the equivalent mutant problem (i.e., the problem of finding a test case which can distinguish the observable behavior of a mutant from the original one), (iii) a test generation approach based on UPPAAL Statistical Model Checker (SMC), and (iv) a test selection criteria based on mutation analysis using our MATS tool.

  • 28.
    Strandberg, Per Erik
    et al.
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Westermo Reserarch and Development AB.
    Enoiu, Eduard Paul
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Afzal, Wasif
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Daniel, Sundmark
    Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
    Feldt, Robert
    Information Flow in Software Testing: An Interview Study with Embedded Software Engineering PractitionersManuscript (preprint) (Other academic)
    Abstract [en]

    Activities in software testing is a challenge for companies that develop embedded systems where multiple functional teams and technologically difficult tasks are common. This study aims at exploring the information flow in software testing, the perceived challenges and good approaches, for a more effective information flow. We conducted semi-structured interviews with twelve software practitioners working at five organizations in the embedded software industry in Sweden. The interviews were analyzed by means of thematic analysis. The data was classified into six themes that affect the information flow in software testing: testing and troubleshooting, communication, processes, technology, artifacts and organization. We further identified a number of challenges such as poor feedback and understanding exactly what has been tested; and approaches such as fast feedback as well as custom automated test reporting; to achieve an improved information flow. Our results indicate that there are many opportunities to improve this information flow: a first mitigation step is to better understand the challenges and approaches. Future work is needed to realize this in practice, for example to shorten feedback cycles between roles, as well as enhance exploration and visualization of test results

1 - 28 of 28
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf