Digitala Vetenskapliga Arkivet

Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
On the Industrial Applicability of Augmented Testing: An Empirical Study
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för programvaruteknik. SERL, SWE.
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för programvaruteknik. SERL, SWE.
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för programvaruteknik. SERL, SWE.ORCID-id: 0000-0002-5179-4205
2020 (engelsk)Inngår i: Proceedings - 2020 IEEE 13th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2020, Institute of Electrical and Electronics Engineers Inc. , 2020, s. 364-371, artikkel-id 9155725Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Testing applications with graphical user Interfaces (GUI) is an important but also a time-consuming task in practice. Tools and frameworks for GUI test automation can make the test execution more efficient and lower the manual labor required for regression testing. However, the test scripts used for automated GUI-based testing still require a substantial development effort and are often reported as sensitive to change, leading to frequent and costly maintenance. The efficiency of development, maintenance, and evolution of such tests are thereby dependent on the readability of scripts and the ease-of-use of test tools/frameworks in which the test scripts are defined. To address these shortcomings in existing state-of-practice techniques, a novel technique referred to as Augmented Testing (AT) has been proposed. AT is defined as testing the System Under Test (SUT) through an Augmented GUI that superimposes information on top of the SUT GUI. The Augmented GUI can provide the user with hints, test data, or other support while also observing and recording the tester's interactions. For this study, a prototype tool, called Scout, has been used that adheres to the AT concept that is evaluated in an industrial empirical study. In the evaluation, quasi-experiments and questionnaire surveys are performed in two workshops, with 12 practitioners from two Swedish companies (Ericsson and Inceptive). Results show that Scout can be used to create equivalent test cases faster, with statistical significance, than creating automated scripts in two popular state-of-practice tools. The study concludes that AT has cost-value benefits, applies to industrial-grade software, and overcomes several deficiencies of state-of-practice GUI testing technologies in terms of ease-of-use. © 2020 IEEE.

sted, utgiver, år, opplag, sider
Institute of Electrical and Electronics Engineers Inc. , 2020. s. 364-371, artikkel-id 9155725
Emneord [en]
Augmented Testing, Industrial Case Study, System Testing, Test Automation, Automation, Graphical user interfaces, Surveys, Testing, Verification, Automated scripts, Empirical studies, Graphical user interfaces (GUI), Questionnaire surveys, Regression testing, Statistical significance, System under test, Time-consuming tasks, Software testing
HSV kategori
Identifikatorer
URN: urn:nbn:se:bth-20530DOI: 10.1109/ICSTW50294.2020.00065ISI: 000620795100048ISBN: 9781728110752 (tryckt)OAI: oai:DiVA.org:bth-20530DiVA, id: diva2:1474683
Konferanse
13th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2020, Porto, Portugal, 23 March 2020 through 27 March 2020
Ingår i projekt
SERT- Software Engineering ReThought, Knowledge FoundationTilgjengelig fra: 2020-10-09 Laget: 2020-10-09 Sist oppdatert: 2025-09-30bibliografisk kontrollert
Inngår i avhandling
1. On overcoming challenges with GUI-based test automation
Åpne denne publikasjonen i ny fane eller vindu >>On overcoming challenges with GUI-based test automation
2024 (engelsk)Doktoravhandling, med artikler (Annet vitenskapelig)
Abstract [en]

Background: Automated testing is widely used in modern software development to check if the software, including its graphical user interface (GUI), meets the expectations in terms of quality and functionality. GUI-based test automation, like other automation, aims to save time and money compared to manual testing without reducing the software quality. While automation has successfully reduced costs for other types of testing (e.g., unit-or integration tests), GUI-based testing has faced technical challenges, some of which have lingered for over a decade. 

Objective: This thesis work aims to contribute to the software engineering body of knowledge by (1) identifying the main challenges in GUI-based test automation and (2) finding technical solutions to mitigate some of the main challenges. One such challenge is to reliably identify GUI elements during test execution to prevent unnecessary repairs. Another problem is the demand for test automation and programming skills when designing stable automated tests at scale. 

Method: We conducted several studies by adopting a multi-methodological approach. First, we performed a systematic literature review to identify the main challenges in GUI-based test automation, followed by multiple studies that propose and evaluate novel approaches to mitigate the main challenges. 

Results: Our first contribution is mapping the challenges in GUI-based test automation reported in academic literature. We mapped the main challenges (i.e. most reported) on a timeline and classified them as essential or accidental. This classification is valuable since future research can focus on the main challenges that we are more likely to mitigate using a technical solution (i.e., accidental). Our second contribution is several approaches that explore novel concepts or advance state-of-the-art techniques to mitigate some of the main accidental challenges. Testing an application through an augmented layer (Augmented Testing) can reduce the demand for test automation and programming skills and mitigate the challenges of creating and maintaining model based tests. Our proposed approach for locating web elements (Similo) can increase the robustness of automated test execution. 

Conclusion: Our results provide alternative approaches and concepts that can mitigate some of the main accidental challenges in GUI-based test automation. With a more robust test execution and tool support for test modeling, we can help reduce the manual labor spent on creating and maintaining automated GUI-based tests. With a reduced cost of automation, testers can focus more on other tasks like requirements, test design, and exploratory testing.

sted, utgiver, år, opplag, sider
Karlskrona: Blekinge Tekniska Högskola, 2024. s. 215
Serie
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 2
Emneord
GUI Testing, Test Automation, Augmented Testing, Test Case Robustness, Web Element Locators, Large Language Models
HSV kategori
Forskningsprogram
Programvaruteknik
Identifikatorer
urn:nbn:se:bth-25638 (URN)978-91-7295-473-1 (ISBN)
Disputas
2024-02-06, J1630, Campus Karlskrona, 13:00 (engelsk)
Opponent
Veileder
Tilgjengelig fra: 2023-11-28 Laget: 2023-11-22 Sist oppdatert: 2025-09-30bibliografisk kontrollert

Open Access i DiVA

fulltext(1676 kB)473 nedlastinger
Filinformasjon
Fil FULLTEXT01.pdfFilstørrelse 1676 kBChecksum SHA-512
d08111c4f5c4aa6b2d79aa9c7b76e5aca50e02b1f94ddbba4471f4d2e9a1a9c380d46b959c0d55d961d4f5b66c6195ea7c61172ffa2453e93634e7382e5f08cd
Type fulltextMimetype application/pdf

Andre lenker

Forlagets fulltekst

Søk i DiVA

Av forfatter/redaktør
Nass, MichelAlégroth, EmilFeldt, Robert
Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar
Totalt: 473 nedlastinger
Antall nedlastinger er summen av alle nedlastinger av alle fulltekster. Det kan for eksempel være tidligere versjoner som er ikke lenger tilgjengelige

doi
isbn
urn-nbn

Altmetric

doi
isbn
urn-nbn
Totalt: 401 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf