Digitala Vetenskapliga Arkivet

Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Why many challenges with GUI test automation (will) remain
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för programvaruteknik.ORCID-id: 0000-0002-8569-2290
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för programvaruteknik.ORCID-id: 0000-0001-7526-3727
Chalmers University of Technology, SWE.
2021 (engelsk)Inngår i: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 138, artikkel-id 106625Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

Context: Automated testing is ubiquitous in modern software development and used to verify requirement conformance on all levels of system abstraction, including the system's graphical user interface (GUI). GUI-based test automation, like other automation, aims to reduce the cost and time for testing compared to alternative, manual approaches. Automation has been successful in reducing costs for other forms of testing (like unit- or integration testing) in industrial practice. However, we have not yet seen the same convincing results for automated GUI-based testing, which has instead been associated with multiple technical challenges. Furthermore, the software industry has struggled with some of these challenges for more than a decade with what seems like only limited progress. Objective: This systematic literature review takes a longitudinal perspective on GUI test automation challenges by identifying them and then investigating why the field has been unable to mitigate them for so many years. Method: The review is based on a final set of 49 publications, all reporting empirical evidence from practice or industrial studies. Statements from the publications are synthesized, based on a thematic coding, into 24 challenges related to GUI test automation. Results: The most reported challenges were mapped chronologically and further analyzed to determine how they and their proposed solutions have evolved over time. This chronological mapping of reported challenges shows that four of them have existed for almost two decades. Conclusion: Based on the analysis, we discuss why the key challenges with GUI-based test automation are still present and why some will likely remain in the future. For others, we discuss possible ways of how the challenges can be addressed. Further research should focus on finding solutions to the identified technical challenges with GUI-based test automation that can be resolved or mitigated. However, in parallel, we also need to acknowledge and try to overcome non-technical challenges. © 2021

sted, utgiver, år, opplag, sider
Elsevier B.V. , 2021. Vol. 138, artikkel-id 106625
Emneord [en]
GUI testing, System testing, Systematic literature review, Test automation, Automation, Cost reduction, Graphical user interfaces, Software design, Testing, Automated testing, Finding solutions, Graphical user interfaces (GUI), Industrial practices, Software industry, Technical challenges, Integration testing
HSV kategori
Identifikatorer
URN: urn:nbn:se:bth-21481DOI: 10.1016/j.infsof.2021.106625ISI: 000672531500005Scopus ID: 2-s2.0-85106242621OAI: oai:DiVA.org:bth-21481DiVA, id: diva2:1560451
Ingår i projekt
SERT- Software Engineering ReThought, Knowledge Foundation
Forskningsfinansiär
Knowledge FoundationTilgjengelig fra: 2021-06-04 Laget: 2021-06-04 Sist oppdatert: 2025-09-30bibliografisk kontrollert
Inngår i avhandling
1. On overcoming challenges with GUI-based test automation
Åpne denne publikasjonen i ny fane eller vindu >>On overcoming challenges with GUI-based test automation
2024 (engelsk)Doktoravhandling, med artikler (Annet vitenskapelig)
Abstract [en]

Background: Automated testing is widely used in modern software development to check if the software, including its graphical user interface (GUI), meets the expectations in terms of quality and functionality. GUI-based test automation, like other automation, aims to save time and money compared to manual testing without reducing the software quality. While automation has successfully reduced costs for other types of testing (e.g., unit-or integration tests), GUI-based testing has faced technical challenges, some of which have lingered for over a decade. 

Objective: This thesis work aims to contribute to the software engineering body of knowledge by (1) identifying the main challenges in GUI-based test automation and (2) finding technical solutions to mitigate some of the main challenges. One such challenge is to reliably identify GUI elements during test execution to prevent unnecessary repairs. Another problem is the demand for test automation and programming skills when designing stable automated tests at scale. 

Method: We conducted several studies by adopting a multi-methodological approach. First, we performed a systematic literature review to identify the main challenges in GUI-based test automation, followed by multiple studies that propose and evaluate novel approaches to mitigate the main challenges. 

Results: Our first contribution is mapping the challenges in GUI-based test automation reported in academic literature. We mapped the main challenges (i.e. most reported) on a timeline and classified them as essential or accidental. This classification is valuable since future research can focus on the main challenges that we are more likely to mitigate using a technical solution (i.e., accidental). Our second contribution is several approaches that explore novel concepts or advance state-of-the-art techniques to mitigate some of the main accidental challenges. Testing an application through an augmented layer (Augmented Testing) can reduce the demand for test automation and programming skills and mitigate the challenges of creating and maintaining model based tests. Our proposed approach for locating web elements (Similo) can increase the robustness of automated test execution. 

Conclusion: Our results provide alternative approaches and concepts that can mitigate some of the main accidental challenges in GUI-based test automation. With a more robust test execution and tool support for test modeling, we can help reduce the manual labor spent on creating and maintaining automated GUI-based tests. With a reduced cost of automation, testers can focus more on other tasks like requirements, test design, and exploratory testing.

sted, utgiver, år, opplag, sider
Karlskrona: Blekinge Tekniska Högskola, 2024. s. 215
Serie
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 2
Emneord
GUI Testing, Test Automation, Augmented Testing, Test Case Robustness, Web Element Locators, Large Language Models
HSV kategori
Forskningsprogram
Programvaruteknik
Identifikatorer
urn:nbn:se:bth-25638 (URN)978-91-7295-473-1 (ISBN)
Disputas
2024-02-06, J1630, Campus Karlskrona, 13:00 (engelsk)
Opponent
Veileder
Tilgjengelig fra: 2023-11-28 Laget: 2023-11-22 Sist oppdatert: 2025-09-30bibliografisk kontrollert

Open Access i DiVA

Fulltekst mangler i DiVA

Andre lenker

Forlagets fulltekstScopus

Søk i DiVA

Av forfatter/redaktør
Nass, MichelAlégroth, Emil
Av organisasjonen
I samme tidsskrift
Information and Software Technology

Søk utenfor DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric

doi
urn-nbn
Totalt: 868 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf