Digitala Vetenskapliga Arkivet

Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Improving Web Element Localization by Using a Large Language Model
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för programvaruteknik. (SERT)ORCID-id: 0000-0002-8569-2290
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för programvaruteknik.ORCID-id: 0000-0001-7526-3727
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för programvaruteknik.ORCID-id: 0000-0002-5179-4205
2024 (Engelska)Ingår i: Software testing, verification & reliability, ISSN 0960-0833, E-ISSN 1099-1689, Vol. 34, nr 7Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Web-based test automation heavily relies on accurately finding web elements. Traditional methods compare attributes but don't grasp the context and meaning of elements and words. The emergence of Large Language Models (LLMs) like GPT-4, which can show human-like reasoning abilities on some tasks, offers new opportunities for software engineering and web element localization. This paper introduces and evaluates VON Similo LLM, an enhanced web element localization approach. Using an LLM, it selects the most likely web element from the top-ranked ones identified by the existing VON Similo method, ideally aiming to get closer to human-like selection accuracy. An experimental study was conducted using 804 web element pairs from 48 real-world web applications. We measured the number of correctly identified elements as well as the execution times, comparing the effectiveness and efficiency of VON Similo LLM against the baseline algorithm. In addition, motivations from the LLM were recorded and analyzed for all instances where the original approach failed to find the right web element. VON Similo LLM demonstrated improved performance, reducing failed localizations from 70 to 39 (out of 804), a 44 percent reduction. Despite its slower execution time and additional costs of using the GPT-4 model, the LLMs human-like reasoning showed promise in enhancing web element localization. LLM technology can enhance web element identification in GUI test automation, reducing false positives and potentially lowering maintenance costs. However, further research is necessary to fully understand LLMs capabilities, limitations, and practical use in GUI testing.

Ort, förlag, år, upplaga, sidor
John Wiley & Sons, 2024. Vol. 34, nr 7
Nyckelord [en]
GUI Testing, Test Automation, Test Case Robustness, Web Element Locators, Large Language Models
Nationell ämneskategori
Datorsystem
Forskningsämne
Programvaruteknik
Identifikatorer
URN: urn:nbn:se:bth-25637DOI: 10.1002/stvr.1893ISI: 001290853000001Scopus ID: 2-s2.0-85201296537OAI: oai:DiVA.org:bth-25637DiVA, id: diva2:1814031
Ingår i projekt
SERT- Software Engineering ReThought, KK-stiftelsen
Forskningsfinansiär
KK-stiftelsen, 20180010Tillgänglig från: 2023-11-22 Skapad: 2023-11-22 Senast uppdaterad: 2025-09-30Bibliografiskt granskad
Ingår i avhandling
1. On overcoming challenges with GUI-based test automation
Öppna denna publikation i ny flik eller fönster >>On overcoming challenges with GUI-based test automation
2024 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

Background: Automated testing is widely used in modern software development to check if the software, including its graphical user interface (GUI), meets the expectations in terms of quality and functionality. GUI-based test automation, like other automation, aims to save time and money compared to manual testing without reducing the software quality. While automation has successfully reduced costs for other types of testing (e.g., unit-or integration tests), GUI-based testing has faced technical challenges, some of which have lingered for over a decade. 

Objective: This thesis work aims to contribute to the software engineering body of knowledge by (1) identifying the main challenges in GUI-based test automation and (2) finding technical solutions to mitigate some of the main challenges. One such challenge is to reliably identify GUI elements during test execution to prevent unnecessary repairs. Another problem is the demand for test automation and programming skills when designing stable automated tests at scale. 

Method: We conducted several studies by adopting a multi-methodological approach. First, we performed a systematic literature review to identify the main challenges in GUI-based test automation, followed by multiple studies that propose and evaluate novel approaches to mitigate the main challenges. 

Results: Our first contribution is mapping the challenges in GUI-based test automation reported in academic literature. We mapped the main challenges (i.e. most reported) on a timeline and classified them as essential or accidental. This classification is valuable since future research can focus on the main challenges that we are more likely to mitigate using a technical solution (i.e., accidental). Our second contribution is several approaches that explore novel concepts or advance state-of-the-art techniques to mitigate some of the main accidental challenges. Testing an application through an augmented layer (Augmented Testing) can reduce the demand for test automation and programming skills and mitigate the challenges of creating and maintaining model based tests. Our proposed approach for locating web elements (Similo) can increase the robustness of automated test execution. 

Conclusion: Our results provide alternative approaches and concepts that can mitigate some of the main accidental challenges in GUI-based test automation. With a more robust test execution and tool support for test modeling, we can help reduce the manual labor spent on creating and maintaining automated GUI-based tests. With a reduced cost of automation, testers can focus more on other tasks like requirements, test design, and exploratory testing.

Ort, förlag, år, upplaga, sidor
Karlskrona: Blekinge Tekniska Högskola, 2024. s. 215
Serie
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 2
Nyckelord
GUI Testing, Test Automation, Augmented Testing, Test Case Robustness, Web Element Locators, Large Language Models
Nationell ämneskategori
Programvaruteknik
Forskningsämne
Programvaruteknik
Identifikatorer
urn:nbn:se:bth-25638 (URN)978-91-7295-473-1 (ISBN)
Disputation
2024-02-06, J1630, Campus Karlskrona, 13:00 (Engelska)
Opponent
Handledare
Tillgänglig från: 2023-11-28 Skapad: 2023-11-22 Senast uppdaterad: 2025-09-30Bibliografiskt granskad

Open Access i DiVA

fulltext(2247 kB)97 nedladdningar
Filinformation
Filnamn FULLTEXT02.pdfFilstorlek 2247 kBChecksumma SHA-512
5007f1e7988f37b66f4bbb20d52dcfa250ee17bdbb4bc2f0e785ef7b3e04b0ad455b6a2d09a4c0144a2d294a6c7387c19abc3b6ff071bee56ec8c3f969839d78
Typ fulltextMimetyp application/pdf

Övriga länkar

Förlagets fulltextScopus

Sök vidare i DiVA

Av författaren/redaktören
Nass, MichelAlégroth, EmilFeldt, Robert
Av organisationen
Institutionen för programvaruteknik
I samma tidskrift
Software testing, verification & reliability
Datorsystem

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 97 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

doi
urn-nbn

Altmetricpoäng

doi
urn-nbn
Totalt: 347 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf