Effectiveness of Exploratory Testing, An empirical scrutiny of the challenges and factors affecting the defect detection efficiency
Independent thesis Advanced level (degree of Master (Two Years))Student thesisAlternative title
Effektiviteten hos utforskande testning, En empirisk granskning av de utmaningar och faktorer som påverkar effektiviteten fel upptäckt (Swedish)
Context: Software testing is an integral part of software development life cycle. To improve the quality of software there are different testing approaches practiced over the years. Traditionally software testing is carried out by following approach focusing on prior test design. While exploratory testing is an approach to test software where the tester does not require to follow a specific test design. But rather, exploratory testing should facilitate the tester in testing the complete system comprehensively. Exploratory testing is seen by some, as a way to conduct simultaneous learning, test design and execution of tests simultaneously. While others point to exploratory testing enabling constant evolution of tests in an easy manner. Objectives: In this study we have investigated the field of exploratory testing in literature and industry to understand its perception and application. Further among the stated claims by practitioners, we selected defect detection efficiency and effectiveness claim for empirical validation through an experiment and survey. Methods: In this study, a systematic literature review, interview, experiment and survey are conducted. In the systematic review a number of article sources are used, including IEEE Xplore, ACM Digital Library, Engineering village, Springer Link, Google Scholar and Books database. The systematic review also includes the gray literature published by the practitioners. The selection of studies was done using two-phase and tollgate approach. A total of 47 references were selected as primary studies. Eight semi-structures interviews were conducted with industry practitioners. Experiment had total 4 iterations and 70 subjects. The subjects were selected from industry and academia. The experimental design used was one factor with two interventions and one response variable. Results: Based on our findings from literature review and interviews, the understanding of exploratory testing has improved over the period but still lacks empirical investigation. The results drawn from experimental and survey data shows that exploratory testing proved effective and efficient in finding more critical bugs in limited time. Conclusions: We conclude that exploratory testing has a lot of potential and much more to offer to testing industry. But more empirical investigation and true facts and figures are required to motivate the testing industry to adapt it. We have reported a number of advantages, disadvantages, challenges and factors in this study. We further investigated the claims stated by the ET practitioners through an experiment and survey. The statistical tests were conducted on the collected data to draw meaningful results. We found statistical significance difference in number of true defects found. Using exploratory testing approach testers found far more defects than test case based testing. Although, there was no statistical significance difference between the two approaches for false defects.
Slutsatser: Vi anser att det experimentella tester har stor potential och mycket mer att erbjuda testning industrin. Men mer empirisk undersökning och sann fakta och siffror är skyldiga att motivera testning industrin att anpassa den. Vi har rapporterat en rad fördelar, nackdelar, utmaningar och faktorer i denna studie. Vi undersökte vidare fordringar anges av ET utövare genom ett experiment och undersökning. De statistiska test genomfördes på insamlade data för att dra meningsfulla resultat. Vi fann statistisk signifikans skillnaden i antalet sann fel som upptäcks. Använda utforskande testning strategi testare fann långt fler fel än testfall baserat testning. Även om det inte fanns någon statistisk signifikans skillnad mellan de två synsätten för falska defekter.
Place, publisher, year, edition, pages
2010. , 85 p.
Exploratory testing, ET, systematic review, defect detection, efficiency, effectiveness
IdentifiersURN: urn:nbn:se:bth-5456Local ID: oai:bth.se:arkivex822127791597B625C12577EA004D36B2OAI: oai:DiVA.org:bth-5456DiVA: diva2:832837
Torkar, Dr. Richard
0046 73 651 80482015-04-222010-11-292015-06-30Bibliographically approved