Change search
ReferencesLink to record
Permanent link

Direct link
Experiment to evaluate an Innovative Test Framework: Automation of non-functional testing
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
2015 (English)Independent thesis Advanced level (degree of Master (Two Years)), 80 credits / 120 HE creditsStudent thesis
Abstract [en]

Context. Performance testing, among other types of non-functional testing, is necessary to assess software quality. Most often, manual approach is employed to test a system for its performance. This approach has several setbacks. The existing body of knowledge lacks empirical evidence on automation of non-functional testing and is largely focused on functional testing.

Objectives. The objective of the present study is to evaluate a test framework that automates performance testing. A large-scale distributed project is selected as the context to achieve this objective. The rationale for choosing such a project is that the proposed test framework was designed with an intention to adapt and tailor according to any project’s characteristics.

Methods. An experiment was conducted with 15 participants at Ericsson R&D department, India to evaluate an automated test framework. Repeated measures design with counter balancing method was used to understand the accuracy and time taken while using the test framework. To assess the ease-of-use of the proposed framework, a questionnaire was distributed among the experiment participants. Statistical techniques were used to accept or reject the hypothesis. The data analysis was performed using Microsoft Excel.

Results. It is observed that the automated test framework is superior to the traditional manual approach. There is a significant reduction in the average time taken to run a test case. Further, the number of errors resulting in a typical testing process is minimized. Also, the time spent by a tester during the actual test is phenomenally reduced while using the automated approach. Finally, as perceived by software testers, the automated approach is easier to use when compared to the manual test approach.

Conclusions. It can be concluded that automation of non-functional testing will result in overall reduction in project costs and improves quality of software tested. This will address important performance aspects such as system availability, durability and uptime. It was observed that it is not sufficient if the software meets the functional requirements, but is also necessary to conform to the non-functional requirements.

Place, publisher, year, edition, pages
2015. , 53 p.
Keyword [en]
performance testing, automation and software testing
National Category
Software Engineering
URN: urn:nbn:se:bth-10940OAI: diva2:868223
External cooperation
Ericsson Research and Development Department, Gurgaon
Subject / course
PA2534 Master's Thesis (120 credits) in Software Engineering
Educational program
PAAXA Master of Science Programme in Software Engineering
2015-09-21, J1630, Blekinge Tekniska Högskola, 371 79 Karlskrona, Blekinge, 09:00 (English)
Available from: 2015-11-11 Created: 2015-11-08 Last updated: 2015-11-11Bibliographically approved

Open Access in DiVA

fulltext(1428 kB)105 downloads
File information
File name FULLTEXT01.pdfFile size 1428 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Eada, Priyanudeep
By organisation
Department of Software Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 105 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 319 hits
ReferencesLink to record
Permanent link

Direct link