Code review has long been a core practice in collaborative software engineering. In this research, we explore how practitioners reflect on code review today and what changes they anticipate in the near future. We then discuss the potential long-term risks of these anticipated changes for the evolution of code review and its role in collaborative software engineering.

Context: Code review has become a core practice in collaborative software engineering, helping ensure code quality, detecting potential bugs, and supporting communication among developers. Prior research has shown that code review practices differ between production and test code, suggesting that established code review guidelines may fall short in the context of test and GUI-based test code. Particularly, GUI-based testing lacks adequate support during the code review process. To address this, we proposed a set of code review guidelines specifically designed for reviewing GUI-based test files, which, however, have not yet been empirically evaluated, limiting their practical relevance. 

Objective: This study empirically assesses the extent to which code review comments on GUI-based tests align (explicitly or implicitly) with the concerns captured by the proposed guidelines, and uses the findings to refine the guideline set.

Method: To achieve this, we sampled code review comments discussing GUI-based test files across 100 open-source projects and manually analyzed 1000 pull requests to determine to what extent the reviewers' comments align with the proposed guidelines.

Results: Review comments aligned with the proposed guidelines in 808 of 1000 pull requests. We found empirical evidence for 25 of the 33 guidelines. The most frequently observed guideline concerns the correct use of testing techniques and exception handling, particularly regarding locators, explicit waits, and timeout behavior.

Conclusion: The observed alignment suggests that the proposed guidelines capture concerns articulated in practice, indicating practical relevance for GUI-based test reviews. This represents an initial step towards providing empirical validation of the proposed guidelines, highlighting their potential value in enhancing the quality of GUI-based test reviews.

Context: Review of software artifacts, such as source or test code, is a common practice in industrial practice. However, although review guidelines are available for source and low-level test code, for GUI-based testing artifacts, such guidelines are missing. Objective: The goal of this work is to define a set of guidelines from literature about production and test code, that can be mapped to GUI-based testing artifacts. Method: A systematic literature review is conducted, using white and gray literature to identify guidelines for source and test code. These synthesized guidelines are then mapped, through examples, to create actionable, and applicable, guidelines for GUI-based testing artifacts. Results: The results of the study are 33 guidelines, summarized in nine guideline categories, that are successfully mapped as applicable to GUI-based testing artifacts. Of the collected literature, only 10 sources contained test-specific code review guidelines. These guideline categories are: perform automated checks, use checklists, provide context information, utilize metrics, ensure readability, visualize changes, reduce complexity, check conformity with the requirements and follow design principles and patterns. Conclusion: This pivotal set of guidelines provides an industrial contribution in filling the gap of general guidelines for review of GUI-based testing artifacts. Additionally, this work highlights, from an academic perspective, the need for future research in this area to also develop guidelines for other specific aspects of GUI-based testing practice, and to take into account other facets of the review process not covered by this work, such as reviewer selection. © 2023 The Author(s)

Context: Code reviews are a well-established practice to ensure code quality, identify potential bugs, promote knowledge sharing, and maintain coding standards within a team or organization.

Objective: This study aims to investigate the specific practices, challenges, and information needs encountered when reviewing GUI-based testing files for web applications, which remain poorly understood.

Method: We conducted a qualitative interview study with 14 software testing professionals from six different companies to explore the distinct aspects of reviewing GUI-based testing code. 

Results: We identified four practices, six challenges, and four information needs related to reviewing GUI-based testing files. The foremost challenge is the validation of GUI-based tests under review. Furthermore, challenges concerning levels of abstraction and test robustness were not addressed in related studies . Additionally, participants proposed six potential improvements for tools and practices to better support the code review process. Notably, the absence of standardized practices and the need to run tests locally were common themes across participants.

Conclusion: The code review process for GUI-based testing differs from that of production code, highlighting the need for practices and tools tailored specifically to the unique demands of GUI-based testing.