Large-scale software systems often experience architectural degradation, affecting maintainability, scalability, and quality. To investigate this, we conducted focus groups with senior practitioners across three large organizations. Our analysis revealed four core challenge areas: managing dependencies, ownership and organizational barriers, balancing agility with stability and cost, as well as documentation drift. These findings offer practical insights for mitigating architectural degradation in complex environments. 

In a large-scale software development product development organization, we found that most developers, although experienced, were lacking architectural knowledge of the specific developed product.

As a remedy, we evaluated whether we could stimulate learning the product architecture by conducting training in how to use the product’s distributed tracing platform, built on the OpenTelemetry standard and the open-source Jaeger Tracing visualization tool.

We planned and participated in a training event, where parts of the organization explored, using experiential learning, how to set up and use tracing to troubleshoot a realistic fault scenario we prepared. Respondents were asked to rate the tool according to the Technology Adoption Model (TAM), and responses were collected on Likert-type scales, analyzed, and summarized using a Bayesian workflow.

Even as tool usage post-training was low, respondents still had a positive attitude toward using the tool, valued the experiential training, and expressed a strong intent to use the tool for program comprehension.

Background. The Software Bill of Materials (SBOM) is a machine-readable list of all the software dependencies included in a software. SBOM emerged as way to assist securing the software supply chain. However, despite mandates from governments to use SBOM, research on this artifact is still in its early stages.

Aims. We want to understand the current state of SBOM in open-source projects, focusing specifically on policy-driven SBOMs—i.e., SBOM created to achieve security goals, such as enhancing project transparency and ensuring compliance, rather than being used as fixtures for tools or artificially generated for benchmarking or academic research purposes.

Method. We performed a mining software repository study to collect and carefully select 620 SBOM files hosted on GitHub. We analyzed the information reported in policy-driven SBOMs and the vulnerabilities associated with the declared dependencies by means of descriptive statistics.

Results. We show that only 0.56% of popular GitHub repositories contain policy-driven SBOM. The declared dependencies contain 2,202 unique vulnerabilities, while 22% of them do not report licensing information.

Conclusion. Our findings provide insights for SBOM usage to support security assessment and licensing. 

Over the last decade, software organizations have increasingly adopted microservices to effectively deal with evolving software systems, frequent demands for new features, and changing technologies. However, microservices are not a silver bullet; their success depends on the specific context and needs of each organization. Therefore, tracking the evolution of architectural complexity indicators is crucial for effective architectural governance and decision-making. In this paper, we explore the relationship between architectural complexity indicators and their evolution, specifically declared dependencies, API endpoints, inter-service communications, size, and technical debt. We used the static source code analysis methods along with SonarQube to measure architectural complexity, collecting data on all indicators over the past two and a half years. Our findings indicate that architectural complexity consistently grows, even within microservices. Most importantly, these indicators co-evolve, making the overall architecture more complicated than expected. Additionally, all complexity indicators grow rapidly when services are small and still evolving. The insights gained from this study can assist organizations in effectively managing their microservices, highlighting when they might be most prone to architectural degradation. 

Background: Software Development organisations tend to organise the development of software-intensive products and services as a constellation of components meant to be developed and maintained by independent, autonomous teams. However, the maintenance and evolution of said products and services require team collaboration and coordination. This collaboration and coordination overhead piles on top of teams’ workload, often hindering teams’ throughput and lead time. Objectives: This paper aims to discuss how the use of pull request data can help identify congested teams when the arrival of new tasks exceeds the team’s ability to close them. To do so, we have conducted an empirical study in a software development organisation developing a large-scale product, to try to characterise congested teams and the characteristics of the code reviews they are involved in. Method: We have conducted a case study to start exploring how code review data can help us model team congestion, and understand whether the features of the code-review network, or the team type (platform vs product), can have a major impact on team congestion. Results: The results show that teams seem to experience varying levels of congestion based on pull request activity, with some indicating potential congestion. However, increased PR accumulation did not consistently lead to longer lead times, as seen in some teams where high PR backlogs did not significantly impact delivery cadence. Conclusions: Our findings suggest that while PR data can indicate potential congestion, its impact on lead time varies across teams. Both technical factors and unobserved contextual elements shape congestion. Deeper insights require combining repository metrics with qualitative inputs. 

The growing complexity, the ever-ending demands for new features, and the need to become faster to remain competitive force software development organizations to rethink their development and value delivery practices. While continuous delivery has become more popular, it still relies mainly on internal metrics, ad-hoc data, and expert opinions. As a result, software organizations stumble to find the balance between improving internal system quality and delivering external value. In fact, understanding and measuring customer value is on itself essential. In this PhD project, we aim for a better understanding of customer value and develop measurement instruments to be integrated with internal perspectives to drive proactive and continuous internal improvement while delivering relevant customer value. 

Context: The evidence for the prevalence of test smells at the unit testing level has relied on the accuracy of detection tools, which have seen intense research in the last two decades. The Eager Test smell, one of the most prevalent, is often identified using simplified detection rules that practitioners find inadequate.

Objective: We aim to improve the rules for detecting the Eager Test smell.

Method: We reviewed the literature on test smells to analyze the definitions and detection rules of the Eager Test smell. We proposed a novel, unambiguous definition of the test smell and a heuristic to address the limitations of the existing rules. We evaluated our heuristic against existing detection rules by manually applying it to 300 unit test cases in Java.

Results: Our review identified 56 relevant studies. We found that inadequate interpretations of original definitions of the Eager Test smell led to imprecise detection rules, resulting in a high level of disagreement in detection outcomes. Also, our heuristic detected patterns of eager and non-eager tests that existing rules missed.

Conclusion: Our heuristic captures the essence of the Eager Test smell more precisely; hence, it may address practitioners’ concerns regarding the adequacy of existing detection rules.

It is commonly accepted that the quality of requirements specifications impacts subsequent software engineering activities. However, we still lack empirical evidence to support organizations in deciding whether their requirements are good enough or impede subsequent activities. We aim to contribute empirical evidence to the effect that requirements quality defects have on a software engineering activity that depends on this requirement. We conduct a controlled experiment in which 25 participants from industry and university generate domain models from four natural language requirements containing different quality defects. We evaluate the resulting models using both frequentist and Bayesian data analysis. Contrary to our expectations, our results show that the use of passive voice only has a minor impact on the resulting domain models. The use of ambiguous pronouns, however, shows a strong effect on various properties of the resulting domain models. Most notably, ambiguous pronouns lead to incorrect associations in domain models. Despite being equally advised against by literature and frequentist methods, the Bayesian data analysis shows that the two investigated quality defects have vastly different impacts on software engineering activities and, hence, deserve different levels of attention. Our employed method can be further utilized by researchers to improve reliable, detailed empirical evidence on requirements quality. © The Author(s) 2024.

Software Bills of Material (SBOMs) are becoming a consolidated-and often enforced by governmental regulations-way to describe software composition. However, based on recent studies, SBOMs suffer from limited support for their consumption and lack information beyond simple dependencies, especially regarding software vulnerabilities. This paper reports the results of a preliminary study in which we augmented SBOMs of 40 open-source projects with information about Common Vulnerabilities and Exposures (CVE) exposed by project dependencies. Our augmented SBOMs have been evaluated by submitting pull requests and by asking project owners to answer a survey. Although, in most cases, augmented SBOMs were not directly accepted because owners required a continuous SBOM update, the received feedback shows the usefulness of the suggested SBOM augmentation.

Background: Modern software systems are often too large and complex for an individual developer to fully oversee, making it difficult to understand the implications of changes. Therefore, most collaborative software projects rely on code review as communication network to foster asynchronous discussions about changes before they are merged. Although prior qualitative studies have revealed that practitioners view code review as a communication network, no formal theory or empirical validation exists. Without formalization and confirmatory evidence, the theory remains uncertain, limiting its credibility, practical relevance, and future development.

Objective: In this thesis, our objective is to (1) formalize the theory of code review as a communication network, (2) empirically evaluate the theory across varied perspectives, contexts, and conditions by quantifying the capability of code review to diffuse information among its participants, (3) demonstrate its practical relevance by applying the theory to the domain of tax compliance in collaborative software engineering, and (4) examine how the role of code review as a communication network for collaborative software engineering may evolve in the future.

Methods: To formalize the theory of code review as a communication network, we developed and validated a simulation model that operationalizes its core propositions about information diffusion among participants. To empirically evaluate the theory, we employed two complementary research approaches. First, we used the simulation model to conduct in silico experiments with closed-source code review systems from Microsoft, Spotify, and Trivago, as well as open-source code review systems from Android, Visual Studio Code, and React, to estimate the upper bound of information diffusion in code review. Second, through an observational study, we quantified the diffusion of information in code review across social, organizational, and architectural boundaries at Spotify. To demonstrate the practical relevance of the theory, we analyzed the code review system of a multinational enterprise as a communication network to reveal the latent collaboration structure among developers across borders, which is taxable. To explore the future of code review as a communication network, we conducted a questionnaire survey with 92 practitioners to gather their expectations and discuss how these anticipated changes may reshape our understanding of code review.

Results: By formalizing the theory of code review as a communication network modelled as a time-varying hypergraph, we were able to empirically demonstrate that traditional time-agnostic models substantially overestimate information diffusion in code review. Throughout our empirical studies, we found substential evidence supporting the theory of code review as a communication network: We confirmed that code review is capable of diffusing information quickly and widely among participants, even at a large scale. We also observed extensive information diffusion across social, organizational, and architectural boundaries at Spotify corroborating our theory. However, we also found that information diffusion patterns in open-source code review systems differ significantly, suggesting that findings from open-source environments may not directly apply to closed-source contexts. Through applying the theory of code review as a communication network in the domain of tax compliance, we were able to uncover the significant and previously unrecognized tax risks associated with collaborative software engineering within multinational enterprises. While practitioners consider code review also in the future a core practice in collaborative software engineering, we identify a potential risk that generative AI may undermine code review’s role as a human communication network.

Conclusion: Our work on understanding code review as a communication network contributes not only to theory-driven, empirical software engineering research but also lays the groundwork for practical applications, particularly in the context of tax compliance. Future research is needed to explore the evolving role of code review as a communication network.