Large-scale software systems often experience architectural degradation, affecting maintainability, scalability, and quality. To investigate this, we conducted focus groups with senior practitioners across three large organizations. Our analysis revealed four core challenge areas: managing dependencies, ownership and organizational barriers, balancing agility with stability and cost, as well as documentation drift. These findings offer practical insights for mitigating architectural degradation in complex environments. 

Context: Traceability is a key quality attribute of artifacts that are used in knowledge-intensive tasks and supports software engineers in producing higher-quality software. Despite its clear benefits, traceability is often neglected in practice due to challenges such as granularity of traces, lack of a common artifact structure, and unclear responsibility. The Taxonomic Trace Links (TTL) approach connects source and target artifacts through a domain-specific taxonomy, aiming to address these common traceability challenges.

Objective: In this study, we empirically evaluate TTL in an industrial setting to identify its strengths and weaknesses for real-world adoption.

Method: We conducted a mixed-methods study at Ericsson involving one of its software products. Quantitative and qualitative data were collected across two traceability use cases. We established trace links between 463 business use cases, 64 test cases, and 277 ISO-standard requirements. Additionally, we held three focus group sessions with practitioners.

Results: We identified two practically relevant scenarios where traceability is required and evaluated TTL in each. Overall, practitioners found TTL to be a useful solution for identifying trace links with reasonable effort. However, developing a domain-specific taxonomy and managing heterogeneous artifact structures were noted as significant challenges.

Conclusion: TTL is a promising approach that can be adopted in practice and enables traceability use cases. However, TTL are not a replacement for traditional trace links, but complementary to enable more traceability use cases, and encourage early trace links creation.

Context: Generative AI (GenAI) is increasingly adopted in software development for tasks such as document generation, data analysis, and code generation.However, evaluating the quality of GenAI applications becomes challenging, as traditional quality measurements may not be fully applicable.

Objective: In this study, we explore how practitioners evaluate the quality of GenAI applications and investigate quality evaluation techniques.

Method: We conducted a multi-case study in three industrial projects from software development companies.We examined four GenAI application domains: document generation, data analysis and insight generation, customer service, and code generation.Data were collected through three workshops and 23 semi-structured interviews with industrial practitioners.

Results: We identified fourteen GenAI use cases and 28 metrics currently used to evaluate the quality of GenAI applications' outputs.We synthesized the identified metrics' usage patterns and challenges based on the collected data.

Conclusions: This study presents practical insights into using metrics to measure GenAI-based system qualities in real industrial settings.Our findings indicate that practitioners use custom-built and context‑specific metrics; combining these with academic metrics can strengthen GenAI system quality evaluation.

In a large-scale software development product development organization, we found that most developers, although experienced, were lacking architectural knowledge of the specific developed product.

As a remedy, we evaluated whether we could stimulate learning the product architecture by conducting training in how to use the product’s distributed tracing platform, built on the OpenTelemetry standard and the open-source Jaeger Tracing visualization tool.

We planned and participated in a training event, where parts of the organization explored, using experiential learning, how to set up and use tracing to troubleshoot a realistic fault scenario we prepared. Respondents were asked to rate the tool according to the Technology Adoption Model (TAM), and responses were collected on Likert-type scales, analyzed, and summarized using a Bayesian workflow.

Even as tool usage post-training was low, respondents still had a positive attitude toward using the tool, valued the experiential training, and expressed a strong intent to use the tool for program comprehension.

Background. The Software Bill of Materials (SBOM) is a machine-readable list of all the software dependencies included in a software. SBOM emerged as way to assist securing the software supply chain. However, despite mandates from governments to use SBOM, research on this artifact is still in its early stages.

Aims. We want to understand the current state of SBOM in open-source projects, focusing specifically on policy-driven SBOMs—i.e., SBOM created to achieve security goals, such as enhancing project transparency and ensuring compliance, rather than being used as fixtures for tools or artificially generated for benchmarking or academic research purposes.

Method. We performed a mining software repository study to collect and carefully select 620 SBOM files hosted on GitHub. We analyzed the information reported in policy-driven SBOMs and the vulnerabilities associated with the declared dependencies by means of descriptive statistics.

Results. We show that only 0.56% of popular GitHub repositories contain policy-driven SBOM. The declared dependencies contain 2,202 unique vulnerabilities, while 22% of them do not report licensing information.

Conclusion. Our findings provide insights for SBOM usage to support security assessment and licensing. 

Over the last decade, software organizations have increasingly adopted microservices to effectively deal with evolving software systems, frequent demands for new features, and changing technologies. However, microservices are not a silver bullet; their success depends on the specific context and needs of each organization. Therefore, tracking the evolution of architectural complexity indicators is crucial for effective architectural governance and decision-making. In this paper, we explore the relationship between architectural complexity indicators and their evolution, specifically declared dependencies, API endpoints, inter-service communications, size, and technical debt. We used the static source code analysis methods along with SonarQube to measure architectural complexity, collecting data on all indicators over the past two and a half years. Our findings indicate that architectural complexity consistently grows, even within microservices. Most importantly, these indicators co-evolve, making the overall architecture more complicated than expected. Additionally, all complexity indicators grow rapidly when services are small and still evolving. The insights gained from this study can assist organizations in effectively managing their microservices, highlighting when they might be most prone to architectural degradation. 

Background: Software Development organisations tend to organise the development of software-intensive products and services as a constellation of components meant to be developed and maintained by independent, autonomous teams. However, the maintenance and evolution of said products and services require team collaboration and coordination. This collaboration and coordination overhead piles on top of teams’ workload, often hindering teams’ throughput and lead time. Objectives: This paper aims to discuss how the use of pull request data can help identify congested teams when the arrival of new tasks exceeds the team’s ability to close them. To do so, we have conducted an empirical study in a software development organisation developing a large-scale product, to try to characterise congested teams and the characteristics of the code reviews they are involved in. Method: We have conducted a case study to start exploring how code review data can help us model team congestion, and understand whether the features of the code-review network, or the team type (platform vs product), can have a major impact on team congestion. Results: The results show that teams seem to experience varying levels of congestion based on pull request activity, with some indicating potential congestion. However, increased PR accumulation did not consistently lead to longer lead times, as seen in some teams where high PR backlogs did not significantly impact delivery cadence. Conclusions: Our findings suggest that while PR data can indicate potential congestion, its impact on lead time varies across teams. Both technical factors and unobserved contextual elements shape congestion. Deeper insights require combining repository metrics with qualitative inputs. 

The growing complexity, the ever-ending demands for new features, and the need to become faster to remain competitive force software development organizations to rethink their development and value delivery practices. While continuous delivery has become more popular, it still relies mainly on internal metrics, ad-hoc data, and expert opinions. As a result, software organizations stumble to find the balance between improving internal system quality and delivering external value. In fact, understanding and measuring customer value is on itself essential. In this PhD project, we aim for a better understanding of customer value and develop measurement instruments to be integrated with internal perspectives to drive proactive and continuous internal improvement while delivering relevant customer value. 

Context: The evidence for the prevalence of test smells at the unit testing level has relied on the accuracy of detection tools, which have seen intense research in the last two decades. The Eager Test smell, one of the most prevalent, is often identified using simplified detection rules that practitioners find inadequate.

Objective: We aim to improve the rules for detecting the Eager Test smell.

Method: We reviewed the literature on test smells to analyze the definitions and detection rules of the Eager Test smell. We proposed a novel, unambiguous definition of the test smell and a heuristic to address the limitations of the existing rules. We evaluated our heuristic against existing detection rules by manually applying it to 300 unit test cases in Java.

Results: Our review identified 56 relevant studies. We found that inadequate interpretations of original definitions of the Eager Test smell led to imprecise detection rules, resulting in a high level of disagreement in detection outcomes. Also, our heuristic detected patterns of eager and non-eager tests that existing rules missed.

Conclusion: Our heuristic captures the essence of the Eager Test smell more precisely; hence, it may address practitioners’ concerns regarding the adequacy of existing detection rules.

It is commonly accepted that the quality of requirements specifications impacts subsequent software engineering activities. However, we still lack empirical evidence to support organizations in deciding whether their requirements are good enough or impede subsequent activities. We aim to contribute empirical evidence to the effect that requirements quality defects have on a software engineering activity that depends on this requirement. We conduct a controlled experiment in which 25 participants from industry and university generate domain models from four natural language requirements containing different quality defects. We evaluate the resulting models using both frequentist and Bayesian data analysis. Contrary to our expectations, our results show that the use of passive voice only has a minor impact on the resulting domain models. The use of ambiguous pronouns, however, shows a strong effect on various properties of the resulting domain models. Most notably, ambiguous pronouns lead to incorrect associations in domain models. Despite being equally advised against by literature and frequentist methods, the Bayesian data analysis shows that the two investigated quality defects have vastly different impacts on software engineering activities and, hence, deserve different levels of attention. Our employed method can be further utilized by researchers to improve reliable, detailed empirical evidence on requirements quality. © The Author(s) 2024.