Large Language Models (LLMs) offer promising capabilities for information retrieval and processing. However, the LLM deployment for querying proprietary enterprise data poses unique challenges, particularly for companies with strict data security policies. This study shares our experience in setting up a secure LLM environment within a FinTech company and utilizing it for enterprise information retrieval while adhering to data privacy protocols. 

We conducted three workshops and 30 interviews with industrial engineers to gather data and requirements. The interviews further enriched the insights collected from the workshops. We report the steps to deploy an LLM solution in an industrial sandboxed environment and lessons learned from the experience. These lessons contain LLM configuration (e.g., chunk_size and top_k settings), local document ingestion, and evaluating LLM outputs.

Our lessons learned serve as a practical guide for practitioners seeking to use private data with LLMs to achieve better usability, improve user experiences, or explore new business opportunities. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) are recognized to have significant effects on industry and business dynamics, not least because of its impact on the preconditions for entrepreneurship. There is yet a lack of knowledge of GenAI as a theme in entrepreneurship research. This paper presents a systematic literature review aimed at identifying and analysing the evolving landscape of research on the effects of GenAI on entrepreneurship. We analyse 83 peer-reviewed articles obtained from leading academic databases: Web of Science and Scopus. Using natural language processing and unsupervised machine learning techniques with TF-IDF vectorization, Principal Component Analysis (PCA), and hierarchical clustering, five major thematic clusters are identified: (1) Digital Transformation & Behavioural Models, (2) GenAI-Enhanced Education & Learning Systems, (3) Sustainable Innovation & Strategic AI Impact, (4) Business Models & Market Trends, and (5) Data-Driven Technological Trends in Entrepreneurship. Based on the review, we discuss future research directions, gaps in the current literature as well as ethical concerns raised in the literature. We pinpoint the need for more “macro-level” research on GenAI and LLMs as external enablers for entrepreneurship and research on effective regulatory frameworks that facilitate business experimentation, innovation and further technology development.

Artificial Intelligence can make manufacturing organisations more effective and efficient, but it is not clear which AI tasks hold the greatest potential. Make-to-order manufacturers must constantly adapt to customers’ unique and rapidly changing needs, and therefore have different challenges than make-to-stock manufacturers. Our ambition is to develop an AI-enabled software system to support manufacturing organisations in improving their processes. To this end, we first seek to understand the data and technology requirements for key AI-enabled tasks in a make-to-order setting and determine the level of performance and explainability needed to address them. We perform a multiple case study of five make-to-order packaging manufacturers, interviewing personnel from sales, production, and supply chain to identify and prioritise operational challenges suitable for AI approaches. Demand forecasting emerges as the most important task, followed by predictive maintenance, quality inspection, complex decision risk estimation, and production planning. Participants emphasise the importance of explainable techniques to ensure trust in the systems. The results highlight a need for a greater control of the production process and a better understanding of customer needs. Although most of the tasks could be solved with current techniques, some, such as intermittent demand forecasting and complex decision risk estimation, would require further development. The study clarifies the potential of AI-enabled systems in make-to-order manufacturing and outlines the steps required to realise it.

Context: Generative Artificial Intelligence (GenAI) and the use of Large Language Models (LLMs) have revolutionized tasks that previously required significant human effort, which has attracted considerable interest from industry stakeholders. This growing interest has accelerated the integration of AI models into various industrial applications. However, the model integration introduces challenges to product quality, as conventional quality measuring methods may fail to assess GenAI systems. Consequently, evaluation techniques for GenAI systems need to be adapted and refined. Examining the current state and applicability of evaluation techniques for the GenAI system outputs is essential.

Objective: This study aims to explore the current metrics, methods, and processes for assessing the outputs of GenAI systems and the potential of risky outputs.

Method: We performed a snowballing literature review to identify metrics, evaluation methods, and evaluation processes from 43 selected papers.

Results: We identified 28 metrics and mapped these metrics to four quality characteristics defined by the ISO/IEC 25023 standard for software systems. Additionally, we discovered three types of evaluation methods to measure the quality of system outputs and a three-step process to assess faulty system outputs. Based on these insights, we suggested a five-step framework for measuring system quality while utilizing GenAI models.

Conclusion: Our findings present a mapping that visualizes candidate metrics to be selected for measuring quality characteristics of GenAI systems, accompanied by step-by-step processes to assist practitioners in conducting quality assessments. 

Context: Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes.

Objective: This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research.

Method: We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures.

Results: Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure.

Conclusions: The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems. 

Context: Software engineering (SE) artifacts and documents, such as requirements specifications, user stories, test cases, and concepts of operations (ConOps), are typically written in natural language, making their manipulation challenging. Natural Language Processing (NLP) is a viable solution for managing these tasks.

Objective: To conduct a systematic literature review to explore the current use of NLP in SE artifacts and tasks, supplementedby a tertiary study focusing on the emerging role of Large Language Models (LLMs) in software engineering re-search.

Method: We searched digital libraries for relevant papers and applied inclusion and exclusion criteria to filter the primary studies. We then analyzed NLP techniques applied to SE documents and examined their usage in this context. Our research methodology followed Kitchenham and Charters’ guidelines. Additionally, we conducted a tertiary study to synthesize findings from existing systematic literature reviews and surveys specifically addressing LLMs in software engineering.

Results: We selected 60 primary studies to identify the most common methods for NLP pipelines, feature extraction, language models, and machine learning algorithms used in SE. We also assessed the purposes of these methods, their benefits for SE, their difficulty, and their contribution to SE advancement. The tertiary study revealed a rapid proliferation of LLM-focused research, with comprehensive reviews documenting exponential growth in publications and widespread adoption across diverse SE tasks.

Conclusion: Requirements are the most frequently addressed artifacts using NLP techniques, with preprocessing and part-of-speech (POS) tagging being widely used. There is a notable increase in the use of large language models for various SE tasks, such as requirements elicitation, source code generation, bug fixing, and software testing. The tertiary study confirms that LLMs represent a pivotal shift in the research landscape, warranting dedicated investigation to understand their transformative impact on NLP applications in software engineering.

Context: As the diversity and complexity of regulations affecting Software-Intensive Products and Services (SIPS) is increasing, software engineers need to address the growing regulatory scrutiny. We argue that, as with any other non-negotiable requirements, SIPS compliance should be addressed early in SIPS engineering—i.e., during requirements engineering (RE).

Objectives: In the conditions of the expanding regulatory landscape, existing research offers scattered insights into regulatory compliance of SIPS. This study addresses the pressing need for a structured overview of the state of the art in software RE and its contribution to regulatory compliance of SIPS.

Method: We conducted a systematic mapping study to provide an overview of the current state of research regarding challenges, principles, and practices for regulatory compliance of SIPS related to RE. We focused on the role of RE and its contribution to other SIPS lifecycle process areas. We retrieved 6914 studies published from 2017 (January 1) until 2023 (December 31) from four academic databases, which we filtered down to 280 relevant primary studies.

Results: We identified and categorized the RE-related challenges in regulatory compliance of SIPS and their potential connection to six types of principles and practices addressing challenges. We found that about 13.6% of the primary studies considered the involvement of both software engineers and legal experts in developing principles and practices. About 20.7% of primary studies considered RE in connection to other process areas. Most primary studies focused on a few popular regulation fields (privacy, quality) and application domains (healthcare, software development, avionics). Our results suggest that there can be differences in terms of challenges and involvement of stakeholders across different fields of regulation.

Conclusion: Our findings highlight the need for an in-depth investigation of stakeholders’ roles, relationships between process areas, and specific challenges for distinct regulatory fields to guide research and practice. 

Recent studies have shown that many software developers do not have sufficient knowledge and understanding of how to develop a privacy-friendly system. This may become a challenge in developing systems complying with data protection laws. To address this issue, we investigated the factors that influence developers' decision-making when developing privacy-sensitive systems.

We conducted an empirical study by means of a survey with 109 practitioners. Our data analysis is based on the principles of social cognitive theory, which includes personal, behavioral, and external environmental factors.

We identified six personal, five behavioral, and five external environment factors that affect how developers make decisions regarding privacy, including confusion between privacy and security and reliance on informal practices and organizational support gaps. These findings contribute to understanding how practitioners and companies consider privacy, showing improvements in formal training and structured support over previous studies yet highlighting persistent challenges in consistent privacy integration. 

Organisations are becoming concerned with effectively dealing with privacy-related requirements. Existing Requirements Engineering methods based on structured natural language suffer from several limitations both in eliciting and specifying privacy requirements. In our previous study, we proposed a structured natural-language approach called the “Privacy Criteria Method” (PCM), which demonstrates potential advantages over user stories. Our goal is to present a PCM evaluation that focused on the opinions of software practitioners from different companies on PCM’s ability to support the specification of privacy requirements and the quality of the privacy requirements specifications produced by these software practitioners. We conducted a multiple case study to evaluate PCM in four different industrial contexts. We gathered and analysed the opinions of 21 practitioners on PCM usage regarding Coverage, Applicability, Usefulness, and Scalability. Moreover, we assessed the syntactic and semantic quality of the PCM artifacts produced by these practitioners. PCM can aid developers in elaborating requirements specifications focused on privacy with good quality. The practitioners found PCM to be useful for their companies’ development processes. PCM is considered a promising method for specifying privacy requirements. Some slight extensions of PCM may be required to tailor the method to the characteristics of the company. © The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2024.

Adversarial Machine Learning (AML) discusses the act of attacking and defending Machine Learning (ML) Models, an essential building block of Artificial Intelligence (AI). ML is applied in many software-intensive products and services and introduces new opportunities and security challenges. AI and ML will gain even more attention from the industry in the future, but threats caused by already-discovered attacks specifically targeting ML models are either overseen, ignored, or mishandled. Current AML research investigates attack and defense scenarios for ML in different industrial settings with a varying degree of maturity with regard to academic rigor and practical relevance. However, to the best of our knowledge, a synthesis of the state of academic rigor and practical relevance is missing. This literature study reviews studies in the area of AML in the context of industry, measuring and analyzing each study's rigor and relevance scores. Overall, all studies scored a high rigor score and a low relevance score, indicating that the studies are thoroughly designed and documented but miss the opportunity to include touch points relatable for practitioners. © 2024 The Author(s)