Digitala Vetenskapliga Arkivet

Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Ethical Tensions in AI-Based Systems
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.ORCID iD: 0000-0003-3812-1492
2025 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis contributes to human-computer interaction (HCI) by exploring how various stakeholders in Swedish public organisations make sense of ethical considerations and negotiate ethical responsibility in the development and use of artificial intelligence (AI)-based systems. 

While high-level ethical frameworks (e.g., guidelines that emphasise principles such as fairness, transparency, and accountability) are intended to guide AI ethics application, prior research reveals that practitioners frequently struggle to translate abstract frameworks into concrete actions within design and use contexts. Responding to calls in HCI for situated, empirical approaches to studying AI ethics in practice, this thesis investigates how stakeholders engage in ethical reasoning through three interconnected dimensions: how they reflect and make sense of ethical considerations, the ethical tensions they encounter when working with AI-based systems, and how ethical responsibility is described and negotiated across AI-based systems’ life cycles.

Drawing on two qualitative case studies combining semi-structured interviews and a multi-stakeholder focus group, the thesis develops an empirically grounded account of stakeholders' ethical reasoning processes. The analysis draws attention to three cross-study themes. First, stakeholders make sense of ethical considerations in situ, shaped by organisational roles, institutional demands, and technological constraints, rather than direct application of abstract frameworks. Second, ethical tensions are not simply obstacles but catalysts that prompt ethical reasoning, surfacing hidden assumptions and conflicts that require stakeholders to renegotiate responsibilities. Third, the negotiation of responsibility is made and remade among actors, shifting across the AI-based system’s life cycle in response to tensions and contextual constraints.

Together, these findings show that ethical reasoning in public sector AI work is best understood as contextual, relational, and evolving – taking shape through the interplay of sense-making, handling tension, and doing responsibilities. In doing so, this thesis invites more reflective (embracing tensions as triggers for ethical reflection), relational (attuned to the shared and negotiated nature of responsibility), and practice-oriented (grounded in the situated ways stakeholders make sense of ethical considerations in everyday work) approaches to Responsible AI.

Place, publisher, year, edition, pages
Stockholm: Department of Computer and Systems Sciences, Stockholm University , 2025. , p. 97
Series
Report Series / Department of Computer & Systems Sciences, ISSN 1101-8526 ; 25-008
Keywords [en]
HCI, AI Ethics, Public Sector, Ethical Tensions, Ethical Responsibility
National Category
Human Computer Interaction
Research subject
Information Society
Identifiers
URN: urn:nbn:se:su:diva-245384ISBN: 978-91-8107-358-4 (print)ISBN: 978-91-8107-359-1 (electronic)OAI: oai:DiVA.org:su-245384DiVA, id: diva2:1988700
Public defence
2025-10-09, Lilla Hörsalen, NOD-huset, Borgarfjordsgatan, 12, Kista, 13:00 (English)
Opponent
Supervisors
Available from: 2025-09-16 Created: 2025-08-13 Last updated: 2025-09-26Bibliographically approved
List of papers
1. Exploring tensions in Responsible AI in practice. An Interview Study on AI practices in and for Swedish Public Organizations
Open this publication in new window or tab >>Exploring tensions in Responsible AI in practice. An Interview Study on AI practices in and for Swedish Public Organizations
2022 (English)In: Scandinavian Journal of Information Systems, ISSN 0905-0167, E-ISSN 1901-0990, Vol. 34, no 2, p. 199-232, article id 6Article in journal (Refereed) Published
Abstract [en]

The increasing use of Artificial Intelligence (AI) systems has sparked discussions regarding developing ethically responsible technology. Consequently, various organizations have released high-level AI ethics frameworks to assist in AI design. However, we still know too little about how AI ethics principles are perceived and work in practice, especially in public organizations. This study examines how AI practitioners perceive ethical issues in their work concerning AI design and how they interpret and put them into practice. We conducted an empirical study consisting of semi-structured qualitative interviews with AI practitioners working in or for public organizations. Taking the lens provided by the “In-Action Ethics” framework and previous studies on ethical tensions, we analyzed practitioners’ interpretations of AI ethics principles and their application in practice. We found tensions between practitioners’ interpretation of ethical principles in their work and ‘ethos tensions.’ In this vein, we argue that understanding the different tensions that can occur in practice and how they are tackled is key to studying ethics in practice. Understanding how AI practitioners perceive and apply ethical principles is necessary for practical ethics to contribute toward an empirically grounded, Responsible AI.

Keywords
Responsible AI, AI ethics in practice, empirical studies on ethics, ethos tensions, AI practitioners.
National Category
Information Systems
Research subject
Computer and Systems Sciences
Identifiers
urn:nbn:se:su:diva-213553 (URN)
Available from: 2023-01-09 Created: 2023-01-09 Last updated: 2025-08-13Bibliographically approved
2. Who Should Act? Distancing and Vulnerability in Technology Practitioners' Accounts of Ethical Responsibility
Open this publication in new window or tab >>Who Should Act? Distancing and Vulnerability in Technology Practitioners' Accounts of Ethical Responsibility
2024 (English)In: Proceedings of the ACM on Human-Computer Interaction (PACMHCI), E-ISSN 2573-0142, Vol. 8, no CSCW1, article id 157Article in journal (Refereed) Published
Abstract [en]

Attending to emotion can shed light on why recognizing an ethical issue and taking responsibility for it can be so demanding. To examine emotions related to taking or not taking responsibility for ethical action, we conducted a semi-structured interview study with 23 individuals working in interaction design and developing AI systems in Scandinavian countries. Through a thematic analysis of how participants attribute ethical responsibility, we identify three ethical stances, that is, discursive approaches to answering the question 'who should act': an individualized I-stance ("the responsibility is mine"), a collective we-stance ("the responsibility is ours"), and a distanced they-stance ("the responsibility is someone else's"). Further, we introduce the concepts of distancing and vulnerability to analyze the emotion work that these three ethical stances place on technology practitioners in situations of low- and high-scale technology development, where they have more or less control over the outcomes of their work. We show how the we- and they-stances let technology practitioners distance themselves from the results of their activity, while the I-stance makes them more vulnerable to emotional and material risks. By illustrating the emotional dimensions involved in recognizing ethical issues and embracing responsibility, our study contributes to the field of Ethics in Practice. We argue that emotions play a pivotal role in technology practitioners' decision-making process, influencing their choices to either take action or refrain from doing so.

Keywords
ethics, emotion, ethical stance, vulnerability, distancing, responsibility, ethics in practice
National Category
Human Computer Interaction
Research subject
Computer and Systems Sciences
Identifiers
urn:nbn:se:su:diva-232987 (URN)10.1145/3637434 (DOI)2-s2.0-85185217502 (Scopus ID)
Available from: 2024-08-29 Created: 2024-08-29 Last updated: 2025-08-13Bibliographically approved
3. Promises and breakages of automated grading systems: a qualitative study in computer science education
Open this publication in new window or tab >>Promises and breakages of automated grading systems: a qualitative study in computer science education
Show others...
2025 (English)In: Education Inquiry, E-ISSN 2000-4508, p. 1-22Article in journal (Refereed) Published
Abstract [en]

Automated grading systems (AGSs) have gained attention for their potential to streamline assessment in higher education. However, their integration into university assessment practice poses challenges, particularly for teachers in computer science seeking to balance their workload while ensuring an adequate and fair assessment of students’ programming skills and knowledge. The present study focuses on individuals with expertise in developing, using, and researching AGSs in higher education, whom we refer to as “AGS experts”. Through semi-structured interviews, we examine how the AGSs they engage with impact their work and assessment practices in computer science education. Drawing on the concept of breakages, we argue that while AGS experts invest time and effort in developing these systems, enticed by the promises of more efficient workload management and improved assessment practices, the actual use may introduce tensions leading to breakages disrupting assessment practices. Our findings illustrate the complexities and the potential impact the deployment of AGS brings to assessment practices within a public university setting and discuss the implications for future research. 

National Category
Educational Sciences
Identifiers
urn:nbn:se:su:diva-241097 (URN)10.1080/20004508.2025.2464996 (DOI)001420361500001 ()2-s2.0-85219706969 (Scopus ID)
Available from: 2025-03-21 Created: 2025-03-21 Last updated: 2026-03-18Bibliographically approved
4. Doing Responsibilities with Automated Grading Systems: An Empirical Multi-Stakeholder Exploration
Open this publication in new window or tab >>Doing Responsibilities with Automated Grading Systems: An Empirical Multi-Stakeholder Exploration
2024 (English)In: NordiCHI '24: Proceedings of the 13th Nordic Conference on Human-Computer Interaction, Association for Computing Machinery (ACM) , 2024, article id 1Conference paper, Published paper (Refereed)
Abstract [en]

Automated Grading Systems (AGSs) are increasingly used in higher education assessment practices, raising issues about the responsibilities of the various stakeholders involved both in their design and use. This study explores how teachers, students, exam administrators, and developers of AGSs perceive and enact responsibilities around such systems. Drawing on a focus group and interview data, we applied Fuchsberger and Frauenberger’s [27] notion of Doing Responsibilities as an analytical lens. This notion, framing responsibility as shared among human and nonhuman actors (e.g., technologies and data), has guided our analysis of how responsibilities are continuously configured and enacted in university assessment practices. The findings illustrate the stakeholders’ perceived and enacted responsibilities at different phases, contributing to the HCI literature on Responsible AI and AGSs by presenting a practical application of the ‘Doing Responsibilities’ framework before, during and after design. We discuss how the findings enrich this notion, emphasising the importance of engaging with nonhumans, considering regulatory aspects of responsibility, and addressing relational tensions within automation.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2024
Keywords
Autograding, Automated Grading Systems, Design, Ethics, Multistakeholder, Responsibility
National Category
Information Systems Human Computer Interaction
Identifiers
urn:nbn:se:su:diva-237201 (URN)10.1145/3679318.3685334 (DOI)001332352300001 ()2-s2.0-85206564238 (Scopus ID)979-8-4007-0966-1 (ISBN)
Conference
NordiCHI 2024: Nordic Conference on Human-Computer Interaction 13-16 October 2024, Uppsala, Sweden.
Available from: 2025-01-09 Created: 2025-01-09 Last updated: 2025-08-13Bibliographically approved

Open Access in DiVA

Ethical Tensions in AI-Based Systems(2998 kB)900 downloads
File information
File name FULLTEXT01.pdfFile size 2998 kBChecksum SHA-512
e8a17bbaeb136343ad95e25d99ae0bf87f0b43a527281cbdf2b5227dd0bdf6b48efcb9a8b2946f5fb79a21d482214b3f865729ac908b6ed32b4221b0080b6a14
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Figueras, Clàudia
By organisation
Department of Computer and Systems Sciences
Human Computer Interaction

Search outside of DiVA

GoogleGoogle Scholar
Total: 905 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 4345 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf