Digitala Vetenskapliga Arkivet

Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
AUTOMATING GAP DETECTION INTECHNICAL DOCUMENTATION: A reranking approach in RAG
Umeå University, Faculty of Science and Technology, Department of Computing Science.
2026 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesis
Abstract [en]

Manual compliance checking between formal requirements and technical documentation is a time consuming task and prone to errors. Formal requirements are important statements that a product must satisfy, drawn from standards used in integrated logistics support. Technical documentation is the product manual an engineer checks against those requirements. This thesis investigates whether adding a reranking step to a retrieval augmented generation pipeline improves the accuracy of automated gap detection between the two. Two pipelines were compared, a standard RAG baseline and an extended RAG pipeline with Coherere ranking model. Both were evaluated against 15 requirements in English, drawn from MIL-STD-1388-2B, a United States military standard for logistic support analysis, and S3000L, an international standard published by ASD. The requirements were checked against a Swedish operator manual for the Hydronic M-IIheating system. The evaluation used context recall, faithfulness, verdict accuracy, and a domain expert review. The differences were marginal, but the reranking pipeline scored higher across all measures. Verdict accuracy increased by 6.7%, and the domain expert rated the reranking outputs higher on correctness (4.60 to 4.20 on a 1-5 scale). Reranking reduced retrieval noise and improved there levance of the context passed to the large language model, which led to more accurate compliance verdicts.

Place, publisher, year, edition, pages
2026.
Series
UMNAD ; 1661
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:umu:diva-256307OAI: oai:DiVA.org:umu-256307DiVA, id: diva2:2082499
External cooperation
Combitech
Examiners
Available from: 2026-07-01 Created: 2026-06-30 Last updated: 2026-07-01Bibliographically approved

Open Access in DiVA

No full text in DiVA

By organisation
Department of Computing Science
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 552 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf