Digitala Vetenskapliga Arkivet

Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Evaluation NMT And LLM Systems For Thai-English And Swedish-English Translation
Umeå universitet, Teknisk-naturvetenskapliga fakulteten, Institutionen för datavetenskap.
2026 (engelsk)Independent thesis Basic level (degree of Bachelor), 10 poäng / 15 hpOppgave
Abstract [en]

This study combines two evaluation methods: three automatic evaluation metrics (BLEU, chrF, and COMET) and LLM-as-a-judge to evaluate translation quality for the low-resource language pair Thai-English. Moreover, this study also investigates translation performance by comparing with the high-resource language pair Swedish-English. Within the scope of the study, four large language model (LLM) systems (Gemini 2.5 Flash, GPT-5.4 mini, Qwen2-72B, Llama 3.3-70B) and one neural machine translation (NMT) system (Google Translate) were used to translate 32 texts drawn from the official governmental and diplomatic domain with a focus on terminological accuracy and semantic preservation. The results show that Gemini 2.5 Flash outperforms Google Translate and all other LLM- based translation systems on Thai-English translation across all three automatic metrics and LLM-as-a-judge. In contrast, Google Translate outperforms Gemini 2.5 Flash across BLEU and chrF on Swedish-English translation. These findings suggest that LLM systems are better suited for translation in low-resource languages and the combination of the evaluation methods provides a complementary evaluation of translation quality.

sted, utgiver, år, opplag, sider
2026. , s. 34
Serie
UMNAD ; 1654
Emneord [en]
Low-resource languages, Thai-English translation, Large language models, Neural machine translation, LLM-as-a-judge, Automatic evaluation metrics
HSV kategori
Identifikatorer
URN: urn:nbn:se:umu:diva-256091OAI: oai:DiVA.org:umu-256091DiVA, id: diva2:2080042
Eksternt samarbeid
Parsd
Utdanningsprogram
Bachelor of Science Programme in Computing Science
Veileder
Examiner
Tilgjengelig fra: 2026-06-26 Laget: 2026-06-26 Sist oppdatert: 2026-06-26bibliografisk kontrollert

Open Access i DiVA

fulltext(2812 kB)814 nedlastinger
Filinformasjon
Fil FULLTEXT01.pdfFilstørrelse 2812 kBChecksum SHA-512
0fcc0098fa2e64d42c74ee95c0d7ad846e3910e866c3c85bcaf8a779d8544b61a4bb5e98b5ca6114f2f33f4db8364eedd1629487d65360480fd62c8029a27767
Type fulltextMimetype application/pdf

Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar
Antall nedlastinger er summen av alle nedlastinger av alle fulltekster. Det kan for eksempel være tidligere versjoner som er ikke lenger tilgjengelige

urn-nbn

Altmetric

urn-nbn
Totalt: 466 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf