Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Evaluation of the Transformer Model for Abstractive Text Summarization
KTH, School of Electrical Engineering and Computer Science (EECS).
2019 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesisAlternative title
Evaluering av Transformermodellen för abstraktiva textsammanfattningar (Swedish)
Abstract [en]

Being able to generate summaries automatically could speed up the spread and retention of information and potentially increase productivity in several fields.

Using RNN-based encoder-decoder models with attention have been successful on a variety of language-related tasks such as automatic summarization but also in the field of machine translation. Lately, the Transformer model has been shown to outperform RNN-based models with attention in the relatedfield of machine translation.

This study compares the Transformer model to a LSTM-based encoderdecoder model with attention on the task of abstractive summarization. Evaluation is done both automatically, using ROUGE score, as well as using human evaluators to estimate the grammar and readability of the generated summaries. The results show that the Transformer model produces better summaries both in terms of ROUGE score and when evaluated with human evaluators.

Abstract [sv]

Att automatiskt kunna generera sammanfattningar ökar möjligheten att snabbt kunna sprida och ta del av information vilket potentiellt kan leda till produktivitetsökningar inom en mängd fält.

RNN-baserade enkoder-dekodermodeller med attention har visat sig vara effektiva inom många språkrelaterade områden såsom automatiskt genererade sammanfattningar men också inom exempelvis automatisk översättning. På senare tid har Transformermodellen överträffat RNN-baserade enkoderdekodermodeller med attention inom det närliggande området automatiska översättningar. Denna uppsats jämför Transformermodellen med en LSTMbaserad enkoder-dekodermodell med attention både genom att använda det automatiska måttet ROUGE, men också genom att jämföra läsbarhet och grammatik i de automatgenererade sammanfattningarna med hjälp av mänskliga utvärderare.

Resultaten visar att Transformermodellen genererar bättre sammanfattningar både utvärderat med ROUGE och när de mänskliga utvärderarna används.

Place, publisher, year, edition, pages
2019. , p. 38
Series
TRITA-EECS-EX ; 2019:563
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:kth:diva-263325OAI: oai:DiVA.org:kth-263325DiVA, id: diva2:1368180
Supervisors
Examiners
Available from: 2019-11-18 Created: 2019-11-06 Last updated: 2019-11-18Bibliographically approved

Open Access in DiVA

fulltext(1522 kB)7 downloads
File information
File name FULLTEXT01.pdfFile size 1522 kBChecksum SHA-512
15259d2d1281a971aeefca00106efaad8e92e2447bec2c5d6cd89cce8383ec1a5f52f1aafd7a444b6663c62374b1205a5d54db1e15343ff5fc405d06896e75ec
Type fulltextMimetype application/pdf

By organisation
School of Electrical Engineering and Computer Science (EECS)
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 7 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 10 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf