Change search
ReferencesLink to record
Permanent link

Direct link
Automated Essay Scoring: Scoring Essays in Swedish
Stockholm University, Faculty of Humanities, Department of Linguistics, Computational Linguistics.
2013 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesis
Abstract [en]

Good writing skills are essential in the education system at all levels. However, the evaluation of essays is labor intensive and can entail a subjective bias. Automated Essay Scoring (AES) is a tool that may be able to save teacher time and provide more objective evaluations. There are several successful AES systems for essays in English that are used in large scale tests. Supervised machine learning algorithms are the core component in developing these systems.

In this project four AES systems were developed and evaluated. The AES systems were based on standard supervised machine learning software, i.e., LDAC, SVM with RBF kernel, polynomial kernel and Extremely Randomized Trees. The training data consisted of 1500 high school essays that had been scored by the students' teachers and blind raters. To evaluate the AES systems, the agreement between blind raters' scores and AES scores was compared to agreement between blind raters' and teacher scores. On average, the agreement between blind raters and the AES systems was better than between blind raters and teachers. The AES based on LDAC software had the best agreement with a quadratic weighted kappa value of 0.475. In comparison, the teachers and blind raters had a value of 0.391. However the AES results do not meet the required minimum agreement of a quadratic weighted kappa of 0.7 as defined by the US based nonprofit organization Educational Testing Services.

Abstract [sv]

Jag har utvecklat och utvärderat fyra system för automatisk betygsättning av uppsatser (AES). LDAC, SVM med RBF kernel, SVM med Polynomial kernel och "Extremely Randomized trees" som är standard klassificerarprogramvaror har använts som grunden för att bygga respektivt AES system.

Place, publisher, year, edition, pages
2013. , 51 p.
Keyword [en]
Automated Essay Scoring, Swedish Essays, supervised machine learning
National Category
General Language Studies and Linguistics
URN: urn:nbn:se:su:diva-87266OAI: diva2:602025
Educational program
Available from: 2013-01-31 Created: 2013-01-31 Last updated: 2015-04-10Bibliographically approved

Open Access in DiVA

Automated Essay Scoring: Scoring Essays in Swedish(1234 kB)917 downloads
File information
File name FULLTEXT01.pdfFile size 1234 kBChecksum SHA-512
Type fulltextMimetype application/pdf

By organisation
Computational Linguistics
General Language Studies and Linguistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 917 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 2417 hits
ReferencesLink to record
Permanent link

Direct link