Digitala Vetenskapliga Arkivet

Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
AI or Human? Detecting Machine-Generated Text Through Perplexity
KTH, School of Engineering Sciences (SCI).
KTH, School of Engineering Sciences (SCI).
2025 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesis
Abstract [en]

This thesis investigates whether a large language model can distinguish between AI-generated and human-written texts by using perplexity—a metric that quantifies how predictable a piece of text is to the model itself. The core hypothesis is that AI-generated text, having been created by a language model, should appear more predictable (lower perplexity), while human writing is generally less predictable (higher perplexity). To test this, we use an open-source language model to evaluate both human and AI-generated samples from a range of curated datasets. We then apply a statistical classifier to see how well perplexity alone can separate the two classes.Our findings show that perplexity works well as a detection signal in controlled scenarios—especially when both the generation and evaluation are performed by the same model and within the same style or domain. However, the separation between AI and human writing becomes less clear when we introduce more varied sources, revealing the limitations of relying on perplexity alone.

 

Place, publisher, year, edition, pages
2025.
Series
TRITA-SCI-GRU ; 2025:252
Keywords [en]
LLM, AI-detection, Perplexity
National Category
Mathematical sciences
Identifiers
URN: urn:nbn:se:kth:diva-366582OAI: oai:DiVA.org:kth-366582DiVA, id: diva2:1982607
Subject / course
Mathematical Statistics
Educational program
Master of Science in Engineering - Engineering Mathematics
Supervisors
Examiners
Available from: 2025-07-08 Created: 2025-07-08 Last updated: 2025-07-08Bibliographically approved

Open Access in DiVA

fulltext(1447 kB)1068 downloads
File information
File name FULLTEXT01.pdfFile size 1447 kBChecksum SHA-512
7b654927a4b484e21b4e405d3c67fb5821813ef520cb79bdf66ae8f9d4d89c119ca31de525ed87e6460525f387c1ddfb93354361f6743cceedba26e842c45528
Type fulltextMimetype application/pdf

By organisation
School of Engineering Sciences (SCI)
Mathematical sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 1073 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 830 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf