Change search
ReferencesLink to record
Permanent link

Direct link
Separating Tweets from Croaks: Detecting Automated Twitter Accounts with Supervised Learning and Synthetically Constructed Training Data
KTH, School of Computer Science and Communication (CSC).
2016 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesisAlternative title
: Automationsdetektion av Twitter-konton med övervakad inlärning och syntetiskt konstruerad träningsmängd (Swedish)
Abstract [en]

In this thesis, we have studied the problem of detecting automated Twitter accounts related to the Ukraine conflict using supervised learning. A striking problem with the collected data set is that it was initially lacking a ground truth. Traditionally, supervised learning approaches rely on manual annotation of training sets, but it incurs tedious work and becomes expensive for large and constantly changing collections.

We present a novel approach to synthetically generate large amounts of labeled Twitter accounts for detection of automation using a rule-based classifier. It significantly reduces the effort and resources needed and speeds up the process of adapting classifiers to changes in the Twitter-domain.

The classifiers were evaluated on a manually annotated test set of 1,000 Twitter accounts. The results show that rule-based classifier by itself achieves a precision of 94.6% and a recall of 52.9%. Furthermore, the results showed that classifiers based on supervised learning could learn from the synthetically generated labels. At best, the these machine learning based classifiers achieved a slightly lower precision of 94.1% compared to the rule-based classifier, but at a significantly better recall of 93.9%

Abstract [sv]

Detta exjobb har undersökt problemet att detektera automatiserade Twitter-konton relaterade till Ukraina-konflikten genom att använda övervakade maskininlärningsmetoder. Ett slående problem med den insamlade datamängden var avsaknaden av träningsexempel. I övervakad maskininlärning brukar man traditionellt manuellt märka upp en träningsmängd. Detta medför dock långtråkigt arbete samt att det blir dyrt förstora och ständigt föränderliga datamängder.

Vi presenterar en ny metod för att syntetiskt generera uppmärkt Twitter-data (klassifieringsetiketter) för detektering av automatiserade konton med en regel-baseradeklassificerare. Metoden medför en signifikant minskning av resurser och anstränging samt snabbar upp processen att anpassa klassificerare till förändringar i Twitter-domänen.

En utvärdering av klassificerare utfördes på en manuellt uppmärkt testmängd bestående av 1,000 Twitter-konton. Resultaten visar att den regelbaserade klassificeraren på egen hand uppnår en precision på 94.6% och en recall på 52.9%. Vidare påvisar resultaten att klassificerare baserat på övervakad maskininlärning kunde lära sig från syntetiskt uppmärkt data. I bästa fall uppnår dessa maskininlärningsbaserade klassificerare en något lägre precision på 94.1%, jämfört med den regelbaserade klassificeraren, men med en betydligt bättre recall på 93.9%.

Place, publisher, year, edition, pages
2016. , 65 p.
Keyword [en]
bot detection, information operations, synthetically constructed training data, social media analysis, classification, learning systems, social networking (online), bot detection, classification performance, classification results, machine learning approaches, military conflicts, semi-automatic, automation
National Category
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-192656OAI: oai:DiVA.org:kth-192656DiVA: diva2:971649
External cooperation
Swedish Defence Research Agency
Subject / course
Computer Science
Educational program
Master of Science in Engineering - Computer Science and Technology
Presentation
2016-06-29, 4523, Lindstedtsvägen 5, Stockholm, 13:01 (English)
Supervisors
Examiners
Available from: 2016-09-19 Created: 2016-09-18 Last updated: 2016-09-19Bibliographically approved

Open Access in DiVA

fulltext(1087 kB)28 downloads
File information
File name FULLTEXT01.pdfFile size 1087 kBChecksum SHA-512
5dc3ae29564ca8177ec4dc97ff53cb02a1c59d9a12b988b8cf706ce6e73b6aa9341e666b50c07d0627445f75dcea2a12afe873a85c71c2951f0b1bd4135463af
Type fulltextMimetype application/pdf

By organisation
School of Computer Science and Communication (CSC)
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 28 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 429 hits
ReferencesLink to record
Permanent link

Direct link