Named Entity Recognition for Short Text Messages
2011 (English)In: Procedia - Social and Behavioral Sciences, ISSN 1877-0428, Vol. 27, no Pacling, 178-187 p.Article in journal (Refereed) Published
This paper describes a named entity recognition (NER) system for short text messages (SMS) running on a mobile platform. Most NER systems deal with text that is structured, formal, well written, with a good grammatical structure, and few spelling errors. SMS text messages lack these qualities and have instead a short-handed and mixed language studded with emoticons, which makes NER a challenge on this kind of material.
We implemented a system that recognizes named entities from SMSes written in Swedish and that runs on an Android cellular telephone. The entities extracted are locations, names, dates, times, and telephone numbers with the idea that extraction of these entities could be utilized by other applications running on the telephone. We started from a regular expression implementation that we complemented with classifiers using logistic regression. We optimized the recognition so that the incoming text messages could be processed on the telephone with a fast response time. We reached an F-score of 86 for strict matches and 89 for partial matches.
Place, publisher, year, edition, pages
Elsevier, 2011. Vol. 27, no Pacling, 178-187 p.
ensemble systems, information extraction, named entity recognition, short text messages, sms
Computer and Information Science
IdentifiersURN: urn:nbn:se:liu:diva-91230DOI: 10.1016/j.sbspro.2011.10.596OAI: oai:DiVA.org:liu-91230DiVA: diva2:616532