Natural Language Processing In A Distributed Environment: A comparative performance analysis of Apache Spark and Hadoop MapReduce
Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesis
A big majority of the data hosted on the internet today is in natural text and therefore understanding natural language and how to effectively process and analyzing text has become a big part of data mining. Natural Language Processing has many applications in fields such as business intelligence and security purposes.The problem with natural language text processing and analyzing is the computational power needed to perform the actual processing, performance of personal computer has not kept up with amounts of data that needs to be processed so another approach with good performance scaling potential is needed.This study does a preliminary comparative performance analysis of processing natural language text in an distributed environment using two popular open-source frameworks, Hadoop MapReduce and Apache Spark.
Place, publisher, year, edition, pages
2016. , 33 p.
Engineering and Technology
IdentifiersURN: urn:nbn:se:umu:diva-126865OAI: oai:DiVA.org:umu-126865DiVA: diva2:1038338
Bachelor of Science Programme in Computing Science