Change search
ReferencesLink to record
Permanent link

Direct link
Continuously adapting continuous Queries for Data Streams in Raincoat
Norwegian University of Science and Technology, Faculty of Information Technology, Mathematics and Electrical Engineering, Department of Computer and Information Science.
Norwegian University of Science and Technology, Faculty of Information Technology, Mathematics and Electrical Engineering, Department of Computer and Information Science.
2013 (English)MasteroppgaveStudent thesis
Abstract [en]

In the last decade, the world wide web has grown from being a platform where users passively viewed content, to an active platform where the users themselves contributed with new content. With this came an explosion of available data that ventures could use to gain market advantage. Not only did the did the amount of available data grow massively, but also newly produced data started to arrive at immense speed. This spawned a new field of specialized computational framework being able to handle the change in the data paradigm. Now, one must be able to process the massive amount of incoming data within a reasonable response time, as well as be able to handle its high velocity. This spurred several ideas for processing fast data. One of these ideas uses SQL-like languages for processing fast data, taking advantage of the years of work on query optimization theory. In the fall of 2012, we proposed and implemented the prototype of Raincoat. Raincoat was developed to ease developers without any experience with distributed programming, providing a familiar interface which they could use to deploy stream filtering jobs to a Storm cluster. As the prototype did not include any query optimization techniques it does not meet the expected performance requirements. In this thesis we research optimization techniques for scaling Raincoat. We explore optimization techniques from different fields including traditional, distributed, parallel, streaming and adaptive query optimization. We propose an adaptive query optimizer, inspired by existing adaptive query optimizers. The focus of the optimizer lies in detecting when an optimization is needed and which optimization techniques that should be applied. In this thesis we explore the possibility of adaptively achieving better performance and scalability by carefully selecting the join order, select order, merging of selection operators, and applying intra-operator parallelism on operators. Based on our results from experiments on the different implemented optimizers, we demonstrate their applicability and their significant contribution in increasing the performance of a Raincoat query.

Place, publisher, year, edition, pages
Institutt for datateknikk og informasjonsvitenskap , 2013. , 130 p.
URN: urn:nbn:no:ntnu:diva-22980Local ID: ntnudaim:8927OAI: diva2:655614
Available from: 2013-10-12 Created: 2013-10-12 Last updated: 2013-10-12Bibliographically approved

Open Access in DiVA

fulltext(2848 kB)329 downloads
File information
File name FULLTEXT01.pdfFile size 2848 kBChecksum SHA-512
Type fulltextMimetype application/pdf
cover(185 kB)5 downloads
File information
File name COVER01.pdfFile size 185 kBChecksum SHA-512
Type coverMimetype application/pdf
attachment(13752 kB)19 downloads
File information
File name ATTACHMENT01.zipFile size 13752 kBChecksum SHA-512
Type attachmentMimetype application/zip

By organisation
Department of Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 329 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 48 hits
ReferencesLink to record
Permanent link

Direct link