Change search
ReferencesLink to record
Permanent link

Direct link
Real-time data stream clustering over sliding windows
Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Computing Science. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Computing Science. (Uppsala Database laboratory)
2016 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

In many applications, e.g. urban traffic monitoring, stock trading, and industrial sensor data monitoring, clustering algorithms are applied on data streams in real-time to find current patterns. Here, sliding windows are commonly used as they capture concept drift.

Real-time clustering over sliding windows is early detection of continuously evolving clusters as soon as they occur in the stream, which requires efficient maintenance of cluster memberships that change as windows slide.

Data stream management systems (DSMSs) provide high-level query languages for searching and analyzing streaming data. In this thesis we extend a DSMS with a real-time data stream clustering framework called Generic 2-phase Continuous Summarization framework (G2CS).  G2CS modularizes data stream clustering by taking as input clustering algorithms which are expressed in terms of a number of functions and indexing structures. G2CS supports real-time clustering by efficient window sliding mechanism and algorithm transparent indexing. A particular challenge for real-time detection of a high number of rapidly evolving clusters is efficiency of window slides for clustering algorithms where deletion of expired data is not supported, e.g. BIRCH. To that end, G2CS includes a novel window maintenance mechanism called Sliding Binary Merge (SBM). To further improve real-time sliding performance, G2CS uses generation-based multi-dimensional indexing where indexing structures suitable for the clustering algorithms can be plugged-in.

Place, publisher, year, edition, pages
Uppsala: Acta Universitatis Upsaliensis, 2016. , 33 p.
Series
Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology, ISSN 1651-6214 ; 1431
Keyword [en]
Data streaming; Sliding windows; Clustering;
National Category
Computer Systems
Research subject
Computer Science with specialization in Database Technology
Identifiers
URN: urn:nbn:se:uu:diva-302799ISBN: 978-91-554-9698-2OAI: oai:DiVA.org:uu-302799DiVA: diva2:967686
Public defence
2016-11-23, ITC 2446, Lägerhyddsvägen 2, Uppsala, 10:00 (English)
Opponent
Supervisors
Available from: 2016-11-02 Created: 2016-09-09 Last updated: 2016-11-16
List of papers
1. Scalable ordered indexing of streaming data
Open this publication in new window or tab >>Scalable ordered indexing of streaming data
2012 (English)In: 3rd International Workshop on Accelerating Data Management Systems using Modern Processor and Storage Architectures, 2012, 11- p.Conference paper (Refereed)
National Category
Computer Science
Identifiers
urn:nbn:se:uu:diva-185068 (URN)
Conference
ADMS 2012, Istanbul, Turkey
Projects
eSSENCE
Available from: 2012-08-27 Created: 2012-11-19 Last updated: 2016-09-09Bibliographically approved
2. Grand challenge: Implementation by frequently emitting parallel windows and user-defined aggregate functions
Open this publication in new window or tab >>Grand challenge: Implementation by frequently emitting parallel windows and user-defined aggregate functions
Show others...
2013 (English)In: Proc. 7th ACM International Conference on Distributed Event-Based Systems, New York: ACM Press, 2013, 325-330 p.Conference paper (Refereed)
Place, publisher, year, edition, pages
New York: ACM Press, 2013
National Category
Computer Science
Identifiers
urn:nbn:se:uu:diva-211954 (URN)10.1145/2488222.2488284 (DOI)978-1-4503-1758-0 (ISBN)
External cooperation:
Conference
DEBS 2013
Available from: 2013-06-29 Created: 2013-12-03 Last updated: 2016-09-09Bibliographically approved
3. Distributed multi-query optimization of continuous clustering queries
Open this publication in new window or tab >>Distributed multi-query optimization of continuous clustering queries
2014 (English)In: Proc. VLDB 2014 PhD Workshop, 2014Conference paper (Refereed)
Abstract [en]

This work addresses the problem of sharing execution plans for queries that continuously cluster streaming data to provide an evolving summary of the data stream. This is challenging since clustering is an expensive task, there might be many clustering queries running simultaneously, each continuous query has a long life time span, and the execution plans often overlap. Clustering is similar to conventional grouped aggregation but cluster formation is more expensive than group formation, which makes incremental maintenance more challenging. The goal of this work is to minimize response time of continuous clustering queries with limited resources through multi-query optimization. To that end, strategies for sharing execution plans between continuous clustering queries are investigated and the architecture of a system is outlined that optimizes the processing of multiple such queries. Since there are many clustering algorithms, the system should be extensible to easily incorporate user defined clustering algorithms.

National Category
Computer Science
Research subject
Computer Science with specialization in Database Technology
Identifiers
urn:nbn:se:uu:diva-302790 (URN)
External cooperation:
Conference
VLDB 2014
Available from: 2016-09-09 Created: 2016-09-09 Last updated: 2016-09-09Bibliographically approved
4. Framework for real-time clustering over sliding windows
Open this publication in new window or tab >>Framework for real-time clustering over sliding windows
2016 (English)In: Proc. 28th International Conference on Scientific and Statistical Database Management, New York: ACM Press, 2016, 1-13 p., 19Conference paper (Refereed)
Place, publisher, year, edition, pages
New York: ACM Press, 2016
National Category
Computer Science
Identifiers
urn:nbn:se:uu:diva-302792 (URN)10.1145/2949689.2949696 (DOI)978-1-4503-4215-5 (ISBN)
External cooperation:
Conference
SSDBM 2016
Available from: 2016-07-18 Created: 2016-09-09 Last updated: 2016-09-12Bibliographically approved

Open Access in DiVA

fulltext(677 kB)56 downloads
File information
File name FULLTEXT01.pdfFile size 677 kBChecksum SHA-512
ff233b0ffb4ccaac879cf18e285ec5a8ba33df5e043c33c19e217f16aa9901dc5e1481dc845900cc4021e99856dad8b55e9ff5731d5067ac7eb4d750dcd07262
Type fulltextMimetype application/pdf
Buy this publication >>

Search in DiVA

By author/editor
Badiozamany, Sobhan
By organisation
Division of Computing ScienceComputing Science
Computer Systems

Search outside of DiVA

GoogleGoogle Scholar
Total: 56 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 256 hits
ReferencesLink to record
Permanent link

Direct link