Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Online aggregate tables: A method forimplementing big data analysis in PostgreSQLusing real time pre-calculations
KTH, School of Computer Science and Communication (CSC).
2017 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesisAlternative title
Realtidsaggregerade tabeller : En metod för analys av stora datamängder i PostgreSQL med hjälp av realtidsuppdaterade förberäkningar (Swedish)
Abstract [en]

In modern user-centric applications, data gathering and analysis is often of vitalimportance. Current trends in data management software show that traditionalrelational databases fail to keep up with the growing data sets. Outsourcingdata analysis often means data is locked in with a particular service, makingtransitions between analysis systems nearly impossible. This thesis implementsand evaluates a data analysis framework implemented completely within a re-lational database. The framework provides a structure for implementations ofonline algorithms of analytical methods to store precomputed results. The re-sult is an even resource utilization with predictable performance that does notdecrease over time. The system keeps all raw data gathered to allow for futureexportation. A full implementation of the framework is tested based on thecurrent analysis requirements of the company Shortcut Labs, and performancemeasurements show no problem with managing data sets of over a billion datapoints.

Abstract [sv]

I moderna användarcentrerade applikationer är insamling och analys av dataofta av affärskritisk vikt. Traditionalla relationsdatabaser har svårt att hanterade ökande datamängderna. Samtidigt medför användning av externa tjänster fördataanalys ofta inlåsning av data, vilket försvårar byte av analystjänst. Dennarapport presenterar och utvärderar ett ramverk för dataanalys som är imple-menterat i en relationsdatabas. Ramverket tillhandahåller strukturer för attförberäkna resultat för analytiska beräkningar på ett effektivt sätt. Resultatetblir en jämn resursanvändning med förutsägbar prestanda som inte försämrasöver tid. Ramverket sparar även all insamlad data vilket möjliggör exporter-ing. Ramverket utvärderas hos företaget Shortcut Labs och resultatet visar attramverket klarar av datamängder på över en miljard punkter.

Place, publisher, year, edition, pages
2017. , 35 p.
Keyword [en]
big data, aggregation, real-time, PostgreSQL
National Category
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-207808OAI: oai:DiVA.org:kth-207808DiVA: diva2:1098599
Educational program
Master of Science in Engineering - Computer Science and Technology
Supervisors
Examiners
Available from: 2017-06-05 Created: 2017-05-24 Last updated: 2017-06-05Bibliographically approved

Open Access in DiVA

fulltext(651 kB)19 downloads
File information
File name FULLTEXT01.pdfFile size 651 kBChecksum SHA-512
0b55aec3d7690126f7f207c98184115483bfe0b7a51f569f2ba13a0d41d9283e430a76073152f05635b79f2b90452c301be03c354239d0d51abe57964e011df5
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Bergmark, Fabian
By organisation
School of Computer Science and Communication (CSC)
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 19 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 74 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf