Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Design of a Network Library for Continuous Deep Analytics
KTH, School of Electrical Engineering and Computer Science (EECS).
2018 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesisAlternative title
Utformning av ett Nätverksbibliotek för Kontinuerlig Djupanalys av Data (Swedish)
Abstract [en]

Data-intensive stream processing applications have escalated in popularity in recent years, producing numerous designs and implementations for handling unbounded streams of high-volume data. The sheer size and dimensionality of these types of data requires multiple machines to push processing throughput of hundreds of millions events per second at low latencies.

Advances in the fields of distributed deep learning and stream processing have highlighted networking-specific challenges and requirements such as flow control and scalable communication abstractions. Existing stream processing frameworks, however, only address subsets of these requirements.

This thesis proposes a design and implementation in the Rust programming language for a modular networking library able to address these requirements together. The design entails protocol framing, buffer management, stream multiplexing, flow control, and stream prioritization. The implemented prototype handles multiplexing of logical streams and credit-based flow control through a flexible application programming interface. The prototype is tested for overall throughput and round-trip latency in a distributed environment, displaying promising results in both categories.

Abstract [sv]

Under de senaste åren har applikationer för dataintensiv ström bearbetning blivit avsevärt mer vanliga. Detta har lett till en uppsjö av modeller och implementationer för hantering av dataströmmar av gränslös volym. Blotta datamängden och dess dimensionalitet kräver otaliga maskiner för att med låg latens hantera hundratals miljoner händelser per sekund.

Framsteg inom området för distribuerad djupinlärning och ström bearbetning har blottlagt nätverksspecifika utmaningar och krav såsom flödeskontroll och skalbara kommunikationsabstraktioner. Nuvarande beräkningssystem för ström bearbetning uppfyller dessvärre bara en del av dessa villkor.

Detta examensarbete presenterar en modell och implementation i programmeringsspråket Rust för ett modulärt nätverksbibliotek som kan hantera alla dessa krav på en gång. Modellen inbegriper datainramning, bufferhantering, ström multiplexing, flödeskontroll och ström prioritering. Prototypen som här implementerats hanterar multiplexing av logiska dataströmmar och kreditbaserad flödeskontroll genom ett flexibelt applikationsgränssnitt. Prototypen har testats i avseende å nätverk genomströmning och tur-och-returtid i ett distribuerat upplägg, med lovande resultat i bägge kategorier.

Place, publisher, year, edition, pages
2018. , p. 72
Series
TRITA-EECS-EX ; 2018:170
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:kth:diva-232129OAI: oai:DiVA.org:kth-232129DiVA, id: diva2:1232485
Subject / course
Computer Science
Educational program
Master of Science - Software Engineering of Distributed Systems
Supervisors
Examiners
Available from: 2018-07-11 Created: 2018-07-11 Last updated: 2018-07-11Bibliographically approved

Open Access in DiVA

fulltext(6972 kB)62 downloads
File information
File name FULLTEXT01.pdfFile size 6972 kBChecksum SHA-512
b96db1dc5bed40339ebb70f33074a21c71b5da4ea6840a0dc6901e15382ea4d4886971a767c34135ce468ce3f49ba5a6363079d296bea987fe6f7b308b3cce63
Type fulltextMimetype application/pdf

By organisation
School of Electrical Engineering and Computer Science (EECS)
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 62 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 111 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf