Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Real-Time Search in Large Networks and Clouds
KTH, School of Electrical Engineering (EES), Communication Networks. KTH, School of Electrical Engineering (EES), Centres, ACCESS Linnaeus Centre.
2013 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Networked systems, such as telecom networks and cloud infrastructures, hold and generate vast amounts of conguration and operational data, only a small portion of which is used today by management applications. The overall goal of this work is to make all this data available through a real-time search process named network search , where queries are invoked, without giving the location or the format of the data, similar to web search. Such a capability will simplify many management applications and enable new classes of realtime management solutions. The fundamental problems in network search relate to search in a vast and dynamic information space and the fact that the information is distributed across a very large system.

The thesis contains several contributions towards engineering a network search system. We present a weakly-structured information model, which enables representation of heterogeneous network data, a keyword-based search language, which supports location- and schema-oblivious search queries, and a distributed search mechanism, which is based on an echo protocol and supports a range of matching and ranking options. The search is performed in a peer-to-peer fashion in a network of search nodes. Each search node maintains a local real-time database of locally sensed conguration and operational information. Many of the concepts we developed for network search are based on results from the elds of information retrieval, web search, and very large databases. The key feature of our solution is that the search process and the computation of the query results is performed on local data inside the network or the cloud. We have build a prototype of the system on a cloud testbed and developed applications that use network search functionality. The performance measurements suggest that it is feasible to engineer a network search system that processes queries at low latency and low overhead, and that can scale to a very large system in the order of 100,000 nodes.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2013. , 87 p.
Series
Trita-EE, ISSN 1653-5146 ; 2013:35
National Category
Telecommunications
Identifiers
URN: urn:nbn:se:kth:diva-128193ISBN: 978-91-7501-879-9 (print)OAI: oai:DiVA.org:kth-128193DiVA: diva2:652076
Presentation
2013-10-14, Lab 2, Osquldas Väg 10, KTH, Stockholm, 10:00 (English)
Opponent
Supervisors
Note

QC 20130930

Available from: 2013-09-30 Created: 2013-09-10 Last updated: 2013-09-30Bibliographically approved
List of papers
1. Management by Network Search
Open this publication in new window or tab >>Management by Network Search
2012 (English)In: Proceedings of the 2012 IEEE Network Operations and Management Symposium, NOMS 2012, New York: IEEE Communications Society, 2012, 146-154 p.Conference paper, Published paper (Refereed)
Abstract [en]

While networked systems hold and generate vast amounts of configuration and operational data, this data is not accessible through a simple, uniform mechanism. Rather, it must be gathered using a range of different protocols and interfaces. Our vision is to make all this data available in a simple format through a realtime search process which runs within the network and aggregates the data into a form needed by applications - a concept we call network search. We believe that such an approach, though challenging, is technically feasible and will enable rapid development of new management applications and advanced network functions. This paper motivates and formulates the concept of network search, compares it to related concepts like web search, outlines a search architecture, describes the design space and research challenges, and reports on a testbed implementation with management applications built for exploratory purposes of this new paradigm.

Place, publisher, year, edition, pages
New York: IEEE Communications Society, 2012
Series
IEEE IFIP Network Operations and Management Symposium, ISSN 1542-1201
Keyword
Advanced networks, Design spaces, Different protocols, Management applications, Network search, Networked systems, Operational data, Real time, Research challenges, Search architecture, Search process, Web searches
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:kth:diva-86144 (URN)10.1109/NOMS.2012.6211893 (DOI)000309517000018 ()2-s2.0-84864250287 (Scopus ID)978-146730268-5 (ISBN)
Conference
IEEE Network Operations and Management Symposium, NOMS 2012; Maui, HI; 16 April 2012 through 20 April 2012
Funder
ICT - The Next Generation
Note

© 2012 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

QC 20120604

Available from: 2012-06-04 Created: 2012-02-13 Last updated: 2016-04-11Bibliographically approved
2. A Query Language for Network Search
Open this publication in new window or tab >>A Query Language for Network Search
2013 (English)In: Proceedings of the 2013 IFIP/IEEE International Symposium on Integrated Network Management, IM 2013, New York: IEEE conference proceedings, 2013, 109-117 p.Conference paper, Published paper (Refereed)
Abstract [en]

Network search makes operational data available in real-time to management applications. In contrast to traditional monitoring, neither the data location nor the data format needs to be known to the invoking process, which simplifies application development, but requires an efficient search plane inside the managed system. This paper presents a query language for network search and discusses how search queries can be executed in a networked system. The search space consists of named objects that are modeled as sets of attribute-value pairs. The data model is more general than the relational model, and the query language is more expressive than relational calculus. The paper shows that distributed query processing can be performed using an echo algorithm and that name resolution can be embedded in query processing. Finally, two use cases for network search are presented, one in networking and one in cloud computing, the latter backed up by a prototype implementation.

Place, publisher, year, edition, pages
New York: IEEE conference proceedings, 2013
Keyword
Network search, management paradigms, distributed management, name resolution
National Category
Computer Systems Communication Systems
Identifiers
urn:nbn:se:kth:diva-102881 (URN)000327159900014 ()2-s2.0-84883472607 (Scopus ID)978-1-4673-5229-1 (ISBN)
Conference
Integrated Network Management (IM 2013), 2013 IFIP/IEEE International Symposium on; Date 27-31 May 2013, Ghent, Belgium
Note

QC 20130809

Available from: 2012-09-27 Created: 2012-09-27 Last updated: 2016-04-11Bibliographically approved
3. Real-Time Search in Clouds
Open this publication in new window or tab >>Real-Time Search in Clouds
2013 (English)In: Proceedings of the 2013 IFIP/IEEE International Symposium on Integrated Network Management, IM 2013, New York: IEEE , 2013, 1070-1071 p.Conference paper, Published paper (Refereed)
Abstract [en]

We developed a novel approach for management of networks/networked systems based on network search [4]. Network search provides a simple, uniform interface, through which human administrators and management applications can obtain network information, configuration or operational, without knowing its schema and location. We believe that the capability of network search will spur the development of new tools for human administrators and enable the rapid development of new classes of network control functions and applications.

Place, publisher, year, edition, pages
New York: IEEE, 2013
Keyword
Management applications, Network control, Network information, Network search, Real-time searches, Uniform interface
National Category
Telecommunications
Identifiers
urn:nbn:se:kth:diva-128192 (URN)000327159900169 ()2-s2.0-84883481353 (Scopus ID)978-1-4673-5229-1 (ISBN)
Conference
Demonstration Track, In Proc. IEEE/IFIP International Symposium on Integrated Network Management (IM), Ghent, Belgium, May 27-30, 2013.
Note

QC 20130930

Available from: 2013-09-10 Created: 2013-09-10 Last updated: 2016-04-11Bibliographically approved
4. Scalable Matching and Ranking for Network Search
Open this publication in new window or tab >>Scalable Matching and Ranking for Network Search
2013 (English)In: 2013 9th International Conference on Network and Service Management (CNSM), IEEE Computer Society, 2013, 251-259 p.Conference paper, Published paper (Refereed)
Abstract [en]

Network search makes operational data available in real-time to management applications. In contrast to traditional monitoring, neither the data location nor the data format needs to be known to the invoking process, which simplifies application development, but requires an efficient search plane inside the managed system. The search plane is realized as a network of search nodes that process search queries in a distributed fashion. This paper introduces matching and ranking for network search queries. We are proposing a semantic for matching and ranking, which is configurable to support different types of management applications - from exact matching for database-style queries to loose, approximate matching, which is appropriate for exploratory purposes. We describe an echo protocol for efficient distributed query processing that supports matching and ranking. Further, we present the design of a search node, which maintains a real-time database of operational information and allows for parallel processing of search queries. A prototype implementation on a cloud testbed shows that the network search system, on a 9-node cluster with 24 core servers, executes 200 global search queries/sec with the 75th percetile latency below 100 milliseconds and with a CPU utilization below 5%. The performance measurements, together with our design, suggest that a system of 100,000 servers processing the same load would exhibit the same overhead per server and a query latency of below 1 sec.

Place, publisher, year, edition, pages
IEEE Computer Society, 2013
Keyword
distributed management, distributed query processing, innetwork management, matching and ranking, Network search
National Category
Telecommunications
Identifiers
urn:nbn:se:kth:diva-128191 (URN)10.1109/CNSM.2013.6727844 (DOI)000345853200042 ()2-s2.0-84894467062 (Scopus ID)978-3-901882-53-1 (ISBN)
Conference
2013 9th International Conference on Network and Service Management, CNSM 2013 and its three collocated Workshops - ICQT 2013, SVM 2013 and SETM 2013; Zurich; Switzerland; 14 October 2013 through 18 October 2013
Note

QC 20130930

Available from: 2013-09-10 Created: 2013-09-10 Last updated: 2016-04-11Bibliographically approved

Open Access in DiVA

fulltext(2716 kB)490 downloads
File information
File name FULLTEXT01.pdfFile size 2716 kBChecksum SHA-512
21c5d95dd0cd5d432b7f71a361d208aa8b8815072dbb5d208e358e0e616823919f32f4b8e16be7a9072aa0b474e7a4622b2fa7b8d12cb5a6b95a1f69e2a57c76
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Uddin, Misbah
By organisation
Communication NetworksACCESS Linnaeus Centre
Telecommunications

Search outside of DiVA

GoogleGoogle Scholar
Total: 490 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 414 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf