Digitala Vetenskapliga Arkivet

Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Vector Symbolic Architectures and their applications: Computing with random vectors in a hyperdimensional space
Luleå tekniska universitet, Institutionen för system- och rymdteknik, Datavetenskap.ORCID-id: 0000-0002-6032-6155
2018 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)Alternativ titel
Vektor symboliska Arkitekturer och deras tillämpningar : Beräkning med slumpmässiga vektorer i ett hyperdimensionellt utrymme (Svenska)
Abstract [en]

The main focus of this thesis lies in a rather narrow subfield of Artificial Intelligence. As any beloved child, it has many names. The most common ones are Vector Symbolic Architectures and Hyperdimensional Computing. Vector Symbolic Architectures are a family of bio-inspired methods of representing and manipulating concepts and their meanings in a high-dimensional space (hence Hyperdimensional Computing). Information in Vector Symbolic Architectures is evenly distributed across representational units, therefore, it is said that they operate with distributed representations. Representational units can be of different nature, however, the thesis concentrates on the case when units have either binary or integer values. 

This thesis includes eleven scientific papers and extends the research area in three directions: theory of Vector Symbolic Architectures, their applications for pattern recognition, and unification of Vector Symbolic Architectures with other neural-like computational approaches. 

Previously, Vector Symbolic Architectures have been used mainly in the area of cognitive computing for representing and reasoning upon semantically bound information, for example, for analogy-based reasoning. This thesis significantly extends the applicability of Vector Symbolic Architectures to an area of pattern recognition. Pattern recognition is the area constantly enlarging its theoretical and practical horizons. Applications of pattern recognition and machine learning can be found in many areas of the present day world including health-care, robotics, manufacturing, economics, automation, transportation, etc. Despite the success in many domains pattern recognition algorithms are still far from being close to their biological vis-a-vis – the brain. In particular, one of the challenges is a large amount of training data required by conventional machine learning algorithms. Therefore, it is important to look for new possibilities in the area via exploring biologically inspired approaches.

All application scenarios, which are considered in the thesis, contribute to the development of the global strategy of creating an information society. Specifically, such important applications as biomedical signal processing, automation systems, and text processing were considered. All applications scenarios used novel methods of mapping data to Vector Symbolic Architectures proposed in the thesis.

In the domain of biomedical signal processing, Vector Symbolic Architectures were applied for three tasks: classification of a modality of medical images, gesture recognition, and assessment of synchronization of cardiovascular signals. In the domain of automation systems, Vector Symbolic Architectures were used for a data-driven fault isolation. In the domain of text processing, Vector Symbolic Architectures were used to search for the longest common substring and to recognize permuted words.

The theoretical contributions of the thesis come in four aspects. First, the thesis proposes several methods for mapping data from its original representation into a distributed representation suitable for further manipulations by Vector Symbolic Architectures. These methods can be used for one-shot learning of patterns of generic sensor stimuli. Second, the thesis presents the analysis of an informational capacity of Vector Symbolic Architectures in the case of binary distributed representations. Third, it is shown how to represent finite state automata using Vector Symbolic Architectures. Fourth, the thesis describes the approach of combining Vector Symbolic Architectures and a cellular automaton.

Finally, the thesis presents the results of unification of two computational approaches with Vector Symbolic Architectures. This is one of the most interesting cross-disciplinary contributions of the thesis. First, it is shown that Bloom Filters – an important data structure for an approximate membership query task – can be treated in terms of Vector Symbolic Architectures. It allows generalizing the process of building the filter. Second, Vector Symbolic Architectures and Echo State Networks (a special kind of recurrent neural networks) were combined together. It is possible to implement Echo State Networks using only integer values in network’s units and much simpler operation for a recurrency operation while preserving the entire dynamics of the network. It results in a simpler architecture with lower requirements on memory and operations. 

Ort, förlag, år, upplaga, sidor
Luleå: Luleå University of Technology, 2018.
Serie
Doctoral thesis / Luleå University of Technology 1 jan 1997 → …, ISSN 1402-1544
Nationell ämneskategori
Annan elektroteknik och elektronik Datorsystem Datavetenskap (datalogi)
Forskningsämne
Kommunikations- och beräkningssystem
Identifikatorer
URN: urn:nbn:se:ltu:diva-68338ISBN: 978-91-7790-110-5 (tryckt)ISBN: 978-91-7790-111-2 (digital)OAI: oai:DiVA.org:ltu-68338DiVA, id: diva2:1197565
Disputation
2018-06-11, A109, Luleå, 10:00 (Engelska)
Opponent
Handledare
Forskningsfinansiär
Vetenskapsrådet, 2015-04677Tillgänglig från: 2018-04-16 Skapad: 2018-04-13 Senast uppdaterad: 2025-10-22Bibliografiskt granskad
Delarbeten
1. Modality Classification of Medical Images with Distributed Representations Based on Cellular Automata Reservoir Computing
Öppna denna publikation i ny flik eller fönster >>Modality Classification of Medical Images with Distributed Representations Based on Cellular Automata Reservoir Computing
2017 (Engelska)Ingår i: Proceedings - International Symposium on Biomedical Imaging, Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE), 2017, s. 1053-1056Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Modality corresponding to medical images is a vital filter in medical image retrieval systems. This article presents the classification of modalities of medical images based on the usage of principles of hyper-dimensional computing and reservoir computing. It is demonstrated that the highest classification accuracy of the proposed method is on a par with the best classical method for the given dataset (83% vs. 84%). The major positive property of the proposed method is that it does not require any optimization routine during the training phase and naturally allows for incremental learning upon the availability of new training data.

Ort, förlag, år, upplaga, sidor
Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE), 2017
Serie
Proceedings. IEEE International Symposium on Biomedical Imaging, E-ISSN 1945-7928
Nationell ämneskategori
Medicinsk bildvetenskap Datavetenskap (datalogi)
Forskningsämne
Kommunikations- och beräkningssystem
Identifikatorer
urn:nbn:se:ltu:diva-61558 (URN)10.1109/ISBI.2017.7950697 (DOI)000414283200243 ()2-s2.0-85023198723 (Scopus ID)9781509011711 (ISBN)
Konferens
2017 IEEE International Symposium on Biomedical Imaging, Melbourne, Australia, 18-21 April 2017
Forskningsfinansiär
Vetenskapsrådet, 2015-04677
Tillgänglig från: 2017-01-20 Skapad: 2017-01-20 Senast uppdaterad: 2025-10-22Bibliografiskt granskad
2. No Two Brains Are Alike: Cloning a Hyperdimensional Associative Memory Using Cellular Automata Computations
Öppna denna publikation i ny flik eller fönster >>No Two Brains Are Alike: Cloning a Hyperdimensional Associative Memory Using Cellular Automata Computations
2018 (Engelska)Ingår i: Biologically Inspired Cognitive Architectures (BICA) for Young Scientists: First International Early Research Career Enhancement School on BICA and Cybersecurity (FIERCES 2017) / [ed] Alexei V. Samsonovich, Valentin V. Klimov, Cham: Springer, 2018, s. 91-100Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

This paper looks beyond of the current focus of research on biologically inspired cognitive systems and considers the problem of replication of its learned functionality. The considered challenge is to replicate the learned knowledge such that uniqueness of the internal symbolic representations is guaranteed. This article takes a neurological argument “no two brains are alike” and suggests an architecture for mapping a content of the trained associative memory built using principles of hyperdimensional computing and Vector Symbolic Architectures into a new and orthogonal basis of atomic symbols. This is done with the help of computations on cellular automata. The results of this article open a way towards a secure usage of cognitive architectures in a variety of practical application domains.

Ort, förlag, år, upplaga, sidor
Cham: Springer, 2018
Serie
Advances in Intelligent Systems and Computing, ISSN 2194-5357 ; 636
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
Kommunikations- och beräkningssystem
Identifikatorer
urn:nbn:se:ltu:diva-63644 (URN)10.1007/978-3-319-63940-6_13 (DOI)000454681600013 ()2-s2.0-85046663041 (Scopus ID)978-3-319-63939-0 (ISBN)978-3-319-63940-6 (ISBN)
Konferens
First International Early Research Career Enhancement School on BICA and Cybersecurity (FIERCES 2017), Moscow, Russia, 1-6 August 2017
Forskningsfinansiär
Vetenskapsrådet
Tillgänglig från: 2017-06-01 Skapad: 2017-06-01 Senast uppdaterad: 2025-10-22Bibliografiskt granskad
3. On bidirectional transitions between localist and distributed representations: The case of common substrings search using Vector Symbolic Architecture
Öppna denna publikation i ny flik eller fönster >>On bidirectional transitions between localist and distributed representations: The case of common substrings search using Vector Symbolic Architecture
2014 (Engelska)Ingår i: Procedia Computer Science, E-ISSN 1877-0509, Vol. 41, s. 104-113Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

The contribution of this article is twofold. First, it presents an encoding approach for seamless bidirectional transitions between localist and distributed representation domains. Second, the approach is demonstrated on the example of using Vector Symbolic Architecture for solving a problem of finding common substrings. The proposed algorithm uses elementary operations on long binary vectors. For the case of two patterns with respective lengths L1 and L2 it requires Θ(L1 + L2 – 1) operations on binary vectors, which is equal to the suffix trees approach – the fastest algorithm for this problem. The simulation results show that in order to be robustly detected by the proposed approach the length of a common substring should be more than 4% of the longest pattern.

Ort, förlag, år, upplaga, sidor
Elsevier, 2014
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
Kommunikations- och beräkningssystem
Identifikatorer
urn:nbn:se:ltu:diva-15592 (URN)10.1016/j.procs.2014.11.091 (DOI)000361488600014 ()2-s2.0-84939250548 (Scopus ID)f200757d-c493-4d45-af4f-2c61f85a73e8 (Lokalt ID)f200757d-c493-4d45-af4f-2c61f85a73e8 (Arkivnummer)f200757d-c493-4d45-af4f-2c61f85a73e8 (OAI)
Konferens
International Conference on Biologically Inspired Cognitive Architectures : Fifth Annual Meeting of the BICA Society 07/11/2014 - 09/11/2014
Anmärkning
Validerad; 2015; Nivå 2; 20140824 (denkle)Tillgänglig från: 2016-09-29 Skapad: 2016-09-29 Senast uppdaterad: 2025-10-21Bibliografiskt granskad
4. Recognizing Permuted Words with Vector Symbolic Architectures: A Cambridge Test for Machines
Öppna denna publikation i ny flik eller fönster >>Recognizing Permuted Words with Vector Symbolic Architectures: A Cambridge Test for Machines
2016 (Engelska)Ingår i: Procedia Computer Science, E-ISSN 1877-0509, Vol. 88, s. 169-175Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

This paper proposes a simple encoding scheme for words using principles of Vector Symbolic Architectures. The proposed encoding allows finding a valid word in the dictionary for a given permuted word (represented using the proposed approach) using only a single operation - calculation of Hamming distance to the distributed representations of valid words in the dictionary. The proposed encoding scheme can be used as an additional processing mechanism for models of word embedding, which also form vectors to represent the meanings of words, in order to match the distorted words in the text to the valid words in the dictionary.

Ort, förlag, år, upplaga, sidor
Elsevier, 2016
Nyckelord
Vector Symbolic Architectures, distributed representation, spell-checking, word embedding
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
Kommunikations- och beräkningssystem
Identifikatorer
urn:nbn:se:ltu:diva-40430 (URN)10.1016/j.procs.2016.07.421 (DOI)000391723200024 ()2-s2.0-85006377707 (Scopus ID)f921c1ad-944a-4427-88f6-7559bdb89676 (Lokalt ID)f921c1ad-944a-4427-88f6-7559bdb89676 (Arkivnummer)f921c1ad-944a-4427-88f6-7559bdb89676 (OAI)
Konferens
7th Annual International Conference on Biologically Inspired Cognitive Architectures, BICA 2016, July 16-19 2016 in New York City
Anmärkning

2017-03-27 (andbra);Konferensartikel i tidskrift

Tillgänglig från: 2016-10-03 Skapad: 2016-10-03 Senast uppdaterad: 2025-10-22Bibliografiskt granskad
5. Fault Detection in the Hyperspace: Towards Intelligent Automation Systems
Öppna denna publikation i ny flik eller fönster >>Fault Detection in the Hyperspace: Towards Intelligent Automation Systems
Visa övriga...
2015 (Engelska)Ingår i: IEEE International Conference on Industrial Informatics: INDIN 2015, Cambridge, UK, July 22-24, 2015. Proceedings, Piscataway, NJ: IEEE Communications Society, 2015, s. 1219-1224, artikel-id 7281909Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

This article presents a methodology for intelligent, biologically inspired fault detection system for generic complex systems of systems. The proposed methodology utilizes the concepts of associative memory and vector symbolic architectures, commonly used for modeling cognitive abilities of human brain. Compared to classical methods of artificial intelligence used in the context of fault detection the proposed methodology shows an unprecedented performance, while featuring zero configuration and simple operations.

Ort, förlag, år, upplaga, sidor
Piscataway, NJ: IEEE Communications Society, 2015
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
Kommunikations- och beräkningssystem
Identifikatorer
urn:nbn:se:ltu:diva-32046 (URN)10.1109/INDIN.2015.7281909 (DOI)2-s2.0-84949512219 (Scopus ID)668a6fde-3e94-4870-a560-0ea6e83a5245 (Lokalt ID)9781479966493 (ISBN)668a6fde-3e94-4870-a560-0ea6e83a5245 (Arkivnummer)668a6fde-3e94-4870-a560-0ea6e83a5245 (OAI)
Konferens
IEEE International Conference on Industrial Informatics : 22/07/2015 - 24/07/2015
Anmärkning
Validerad; 2016; Nivå 1; 20150522 (denkle)Tillgänglig från: 2016-09-30 Skapad: 2016-09-30 Senast uppdaterad: 2018-07-10Bibliografiskt granskad
6. Holographic Graph Neuron: a Bio-Inspired Architecture for Pattern Processing
Öppna denna publikation i ny flik eller fönster >>Holographic Graph Neuron: a Bio-Inspired Architecture for Pattern Processing
Visa övriga...
2017 (Engelska)Ingår i: IEEE Transactions on Neural Networks and Learning Systems, ISSN 2162-237X, E-ISSN 2162-2388, Vol. 28, nr 6, s. 1250-1262Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

This article proposes the use of Vector Symbolic Architectures for implementing Hierarchical Graph Neuron, an architecture for memorizing patterns of generic sensor stimuli. The adoption of a Vector Symbolic representation ensures a one-layered design for the approach, while maintaining the previously reported properties and performance characteristics of Hierarchical Graph Neuron, and also improving the noise resistance of the architecture. The proposed architecture enables a linear (with respect to the number of stored entries) time search for an arbitrary sub-pattern.

Ort, förlag, år, upplaga, sidor
IEEE, 2017
Nationell ämneskategori
Datavetenskap (datalogi)
Forskningsämne
Kommunikations- och beräkningssystem
Identifikatorer
urn:nbn:se:ltu:diva-11974 (URN)10.1109/TNNLS.2016.2535338 (DOI)000401982100001 ()26978836 (PubMedID)2-s2.0-84960540059 (Scopus ID)b0709bbc-d372-4032-90cc-1dca4ff3a12d (Lokalt ID)b0709bbc-d372-4032-90cc-1dca4ff3a12d (Arkivnummer)b0709bbc-d372-4032-90cc-1dca4ff3a12d (OAI)
Anmärkning

Validerad;2017;Nivå 2;2017-06-01 (andbra)

Tillgänglig från: 2016-09-29 Skapad: 2016-09-29 Senast uppdaterad: 2025-10-21Bibliografiskt granskad
7. Classification and Recall With Binary Hyperdimensional Computing: Tradeoffs in Choice of Density and Mapping Characteristics
Öppna denna publikation i ny flik eller fönster >>Classification and Recall With Binary Hyperdimensional Computing: Tradeoffs in Choice of Density and Mapping Characteristics
Visa övriga...
2018 (Engelska)Ingår i: IEEE Transactions on Neural Networks and Learning Systems, ISSN 2162-237X, E-ISSN 2162-2388, Vol. 29, nr 12, s. 5880-5898Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Hyperdimensional (HD) computing is a promising paradigm for future intelligent electronic appliances operating at low power. This paper discusses tradeoffs of selecting parameters of binary HD representations when applied to pattern recognition tasks. Particular design choices include density of representations and strategies for mapping data from the original representation. It is demonstrated that for the considered pattern recognition tasks (using synthetic and real-world data) both sparse and dense representations behave nearly identically. This paper also discusses implementation peculiarities which may favor one type of representations over the other. Finally, the capacity of representations of various densities is discussed.

Ort, förlag, år, upplaga, sidor
IEEE, 2018
Nationell ämneskategori
Datorsystem
Forskningsämne
Kommunikations- och beräkningssystem
Identifikatorer
urn:nbn:se:ltu:diva-68400 (URN)10.1109/TNNLS.2018.2814400 (DOI)000451230100008 ()29993669 (PubMedID)2-s2.0-85045214003 (Scopus ID)
Forskningsfinansiär
Vetenskapsrådet, 2015- 04677
Anmärkning

Validerad;2018;Nivå 2;2018-12-05 (svasva)

Tillgänglig från: 2018-04-18 Skapad: 2018-04-18 Senast uppdaterad: 2025-10-22Bibliografiskt granskad
8. Associative Synthesis of Finite State Automata Model of a Controlled Object with Hyperdimensional Computing
Öppna denna publikation i ny flik eller fönster >>Associative Synthesis of Finite State Automata Model of a Controlled Object with Hyperdimensional Computing
2017 (Engelska)Ingår i: Proceedings IECON 2017: 43rd Annual Conference of the IEEE Industrial Electronics Society, Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE), 2017, s. 3276-3281Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

The main contribution of this paper is a study of the applicability of hyperdimensional computing and learning with an associative memory for modeling the dynamics of complex automation systems. Specifically, the problem of learning an evidence-based model of a plant in a distributed automation and control system is considered. The model is learned in the form a finite state automata. 

Ort, förlag, år, upplaga, sidor
Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE), 2017
Serie
IEEE Industrial Electronics Society, ISSN 1553-572X
Nationell ämneskategori
Datorsystem
Forskningsämne
Kommunikations- och beräkningssystem
Identifikatorer
urn:nbn:se:ltu:diva-66594 (URN)10.1109/IECON.2017.8216554 (DOI)000427164803042 ()2-s2.0-85046676612 (Scopus ID)9781538611272 (ISBN)
Konferens
43rd Annual Conference of the IEEE Industrial Electronics Society, IECON 2017, Bejing, China, 29 October - 1 November 2017
Tillgänglig från: 2017-11-16 Skapad: 2017-11-16 Senast uppdaterad: 2025-10-22Bibliografiskt granskad

Open Access i DiVA

fulltext(21137 kB)3688 nedladdningar
Filinformation
Filnamn FULLTEXT01.pdfFilstorlek 21137 kBChecksumma SHA-512
a77655bb9897f99fc85bbb503932e8d67586c552368c62bf58761c1a47b2784b5b8d808fafef4dc2bce35e72438d022d1de71bbd7c666dcc51b8be2c0f14f744
Typ fulltextMimetyp application/pdf

Sök vidare i DiVA

Av författaren/redaktören
Kleyko, Denis
Av organisationen
Datavetenskap
Annan elektroteknik och elektronikDatorsystemDatavetenskap (datalogi)

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 3692 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 10291 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf