Digitala Vetenskapliga Arkivet

Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Fluent human–robot dialogues about grounded objects in home environments
Örebro University, School of Science and Technology. (AASS)
Örebro University, School of Science and Technology. Department for Speech, Music and Hearing (TMH), Royal Institute of Technology (KTH), Stockholm, Sweden.
Örebro University, School of Science and Technology. (AASS)ORCID iD: 0000-0002-3122-693X
2014 (English)In: Cognitive Computation, ISSN 1866-9956, E-ISSN 1866-9964, Vol. 6, no 4, p. 914-927Article in journal (Refereed) Published
Abstract [en]

To provide a spoken interaction between robots and human users, an internal representation of the robots sensory information must be available at a semantic level and accessible to a dialogue system in order to be used in a human-like and intuitive manner. In this paper, we integrate the fields of perceptual anchoring (which creates and maintains the symbol-percept correspondence of objects) in robotics with multimodal dialogues in order to achieve a fluent interaction between humans and robots when talking about objects. These everyday objects are located in a so-called symbiotic system where humans, robots, and sensors are co-operating in a home environment. To orchestrate the dialogue system, the IrisTK dialogue platform is used. The IrisTK system is based on modelling the interaction of events, between different modules, e.g. speech recognizer, face tracker, etc. This system is running on a mobile robot device, which is part of a distributed sensor network. A perceptual anchoring framework, recognizes objects placed in the home and maintains a consistent identity of the objects consisting of their symbolic and perceptual data. Particular effort is placed on creating flexible dialogues where requests to objects can be made in a variety of ways. Experimental validation consists of evaluating the system when many objects are possible candidates for satisfying these requests.

Place, publisher, year, edition, pages
Springer, 2014. Vol. 6, no 4, p. 914-927
Keywords [en]
Human–robot interaction, Perceptual anchoring, Symbol grounding, Spoken dialogue systems, Social robotics
National Category
Robotics and automation
Research subject
Computer Science; Information technology
Identifiers
URN: urn:nbn:se:oru:diva-39388DOI: 10.1007/s12559-014-9291-yISI: 000345994900022Scopus ID: 2-s2.0-84916227381OAI: oai:DiVA.org:oru-39388DiVA, id: diva2:769246
Funder
Swedish Research Council, 2011-6104Available from: 2014-12-06 Created: 2014-12-06 Last updated: 2025-02-09Bibliographically approved
In thesis
1. Studies in Semantic Modeling of Real-World Objects using Perceptual Anchoring
Open this publication in new window or tab >>Studies in Semantic Modeling of Real-World Objects using Perceptual Anchoring
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Autonomous agents, situated in real-world scenarios, need to maintain consonance between the perceived world (through sensory capabilities) and their internal representation of the world in the form of symbolic knowledge. An approach for modeling such representations of objects is through the concept of perceptual anchoring, which, by definition, handles the problem of creating and maintaining, in time and space, the correspondence between symbols and sensor data that refer to the same physical object in the external world.

The work presented in this thesis leverages notations found within perceptual anchoring to address the problem of real-world semantic world modeling, emphasizing, in particular, sensor-driven bottom-up acquisition of perceptual data. The proposed method for handling the attribute values that constitute the perceptual signature of an object is to first integrate and explore available resources of information, such as a Convolutional Neural Network (CNN) to classify objects on the perceptual level. In addition, a novel anchoring matching function is proposed. This function introduces both the theoretical procedure for comparing attribute values, as well as establishes the use of a learned model that approximates the anchoring matching problem. To verify the proposed method, an evaluation using human judgment to collect annotated ground truth data of real-world objects is further presented. The collected data is subsequently used to train and validate different classification algorithms, in order to learn how to correctly anchor objects, and thereby learn to invoke correct anchoring functionality.

There are, however, situations that are difficult to handle purely from the perspective of perceptual anchoring, e.g., situations where an object is moved during occlusion. In the absence of perceptual observations, it is necessary to couple the anchoring procedure with probabilistic object tracking to speculate about occluded objects, and hence, maintain a consistent world model. Motivated by the limitation in the original anchoring definition, which prohibited the modeling of the history of an object, an extension to the anchoring definition is also presented. This extension permits the historical trace of an anchored object to be maintained and used for the purpose of learning additional properties of an object, e.g., learning of the action applied to an object.

Place, publisher, year, edition, pages
Örebro: Örebro University, 2019. p. 93
Series
Örebro Studies in Technology, ISSN 1650-8580 ; 83
Keywords
Perceptual Anchoring, Semantic World Modeling, Sensor-Driven Acquisition of Data, Object Recognition, Object Classification, Symbol Grounding, Probabilistic Object Tracking
National Category
Information Systems
Identifiers
urn:nbn:se:oru:diva-73175 (URN)978-91-7529-283-0 (ISBN)
Public defence
2019-04-29, Örebro universitet, Teknikhuset, Hörsal T, Fakultetsgatan 1, Örebro, 13:15 (English)
Opponent
Supervisors
Available from: 2019-03-18 Created: 2019-03-18 Last updated: 2020-02-14Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Persson, AndreasAl Moubayed, SamerLoutfi, Amy
By organisation
School of Science and Technology
In the same journal
Cognitive Computation
Robotics and automation

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 919 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf