Change search
ReferencesLink to record
Permanent link

Direct link
Finding Your Way from the Bed to the Kitchen: Re-enacting and Re-combining Sensorimotor Episodes Learned from Human Demonstration
University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. (Interaction Lab)ORCID iD: 0000-0002-6568-9342
University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. (Interaction Lab)
University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. Göteborgs Universitet, Tillämpad IT. (Interaction Lab)
University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. Department of Computer and Information Science, Linköping University. (Interaction Lab)ORCID iD: 0000-0001-6883-2450
2016 (English)In: Frontiers in Robotics and AI, ISSN 1534-5955, E-ISSN 1662-3770, Vol. 3, no 9Article in journal (Refereed) Published
Abstract [en]

Several simulation theories have been proposed as an explanation for how humans and other agents internalize an "inner world" that allows them to simulate interactions with the external real world - prospectively and retrospectively. Such internal simulation of interaction with the environment has been argued to be a key mechanism behind mentalizing and planning. In the present work, we study internal simulations in a robot acting in a simulated human environment. A model of sensory-motor interactions with the environment is generated from human demonstrations, and tested on a Robosoft Kompai robot. The model is used as a controller for the robot, reproducing the demonstrated behavior. Information from several different demonstrations is mixed, allowing the robot to produce novel paths through the environment, towards a goal specified by top-down contextual information. 

The robot model is also used in a covert mode, where actions are inhibited and perceptions are generated by a forward model. As a result, the robot generates an internal simulation of the sensory-motor interactions with the environment. Similar to the overt mode, the model is able to reproduce the demonstrated behavior as internal simulations. When experiences from several demonstrations are combined with a top-down goal signal, the system produces internal simulations of novel paths through the environment. These results can be understood as the robot imagining an "inner world" generated from previous experience, allowing it to try out different possible futures without executing actions overtly.

We found that the success rate in terms of reaching the specified goal was higher during internal simulation, compared to overt action. These results are linked to a reduction in prediction errors generated during covert action. Despite the fact that the model is quite successful in terms of generating covert behavior towards specified goals, internal simulations display different temporal distributions compared to their overt counterparts. Links to human cognition and specifically mental imagery are discussed.

Place, publisher, year, edition, pages
Lausanne, Switzerland: Frontiers , 2016. Vol. 3, no 9
Keyword [en]
compositionality, internal simulation, learning from demonstration, simulation theory, predictive sequence learning, prospection, embodied cognition, imagination, representation
National Category
Computer and Information Science
Research subject
URN: urn:nbn:se:his:diva-12075DOI: 10.3389/frobt.2016.00009ISI: 000382475100002OAI: diva2:915452
Available from: 2016-03-30 Created: 2016-03-30 Last updated: 2016-09-30Bibliographically approved

Open Access in DiVA

fulltext(14879 kB)93 downloads
File information
File name FULLTEXT01.pdfFile size 14879 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Search in DiVA

By author/editor
Billing, ErikSvensson, HenrikLowe, RobertZiemke, Tom
By organisation
School of InformaticsThe Informatics Research Centre
In the same journal
Frontiers in Robotics and AI
Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 93 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 350 hits
ReferencesLink to record
Permanent link

Direct link