Change search
ReferencesLink to record
Permanent link

Direct link
Semantic Mapping with Mobile Robots
KTH, School of Computer Science and Communication (CSC), Computer Vision and Active Perception, CVAP.ORCID iD: 0000-0002-1396-0102
2011 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

After decades of unrealistic predictions and expectations, robots have finally escaped from industrial workplaces and made their way into our homes,offices, museums and other public spaces. These service robots are increasingly present in our environments and many believe that it is in the area ofservice and domestic robotics that we will see the largest growth within thenext few years. In order to realize the dream of robot assistants performing human-like tasks together with humans in a seamless fashion, we need toprovide them with the fundamental capability of understanding complex, dynamic and unstructured environments. More importantly, we need to enablethem the sharing of our understanding of space to permit natural cooper-ation. To this end, this thesis addresses the problem of building internalrepresentations of space for artificial mobile agents populated with humanspatial semantics as well as means for inferring that semantics from sensoryinformation. More specifically, an extensible approach to place classificationis introduced and used for mobile robot localization as well as categorizationand extraction of spatial semantic concepts from general place appearance andgeometry. The models can be incrementally adapted to the dynamic changesin the environment and employ efficient ways for cue integration, sensor fu-sion and confidence estimation. In addition, a system and representationalapproach to semantic mapping is presented. The system incorporates and in-tegrates semantic knowledge from multiple sources such as the geometry andgeneral appearance of places, presence of objects, topology of the environmentas well as human input. A conceptual map is designed and used for modelingand reasoning about spatial concepts and their relations to spatial entitiesand their semantic properties. Finally, the semantic mapping algorithm isbuilt into an integrated robotic system and shown to substantially enhancethe performance of the robot on the complex task of active object search. Thepresented evaluations show the effectiveness of the system and its underlyingcomponents and demonstrate applicability to real-world problems in realistichuman settings.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology , 2011. , xiii, 52 p.
Series
Trita-CSC-A, ISSN 1653-5723 ; 2011:10
National Category
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-34171ISBN: 978-91-7501-039-7OAI: oai:DiVA.org:kth-34171DiVA: diva2:419628
Public defence
2011-06-10, Sal F3, Lindstedtsvägen 26, KTH, Stockholm, 13:00 (English)
Opponent
Supervisors
Note
QC 20110527Available from: 2011-05-27 Created: 2011-05-27 Last updated: 2011-05-27Bibliographically approved
List of papers
1. A realistic benchmark for visual indoor place recognition
Open this publication in new window or tab >>A realistic benchmark for visual indoor place recognition
2010 (English)In: Robotics and Autonomous Systems, ISSN 0921-8890, E-ISSN 0921-8830, Vol. 58, no 1, 81-96 p.Article in journal (Refereed) Published
Abstract [en]

An important competence for a mobile robot system is the ability to localize and perform context interpretation. This is required to perform basic navigation and to facilitate local specific services. Recent advances in vision have made this modality a viable alternative to the traditional range sensors, and visual place recognition algorithms emerged as a useful and widely applied tool for obtaining information about robot's position. Several place recognition methods have been proposed using vision alone or combined with sonar and/or laser. This research calls for standard benchmark datasets for development, evaluation and comparison of solutions. To this end, this paper presents two carefully designed and annotated image databases augmented with an experimental procedure and extensive baseline evaluation. The databases were gathered in an uncontrolled indoor office environment using two mobile robots and a standard camera. The acquisition spanned across a time range of several months and different illumination and weather conditions. Thus, the databases are very well suited for evaluating the robustness of algorithms with respect to a broad range of variations, often occurring in real-world settings. We thoroughly assessed the databases with a purely appearance-based place recognition method based on support vector machines and two types of rich visual features (global and local).

Keyword
Visual place recognition, Robot topological localization, Standard, robotic benchmark, localization, appearance, map
National Category
Computer and Information Science
Identifiers
urn:nbn:se:kth:diva-19070 (URN)10.1016/j.robot.2009.07.025 (DOI)000272968200008 ()2-s2.0-70450221694 (ScopusID)
Funder
Swedish Research Council, 2005-3600-Complex
Note
QC 20100525Available from: 2010-08-05 Created: 2010-08-05 Last updated: 2011-12-09Bibliographically approved
2. The more you learn, the less you store: Memory-controlled incremental SVM for visual place recognition
Open this publication in new window or tab >>The more you learn, the less you store: Memory-controlled incremental SVM for visual place recognition
2010 (English)In: Image and Vision Computing, ISSN 0262-8856, Vol. 28, no 7, 1080-1097 p.Article in journal (Refereed) Published
Abstract [en]

The capability to learn from experience is a key property for autonomous cognitive systems working in realistic settings. To this end, this paper presents an SVM-based algorithm, capable of learning model representations incrementally while keeping under control memory requirements. We combine an incremental extension of SVMs [43] with a method reducing the number of support vectors needed to build the decision function without any loss in performance [15] introducing a parameter which permits a user-set trade-off between performance and memory. The resulting algorithm is able to achieve the same recognition results as the original incremental method while reducing the memory growth. Our method is especially suited to work for autonomous systems in realistic settings. We present experiments on two common scenarios in this domain: adaptation in presence of dynamic changes and transfer of knowledge between two different autonomous agents, focusing in both cases on the problem of visual place recognition applied to mobile robot topological localization. Experiments in both scenarios clearly show the power of our approach.

Keyword
Incremental learning, Knowledge transfer, Support vector machines, Place recognition, Visual robot localization
National Category
Computer and Information Science
Identifiers
urn:nbn:se:kth:diva-27548 (URN)10.1016/j.imavis.2010.01.015 (DOI)000278233900003 ()2-s2.0-77950866368 (ScopusID)
Funder
EU, FP7, Seventh Framework Programme, 215181Swedish Research Council, 2005-3600-Complex
Note
QC 20101216Available from: 2010-12-16 Created: 2010-12-13 Last updated: 2011-05-27Bibliographically approved
3. Confidence-based cue integration for visual place recognition
Open this publication in new window or tab >>Confidence-based cue integration for visual place recognition
2007 (English)In: 2007 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS, VOLS 1-9, NEW YORK: IEEE , 2007, 2400-2407 p.Conference paper (Refereed)
Abstract [en]

A distinctive feature of intelligent systems is their capability to analyze their level of expertise for a given task; in other words, they know what they know. As a way towards this ambitious goal, this paper presents a recognition algorithm able to measure its own level of confidence and, in case of uncertainty, to seek for extra information so to increase its own knowledge and ultimately achieve better performance. We focus on the visual place recognition problem for topological localization, and we take an SVM approach. We propose a new method for measuring the confidence level of the classification output, based on the distance of a test image and the average distance of training vectors. This method is combined with a discriminative accumulation scheme for cue integration. We show with extensive experiments that the resulting algorithm achieves better performances for two visual cues than the classic single cue SVM on the same task, while minimising the computational load. More important, our method provides a reliable measure of the level of confidence of the decision.

Place, publisher, year, edition, pages
NEW YORK: IEEE, 2007
Keyword
Arsenic compounds, Evolutionary algorithms, Intelligent robots, Robotics, Support vector machines
National Category
Computer and Information Science
Identifiers
urn:nbn:se:kth:diva-34155 (URN)10.1109/IROS.2007.4399493 (DOI)000254073201157 ()2-s2.0-51349101162 (ScopusID)978-1-4244-0911-2 (ISBN)
Conference
IEEE/RSJ International Conference on Intelligent Robots and Systems. San Diego, CA. OCT 29-NOV 02, 2007
Available from: 2011-05-27 Created: 2011-05-27 Last updated: 2011-09-14Bibliographically approved
4. Multi-modal Semantic Place Classification
Open this publication in new window or tab >>Multi-modal Semantic Place Classification
2010 (English)In: The international journal of robotics research, ISSN 0278-3649, E-ISSN 1741-3176, Vol. 29, no 2-3, 298-320 p.Article in journal (Refereed) Published
Abstract [en]

The ability to represent knowledge about space and its position therein is crucial for a mobile robot. To this end, topological and semantic descriptions are gaining popularity for augmenting purely metric space representations. In this paper we present a multi-modal place classification system that allows a mobile robot to identify places and recognize semantic categories in an indoor environment. The system effectively utilizes information from different robotic sensors by fusing multiple visual cues and laser range data. This is achieved using a high-level cue integration scheme based on a Support Vector Machine (SVM) that learns how to optimally combine and weight each cue. Our multi-modal place classification approach can be used to obtain a real-time semantic space labeling system which integrates information over time and space. We perform an extensive experimental evaluation of the method for two different platforms and environments, on a realistic off-line database and in a live experiment on an autonomous robot. The results clearly demonstrate the effectiveness of our cue integration scheme and its value for robust place classification under varying conditions.

Keyword
recognition, sensor fusion, localization, multi-modal place, classification, sensor and cue integration, semantic annotation of space, image, representations, vision
National Category
Computer and Information Science
Identifiers
urn:nbn:se:kth:diva-19266 (URN)10.1177/0278364909356483 (DOI)000275038200010 ()2-s2.0-77949376736 (ScopusID)
Funder
Swedish Research Council, 2005-3600-Complex
Note
QC 20100525Available from: 2010-08-05 Created: 2010-08-05 Last updated: 2011-12-09Bibliographically approved
5. Representing spatial knowledge in mobile cognitive systems
Open this publication in new window or tab >>Representing spatial knowledge in mobile cognitive systems
Show others...
2010 (English)In: Intelligent Autonomous Systems 11, IAS 2010, 2010, 133-142 p.Conference paper (Refereed)
Abstract [en]

A cornerstone for cognitive mobile agents is to represent the vast body of knowledge about space in which they operate. In order to be robust and efficient, such representation must address requirements imposed on the integrated system as a whole, but also resulting from properties of its components. In this paper, we carefully analyze the problem and design a structure of a spatial knowledge representation for a cognitive mobile system. Our representation is layered and represents knowledge at different levels of abstraction. It deals with complex, crossmodal, spatial knowledge that is inherently uncertain and dynamic. Furthermore, it incorporates discrete symbols that facilitate communication with the user and components of a cognitive system. We present the structure of the representation and propose concrete instantiations.

Keyword
Body of knowledge, Cross-modal, Integrated systems, Levels of abstraction, Mobile systems, Spatial knowledge, Spatial knowledge representations
National Category
Computer and Information Science
Identifiers
urn:nbn:se:kth:diva-34157 (URN)10.3233/978-1-60750-613-3-133 (DOI)2-s2.0-84871631070 (ScopusID)978-160750612-6 (ISBN)
Conference
11th International Conference on Intelligent Autonomous Systems, IAS 2010; Ottawa, ON; Canada; 30 August 2010 through 1 September 2010
Funder
EU, FP7, Seventh Framework Programme, CogX
Note

QC 20110527

Available from: 2011-05-27 Created: 2011-05-27 Last updated: 2014-08-15Bibliographically approved
6. Understanding the Real World: Combining Objects, Appearance, Geometry and Topology for Semantic Mapping
Open this publication in new window or tab >>Understanding the Real World: Combining Objects, Appearance, Geometry and Topology for Semantic Mapping
2011 (English)Report (Other academic)
Abstract [en]

A cornerstone for mobile robots operating in man-made environments and interacting with humans is representing and understanding the human semantic concepts of space. In this report, we present a multi-layered semantic mapping algorithm able to combine information about the existence of objects in the environment with knowledge about the topology and semantic properties of space such as room size, shape and general appearance. We use it to infer semantic categories of rooms and predict existence of objects and values of other spatial properties. We perform experiments offline and online on a mobile robot showing the efficiency and usefulness of our system.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2011. 20 p.
National Category
Computer and Information Science
Identifiers
urn:nbn:se:kth:diva-34158 (URN)
Note
QC 20110527Available from: 2011-05-27 Created: 2011-05-27 Last updated: 2011-05-27Bibliographically approved
7. Exploiting probabilistic knowledge under uncertain sensing for efficient robot behaviour
Open this publication in new window or tab >>Exploiting probabilistic knowledge under uncertain sensing for efficient robot behaviour
Show others...
2011 (English)In: 22nd International Joint Conference on Artificial Intelligence, 2011Conference paper (Refereed)
Abstract [en]

Robots must perform tasks efficiently and reliably while acting under uncertainty. One way to achieve efficiency is to give the robot common-sense knowledge about the structure of the world. Reliable robot behaviour can be achieved by modelling the uncertainty in the world probabilistically. We present a robot system that combines these two approaches and demonstrate the improvements in efficiency and reliability that result. Our first contribution is a probabilistic relational model integrating common-sense knowledge about the world in general, with observations of a particularenvironment. Our second contribution is a continual planning system which isable to plan in the large problems posed by that model, by automatically switching between decision-theoretic and classical procedures. We evaluate our system on objects earch tasks in two different real-world indoor environments. By reasoning about the trade-offs between possible courses of action with different informational effects, and exploiting the cues and general structures of those environments, our robot is able to consistently demonstrate efficient and reliable goal-directed behaviour.

National Category
Computer and Information Science
Identifiers
urn:nbn:se:kth:diva-34159 (URN)2-s2.0-84881058154 (ScopusID)
Conference
22nd International Joint Conference on Artificial Intelligence (IJCAI’ 11), Barcelona, Spain, July 2011
Note
QC 20110527Available from: 2011-05-27 Created: 2011-05-27 Last updated: 2011-05-27Bibliographically approved

Open Access in DiVA

fulltext(1687 kB)1849 downloads
File information
File name FULLTEXT01.pdfFile size 1687 kBChecksum SHA-512
dce0c4f607f0be91608089ce749fda734485fa1ddee8bcafde4224adb4c0c6a9e12aa4f33b66b3f3bc6b06390eb7b0548a4ef9ec90aeb6c66489d6dbb8bb8288
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Pronobis, Andrzej
By organisation
Computer Vision and Active Perception, CVAP
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 1849 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 1460 hits
ReferencesLink to record
Permanent link

Direct link