Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Object Instance Detection and Dynamics Modeling in a Long-Term Mobile Robot Context
KTH, School of Computer Science and Communication (CSC), Robotics, perception and learning, RPL.ORCID iD: 0000-0003-1189-6634
2017 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

In the last years, simple service robots such as autonomous vacuum cleaners and lawn mowers have become commercially available and increasingly common. The next generation of service robots should perform more advanced tasks, such as to clean up objects. Robots then need to learn to robustly navigate, and manipulate, cluttered environments, such as an untidy living room. In this thesis, we focus on representations for tasks such as general cleaning and fetching of objects. We discuss requirements for these specific tasks, and argue that solving them would be generally useful, because of their object-centric nature. We rely on two fundamental insights in our approach to understand environments on a fine-grained level. First, many of today's robot map representations are limited to the spatial domain, and ignore that there is a time axis that constrains how much an environment may change during a given period. We argue that it is of critical importance to also consider the temporal domain. By studying the motion of individual objects, we can enable tasks such as general cleaning and object fetching. The second insight comes from that mobile robots are becoming more robust. They can therefore collect large amounts of data from those environments. With more data, unsupervised learning of models becomes feasible, allowing the robot to adapt to changes in the environment, and to scenarios that the designer could not foresee. We view these capabilities as vital for robots to become truly autonomous. The combination of unsupervised learning and dynamics modelling creates an interesting symbiosis: the dynamics vary between different environments and between the objects in one environment, and learning can capture these variations. A major difficulty when modeling environment dynamics is that the whole environment can not be observed at one time, since the robot is moving between different places. We demonstrate how this can be dealt with in a principled manner, by modeling several modes of object movement. We also demonstrate methods for detection and learning of objects and structures in the static parts of the maps. Using the complete system, we can represent and learn many aspects of the full environment. In real-world experiments, we demonstrate that our system can keep track of varied objects in large and highly dynamic environments.​

Abstract [sv]

Under de senaste åren har enklare service-robotar, såsom autonoma dammsugare och gräsklippare, börjat säljas, och blivit alltmer vanliga. Nästa generations service-robotar förväntas utföra mer komplexa uppgifter, till exempel att städa upp utspridda föremål i ett vardagsrum. För att uppnå detta måste robotarna kunna navigera i ostrukturerade miljöer, och förstå hur de kan bringas i ordning. I denna avhandling undersöker vi abstrakta representationer som kan förverkliga generalla städrobotar, samt robotar som kan hämta föremål. Vi diskuterar vad dessa specifika tillämpningar kräver i form av representationer, och argumenterar för att en lösning på dessa problem vore mer generellt applicerbar på grund av uppgifternas föremåls-centrerade natur. Vi närmar oss uppgiften genom två viktiga insikter. Till att börja medär många av dagens robot-representationer begränsade till rumsdomänen. De utelämnar alltså att modellera den variation som sker över tiden, och utnyttjar därför inte att rörelsen som kan ske under en given tidsperiod är begränsad. Vi argumenterar för att det är kritiskt att också inkorperara miljöns rörelse i robotens modell. Genom att modellera omgivningen på en föremåls-nivå möjliggörs tillämpningar som städning och hämtning av rörliga objekt. Den andra insikten kommer från att mobila robotar nu börjar bli så robusta att de kan patrullera i en och samma omgivning under flera månader. Dekan därför samla in stora mängder data från enskilda omgivningar. Med dessa stora datamängder börjar det bli möjligt att tillämpa så kallade "unsupervised learning"-metoder för att lära sig modeller av enskilda miljöer utan mänsklig inblandning. Detta tillåter robotarna att anpassa sig till förändringar i omgivningen, samt att lära sig koncept som kan vara svåra att förutse på förhand. Vi ser detta som en grundläggande förmåga hos en helt autonom robot. Kombinationen av unsupervised learning och modellering av omgivningens dynamik är intressant. Eftersom dynamiken varierar mellan olika omgivningar,och mellan olika objekt, kan learning hjälpa oss att fånga dessa variationer,och skapa mer precisa dynamik-modeller. Något som försvårar modelleringen av omgivningens dynamik är att roboten inte kan observera hela omgivningen på samma gång. Detta betyder att saker kan flyttas långa sträckor mellan två observationer. Vi visar hur man kan adressera detta i modellen genom att inlemma flera olika sätt som ett föremål kan flyttas på. Det resulterande systemet är helt probabilistiskt, och kan hålla reda på samtliga föremål i robotens omgivning. Vi demonstrerar även metoder för att upptäcka och lära sig föremål i den statiska delen av omgivningen. Med det kombinerade systemet kan vi således representera och lära oss många aspekter av robotens omgivning. Genom experiment i mänskliga miljöer visar vi att systemet kan hålla reda på olika sorters föremål i stora, och dynamiska, miljöer.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2017. , p. 58
Series
TRITA-CSC-A, ISSN 1653-5723 ; 2017:27
Keywords [en]
robotics, long-term, mapping, tracking, unsupervised learning, estimation, object modeling
National Category
Robotics
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-219813ISBN: 978-91-7729-638-6 (print)OAI: oai:DiVA.org:kth-219813DiVA, id: diva2:1165380
Public defence
2018-01-19, F3, Lindstedtsvägen 26, KTH Campus, Stockholm, 14:00 (English)
Opponent
Supervisors
Note

QC 20171213

Available from: 2017-12-13 Created: 2017-12-13 Last updated: 2017-12-14Bibliographically approved
List of papers
1. Efficient retrieval of arbitrary objects from long-term robot observations
Open this publication in new window or tab >>Efficient retrieval of arbitrary objects from long-term robot observations
2017 (English)In: Robotics and Autonomous Systems, ISSN 0921-8890, E-ISSN 1872-793X, Vol. 91, p. 139-150Article in journal (Refereed) Published
Abstract [en]

We present a novel method for efficient querying and retrieval of arbitrarily shaped objects from large amounts of unstructured 3D point cloud data. Our approach first performs a convex segmentation of the data after which local features are extracted and stored in a feature dictionary. We show that the representation allows efficient and reliable querying of the data. To handle arbitrarily shaped objects, we propose a scheme which allows incremental matching of segments based on similarity to the query object. Further, we adjust the feature metric based on the quality of the query results to improve results in a second round of querying. We perform extensive qualitative and quantitative experiments on two datasets for both segmentation and retrieval, validating the results using ground truth data. Comparison with other state of the art methods further enforces the validity of the proposed method. Finally, we also investigate how the density and distribution of the local features within the point clouds influence the quality of the results.

Place, publisher, year, edition, pages
ELSEVIER SCIENCE BV, 2017
Keywords
Mapping, Mobile robotics, Point cloud, Segmentation, Retrieval
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-205426 (URN)10.1016/j.robot.2016.12.013 (DOI)000396949800012 ()2-s2.0-85015091269 (Scopus ID)
Note

QC 20170522

Available from: 2017-05-22 Created: 2017-05-22 Last updated: 2018-01-13Bibliographically approved
2. Detection and Tracking of General Movable Objects in Large 3D Maps
Open this publication in new window or tab >>Detection and Tracking of General Movable Objects in Large 3D Maps
(English)Manuscript (preprint) (Other academic)
Abstract [en]

This paper studies the problem of detection and tracking of general objects with long-term dynamics, observed by a mobile robot moving in a large environment. A key problem is that due to the environment scale, it can only observe a subset of the objects at any given time. Since some time passes between observations of objects in different places, the objects might be moved when the robot is not there. We propose a model for this movement in which the objects typically only move locally, but with some small probability they jump longer distances, through what we call global motion. For filtering, we decompose the posterior over local and global movements into two linked processes. The posterior over the global movements and measurement associations is sampled, while we track the local movement analytically using Kalman filters. This novel filter is evaluated on point cloud data gathered autonomously by a mobile robot over an extended period of time. We show that tracking jumping objects is feasible, and that the proposed probabilistic treatment outperforms previous methods when applied to real world data. The key to efficient probabilistic tracking in this scenario is focused sampling of the object posteriors.

Keywords
mobile robot, multi-target tracking, movable objects, mapping
National Category
Robotics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-219817 (URN)
Note

Submitted to IEEE Transactions on Robotics

QC 20171215

Available from: 2017-12-13 Created: 2017-12-13 Last updated: 2017-12-15Bibliographically approved
3. Multiple Object Detection, Tracking and Long-Term Dynamics Learning in Large 3D Maps
Open this publication in new window or tab >>Multiple Object Detection, Tracking and Long-Term Dynamics Learning in Large 3D Maps
(English)Manuscript (preprint) (Other academic)
Abstract [en]

In this work, we present a method for tracking and learning the dynamics of all objects in a large scale robot environment. A mobile robot patrols the environment and visits the different locations one by one. Movable objects are discovered by change detection, and tracked throughout the robot deployment. For tracking, we extend our previous Rao-Blackwellized particle filter with birth and death processes, enabling the method to handle an arbitrary number of objects. Target births and associations are sampled using Gibbs sampling. The parameters of the system are then learnt using the Expectation Maximization algorithm in an unsupervised fashion. The system therefore enables learning of the dynamics of one particular environment, and of its objects. The algorithm is evaluated on data collected autonomously by a mobile robot in an office environment during a real-world deployment. We show that the algorithm automatically identifies and tracks the moving objects within 3D maps and infers plausible dynamics models, significantly decreasing the modeling bias of our previous work. The proposed method represents an improvement over previous methods for environment dynamics learning as it allows for learning of fine grained processes.

Keywords
mobile robot, multi-target tracking, movable objects, mapping, learning, dynamics
National Category
Robotics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-219818 (URN)
Note

To be submitted to IEEE Transactions on Robotics

QC 20171215

Available from: 2017-12-13 Created: 2017-12-13 Last updated: 2017-12-15Bibliographically approved
4. Querying 3D Data by Adjacency Graphs
Open this publication in new window or tab >>Querying 3D Data by Adjacency Graphs
2015 (English)In: Computer Vision Systems / [ed] Nalpantidis, Lazaros and Krüger, Volker and Eklundh, Jan-Olof and Gasteratos, Antonios, Springer Publishing Company, 2015, p. 243-252Chapter in book (Refereed)
Abstract [en]

The need for robots to search the 3D data they have saved is becoming more apparent. We present an approach for finding structures in 3D models such as those built by robots of their environment. The method extracts geometric primitives from point cloud data. An attributed graph over these primitives forms our representation of the surface structures. Recurring substructures are found with frequent graph mining techniques. We investigate if a model invariant to changes in size and reflection using only the geometric information of and between primitives can be discriminative enough for practical use. Experiments confirm that it can be used to support queries of 3D models.

Place, publisher, year, edition, pages
Springer Publishing Company, 2015
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 9163
Keywords
Object retrieval, 3D data, point cloud
National Category
Computer Sciences Robotics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-177976 (URN)10.1007/978-3-319-20904-3_23 (DOI)000364183300023 ()2-s2.0-84949036031 (Scopus ID)978-3-319-20904-3; 978-3-319-20903-6 (ISBN)
Conference
10th International Conference on Computer Vision Systems (ICVS), JUL 06-09, 2015, Copenhagen, DENMARK
Funder
EU, FP7, Seventh Framework Programme, 600623
Note

QC 20160321

Available from: 2015-12-02 Created: 2015-11-30 Last updated: 2018-01-10Bibliographically approved

Open Access in DiVA

fulltext(19827 kB)154 downloads
File information
File name FULLTEXT01.pdfFile size 19827 kBChecksum SHA-512
453080d2aef94cb2db3a70f0bb597009b85234166952d37001d07aa5594213ca147e3a201df6fa504cf5104e70d3f051b7e7a12e271d4af3ed93fddb4795bcbe
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Bore, Nils
By organisation
Robotics, perception and learning, RPL
Robotics

Search outside of DiVA

GoogleGoogle Scholar
Total: 154 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 798 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf