Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Robot Manipulation Planning Among Obstacles: Grasping, Placing and Rearranging
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0002-5821-7435
2020 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis presents planning algorithms for three different robot manipulation tasks: fingertip grasping, object placing and rearranging. Herein, we place special attention on addressing these tasks in the presence of obstacles. Obstacles are frequently encountered in human-centered environments and constrain a robot's motion and ability to manipulate objects. In narrow shelves, for example, even the common task of pick-and-place becomes challenging. A shelf is difficult to navigate and many potential grasps and placements are inaccessible. Hence, to solve such tasks, specialized manipulation planning algorithms are required that can cope with the presence of obstacles.

For fingertip grasping, we first present a framework to learn models that encode which grasps a given dexterous robot hand can reach. These models are then used to facilitate planning and optimization of fingertip grasps. Next, we address the presence of obstacles and integrate fingertip grasp and motion planning to generate grasps that are reachable by a robot in complex scenes.

For object placing, we first present an algorithm that plans the placement of a grasped object among obstacles so that a user-given placement objective is maximized. We then extend this algorithm, and incorporate planning in-hand manipulation to increase the set of placements a robot can reach.

Lastly, we go beyond pure collision avoidance and study object rearrangement planning. Specifically, we consider the special case of non-prehensile rearrangement, where a robot rearranges multiple objects through pushing. First, we present how a kinodynamic motion planning algorithm can be augmented with learned models to rearrange a few target objects among movable and static obstacles. We then present how we can use Monte Carlo tree search to solve a large-scale rearrangement problem, where a robot is tasked to spatially sort many objects according to a user-assigned class membership.

Abstract [sv]

Den här avhandlingen presenterar planeringsalgoritmer för tre olika ma-nipulationsuppgifter för robotar i närheten av hinder: att greppa med hjälpav fingertopparna, att placera objekt och att arrangera om flertalet objekt iolika konfigurationer. Hinder finns oftast i människors miljöer och begränsaren robots rörelse och förmåga att manipulera objekt. Till exempel är den tillsynes enkla uppgiften att hämta och placera objekt i smala hyllor mycketsvår för robotar. Planering av manipulering i detta fall blir svårt därför attmånga rörelser och grepp kommer att kollidera med hinder. För att lösa dessauppgifter behöver man speciella algoritmer som kan hantera dessa hinder imiljön.

För grepp med fingertoppar presenterar vi ett ramverk för inlärning avmodeller som representerar vilka grepp en robothand kan utföra. Dessa mo-deller används sedan för att planera och optimera grepp som sker med hjälpav fingertopparna. Därefter integreras planeringsalgoritmen med rörelsepla-nering för att kunna planera fingertoppsgrepp där också hinder existerar iobjektets närhet.

För objektsplaceringar presenterar vi ett ramverk för att planera hur enrobot kan transportera ett greppat objekt till en placering som optimerar engiven kostnadsfunktion. Därefter utvecklar vi detta ramverk för att visa hurså kallad in-hand manipulation (att byta grepp utan att släppa objektet) kanöka antalet möjliga placeringar som roboten kan utföra.

Till slut utökar vi våra metoder bortom endast undvikande av kollisionoch studerar planering för omarrangering av objekt. Vi studerar det speciellafallet non-prehensile rearrangement där en robot måste ordna om flertaletobjekt genom att skjuta dem framför sig. Vi presenterar först hur en kino-dynamisk rörelseplaneringsalgoritm kan förbättras genom inlärda modeller.Detta görs i syftet att hitta rörelsesekvenser som kan ordna flertalet objekt tillgivna konfigurationer. Sedan presenterar vi hur vi kan använda Monte Carlo-trädsök för att lösa ett stort sådant problem där objekt ska om-arrangeras. Idetta problem ska roboten sortera objekt enligt kategorier som en användarehar specifierat.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2020. , p. 52
Series
TRITA-EECS-AVL ; 2020:6
Keywords [en]
robot manipulation planning, sampling-based planning, fingertip grasp planning, placement planning, rearrangement planning
National Category
Computer Sciences Computer Vision and Robotics (Autonomous Systems)
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-266792ISBN: 978-91-7873-411-5 (print)OAI: oai:DiVA.org:kth-266792DiVA, id: diva2:1387713
Public defence
2020-02-17, F3, Lindstedtsvägen 26, Stockholm, 13:00 (English)
Opponent
Supervisors
Note

QC 20200124

Available from: 2020-01-24 Created: 2020-01-22 Last updated: 2020-01-24Bibliographically approved
List of papers
1. On the Evolution of Fingertip Grasping Manifolds
Open this publication in new window or tab >>On the Evolution of Fingertip Grasping Manifolds
Show others...
2016 (English)In: IEEE International Conference on Robotics and Automation, IEEE Robotics and Automation Society, 2016, p. 2022-2029, article id 7487349Conference paper, Published paper (Refereed)
Abstract [en]

Efficient and accurate planning of fingertip grasps is essential for dexterous in-hand manipulation. In this work, we present a system for fingertip grasp planning that incrementally learns a heuristic for hand reachability and multi-fingered inverse kinematics. The system consists of an online execution module and an offline optimization module. During execution the system plans and executes fingertip grasps using Canny’s grasp quality metric and a learned random forest based hand reachability heuristic. In the offline module, this heuristic is improved based on a grasping manifold that is incrementally learned from the experiences collected during execution. The system is evaluated both in simulation and on a SchunkSDH dexterous hand mounted on a KUKA-KR5 arm. We show that, as the grasping manifold is adapted to the system’s experiences, the heuristic becomes more accurate, which results in an improved performance of the execution module. The improvement is not only observed for experienced objects, but also for previously unknown objects of similar sizes.

Place, publisher, year, edition, pages
IEEE Robotics and Automation Society, 2016
Keywords
Fingertip Grasping, Grasping Manifold
National Category
Robotics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-187060 (URN)10.1109/ICRA.2016.7487349 (DOI)000389516201112 ()2-s2.0-84977471090 (Scopus ID)978-1-4673-8026-3 (ISBN)
Conference
IEEE International Conference on Robotics and Automation
Projects
RobDream
Note

QC 20160517

Available from: 2016-05-16 Created: 2016-05-16 Last updated: 2020-01-22Bibliographically approved
2. Integrating motion and hierarchical fingertip grasp planning
Open this publication in new window or tab >>Integrating motion and hierarchical fingertip grasp planning
2017 (English)In: 2017 IEEE International Conference on Robotics and Automation (ICRA), Institute of Electrical and Electronics Engineers (IEEE), 2017, p. 3439-3446, article id 7989392Conference paper (Refereed)
Abstract [en]

In this work, we present an algorithm that simultaneously searches for a high quality fingertip grasp and a collision-free path for a robot hand-arm system to achieve it. The algorithm combines a bidirectional sampling-based motion planning approach with a hierarchical contact optimization process. Rather than tackling these problems in a decoupled manner, the grasp optimization is guided by the proximity to collision-free configurations explored by the motion planner. We implemented the algorithm for a 13-DoF manipulator and show that it is capable of efficiently planning reachable high quality grasps in cluttered environments. Further, we show that our algorithm outperforms a decoupled integration in terms of planning runtime.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2017
Series
Proceedings - IEEE International Conference on Robotics and Automation, ISSN 1050-4729
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-213525 (URN)10.1109/ICRA.2017.7989392 (DOI)2-s2.0-85027994091 (Scopus ID)9781509046331 (ISBN)
Conference
2017 IEEE International Conference on Robotics and Automation, ICRA 2017, Marina Bay Sands International Convention Centre Singapore, Singapore, 29 May 2017 through 3 June 2017
Note

QC 20170904

Available from: 2017-09-04 Created: 2017-09-04 Last updated: 2020-01-22Bibliographically approved
3. Object Placement Planning and Optimization for Robot Manipulators
Open this publication in new window or tab >>Object Placement Planning and Optimization for Robot Manipulators
2019 (English)In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019), 2019Conference paper, Published paper (Refereed)
Abstract [en]

We address the problem of planning the placement of a rigid object with a dual-arm robot in a cluttered environment. In this task, we need to locate a collision-free pose for the object that a) facilitates the stable placement of the object, b) is reachable by the robot and c) optimizes a user-given placement objective. In addition, we need to select which robot arm to perform the placement with. To solve this task, we propose an anytime algorithm that integrates sampling-based motion planning with a novel hierarchical search for suitable placement poses. Our algorithm incrementally produces approach motions to stable placement poses, reaching placements with better objective as runtime progresses. We evaluate our approach for two different placement objectives, and observe its effectiveness even in challenging scenarios.

Keywords
Motion planning, Object placing
National Category
Computer Vision and Robotics (Autonomous Systems)
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-264015 (URN)
Conference
International Conference on Intelligent Robots and Systems (IROS), Macau, China, November 4-8, 2019
Funder
Swedish Foundation for Strategic Research Knut and Alice Wallenberg Foundation
Note

QC 20191210

Available from: 2019-11-20 Created: 2019-11-20 Last updated: 2020-01-31Bibliographically approved
4. Placing Objects with prior In-Hand Manipulation using Dexterous Manipulation Graphs
Open this publication in new window or tab >>Placing Objects with prior In-Hand Manipulation using Dexterous Manipulation Graphs
Show others...
2019 (English)Conference paper, Published paper (Refereed)
Abstract [en]

We address the problem of planning the placement of a grasped object with a robot manipulator. More specifically, the robot is tasked to place the grasped object such that a placement preference function is maximized. For this, we present an approach that uses in-hand manipulation to adjust the robot’s initial grasp to extend the set of reachable placements. Given an initial grasp, the algorithm computes a set of grasps that can be reached by pushing and rotating the object in-hand. With this set of reachable grasps, it then searches for a stable placement that maximizes the preference function. If successful it returns a sequence of in-hand pushes to adjust the initial grasp to a more advantageous grasp together with a transport motion that carries the object to the placement. We evaluate our algorithm’s performance on various placing scenarios, and observe its effectiveness also in challenging scenes containing many obstacles. Our experiments demonstrate that re-grasping with in-hand manipulation increases the quality of placements the robot can reach. In particular, it enables the algorithm to find solutions in situations where safe placing with the initial grasp wouldn’t be possible.

National Category
Robotics
Identifiers
urn:nbn:se:kth:diva-262882 (URN)
Conference
IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), Toronto, Canada, October 15-17, 2019.
Note

QC 20191115

Available from: 2019-10-22 Created: 2019-10-22 Last updated: 2020-01-22Bibliographically approved
5. Learning Manipulation States and Actions for Efficient Non-prehensile Rearrangement Planning
Open this publication in new window or tab >>Learning Manipulation States and Actions for Efficient Non-prehensile Rearrangement Planning
Show others...
(English)Manuscript (preprint) (Other academic)
Abstract [en]

This paper addresses non-prehensile rearrangement planning problems where a robot is tasked to rearrange objects among obstacles on a planar surface. We present an efficient planning algorithm that is designed to impose few assumptions on the robot's non-prehensile manipulation abilities and is simple to adapt to different robot embodiments. For this, we combine sampling-based motion planning with reinforcement learning and generative modeling. Our algorithm explores the composite configuration space of objects and robot as a search over robot actions, forward simulated in a physics model. This search is guided by a generative model that provides robot states from which an object can be transported towards a desired state, and a learned policy that provides corresponding robot actions. As an efficient generative model, we apply Generative Adversarial Networks. We implement and evaluate our approach for robots endowed with configuration spaces in SE(2). We demonstrate empirically the efficacy of our algorithm design choices and observe more than 2x speedup in planning time on various test scenarios compared to a state-of-the-art approach.

Keywords
Rearrangement planning, manipulation planning, robot pushing, generative adversarial networks
National Category
Computer Vision and Robotics (Autonomous Systems)
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-266790 (URN)
Note

QC 20200127

Available from: 2020-01-22 Created: 2020-01-22 Last updated: 2020-01-31Bibliographically approved
6. Multi-Object Rearrangement with Monte Carlo Tree Search: A Case Study on Planar Nonprehensile Sorting
Open this publication in new window or tab >>Multi-Object Rearrangement with Monte Carlo Tree Search: A Case Study on Planar Nonprehensile Sorting
Show others...
(English)Manuscript (preprint) (Other academic)
Abstract [en]

In this work, we address a planar non-prehensile sorting task. Here, a robot needs to push many densely packed objects belonging to different classes into a configuration where these classes are clearly separated from each other. To achieve this, we propose to employ Monte Carlo tree search equipped with a task-specific heuristic function. We evaluate the algorithm on various simulated sorting tasks and observe its effectiveness in reliably sorting up to 40 convex objects. In addition, we observe that the algorithm is capable to also sort non-convex objects, as well as convex objects in the presence of immovable obstacles.

National Category
Computer Vision and Robotics (Autonomous Systems)
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-266783 (URN)
Note

QC 20200124

Available from: 2020-01-21 Created: 2020-01-21 Last updated: 2020-01-31Bibliographically approved

Open Access in DiVA

fulltext(2560 kB)98 downloads
File information
File name FULLTEXT01.pdfFile size 2560 kBChecksum SHA-512
75e2a4ba9f1799d9e006de42348a688e740c2b7e75963364b2aca93a2cecb799a050cc654562ab1edd7bf652cce432f89e42a2ed289da5a79b0b08ba7a6a96eb
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Haustein, Joshua Alexander
By organisation
Robotics, Perception and Learning, RPL
Computer SciencesComputer Vision and Robotics (Autonomous Systems)

Search outside of DiVA

GoogleGoogle Scholar
Total: 98 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 652 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf