Digitala Vetenskapliga Arkivet

Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Learning to Make Safe Real-Time Decisions Under Uncertainty for Autonomous Robots
Linköping University, Department of Computer and Information Science, Artificial Intelligence and Integrated Computer Systems. Linköping University, Faculty of Science & Engineering.
2020 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Robots are increasingly expected to go beyond controlled environments in laboratories and factories, to act autonomously in real-world workplaces and public spaces. Autonomous robots navigating the real world have to contend with a great deal of uncertainty, which poses additional challenges. Uncertainty in the real world accrues from several sources. Some of it may originate from imperfect internal models of reality. Other uncertainty is inherent, a direct side effect of partial observability induced by sensor limitations and occlusions. Regardless of the source, the resulting decision problem is unfortunately computationally intractable under uncertainty. This poses a great challenge as the real world is also dynamic. It  will not pause while the robot computes a solution. Autonomous robots navigating among people, for example in traffic, need to be able to make split-second decisions. Uncertainty is therefore often neglected in practice, with potentially catastrophic consequences when something unexpected happens. The aim of this thesis is to leverage recent advances in machine learning to compute safe real-time approximations to decision-making under uncertainty for real-world robots. We explore a range of methods, from probabilistic to deep learning, as well as different combinations with optimization-based methods from robotics, planning and control. Driven by applications in robot navigation, and grounded in experiments with real autonomous quadcopters, we address several parts of this problem. From reducing uncertainty by learning better models, to directly approximating the decision problem itself, all the while attempting to satisfy both the safety and real-time requirements of real-world autonomy.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2020. , p. 55
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 2051
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:liu:diva-163419DOI: 10.3384/diss.diva-163419ISBN: 9789179298890 (print)OAI: oai:DiVA.org:liu-163419DiVA, id: diva2:1417201
Public defence
2020-04-29, Ada Lovelace, hus B, Linköpings Universitet, Campus Valla, Linköping, 13:15 (English)
Opponent
Supervisors
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)Swedish Foundation for Strategic Research ELLIIT - The Linköping‐Lund Initiative on IT and Mobile CommunicationsAvailable from: 2020-04-06 Created: 2020-03-26 Last updated: 2020-04-06Bibliographically approved
List of papers
1. Model-Predictive Control with Stochastic Collision Avoidance using Bayesian Policy Optimization
Open this publication in new window or tab >>Model-Predictive Control with Stochastic Collision Avoidance using Bayesian Policy Optimization
2016 (English)In: IEEE International Conference on Robotics and Automation (ICRA), 2016, Institute of Electrical and Electronics Engineers (IEEE), 2016, p. 4597-4604Conference paper, Published paper (Refereed)
Abstract [en]

Robots are increasingly expected to move out of the controlled environment of research labs and into populated streets and workplaces. Collision avoidance in such cluttered and dynamic environments is of increasing importance as robots gain more autonomy. However, efficient avoidance is fundamentally difficult since computing safe trajectories may require considering both dynamics and uncertainty. While heuristics are often used in practice, we take a holistic stochastic trajectory optimization perspective that merges both collision avoidance and control. We examine dynamic obstacles moving without prior coordination, like pedestrians or vehicles. We find that common stochastic simplifications lead to poor approximations when obstacle behavior is difficult to predict. We instead compute efficient approximations by drawing upon techniques from machine learning. We propose to combine policy search with model-predictive control. This allows us to use recent fast constrained model-predictive control solvers, while gaining the stochastic properties of policy-based methods. We exploit recent advances in Bayesian optimization to efficiently solve the resulting probabilistically-constrained policy optimization problems. Finally, we present a real-time implementation of an obstacle avoiding controller for a quadcopter. We demonstrate the results in simulation as well as with real flight experiments.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2016
Series
Proceedings of IEEE International Conference on Robotics and Automation, ISSN 1050-4729
Keywords
Robot Learning, Collision Avoidance, Robotics, Bayesian Optimization, Model Predictive Control
National Category
Robotics Computer Sciences
Identifiers
urn:nbn:se:liu:diva-126769 (URN)10.1109/ICRA.2016.7487661 (DOI)000389516203138 ()
Conference
IEEE International Conference on Robotics and Automation (ICRA), 2016, Stockholm, May 16-21
Projects
CADICSELLIITNFFP6CUASSHERPA
Funder
Linnaeus research environment CADICSELLIIT - The Linköping‐Lund Initiative on IT and Mobile CommunicationsEU, FP7, Seventh Framework ProgrammeSwedish Foundation for Strategic Research
Available from: 2016-04-04 Created: 2016-04-04 Last updated: 2020-03-26Bibliographically approved
2. Receding-Horizon Lattice-based Motion Planning with Dynamic Obstacle Avoidance
Open this publication in new window or tab >>Receding-Horizon Lattice-based Motion Planning with Dynamic Obstacle Avoidance
Show others...
2018 (English)In: 2018 IEEE Conference on Decision and Control (CDC), Institute of Electrical and Electronics Engineers (IEEE), 2018, p. 4467-4474Conference paper, Published paper (Refereed)
Abstract [en]

A key requirement of autonomous vehicles is the capability to safely navigate in their environment. However, outside of controlled environments, safe navigation is a very difficult problem. In particular, the real-world often contains both complex 3D structure, and dynamic obstacles such as people or other vehicles. Dynamic obstacles are particularly challenging, as a principled solution requires planning trajectories with regard to both vehicle dynamics, and the motion of the obstacles. Additionally, the real-time requirements imposed by obstacle motion, coupled with real-world computational limitations, make classical optimality and completeness guarantees difficult to satisfy. We present a unified optimization-based motion planning and control solution, that can navigate in the presence of both static and dynamic obstacles. By combining optimal and receding-horizon control, with temporal multi-resolution lattices, we can precompute optimal motion primitives, and allow real-time planning of physically-feasible trajectories in complex environments with dynamic obstacles. We demonstrate the framework by solving difficult indoor 3D quadcopter navigation scenarios, where it is necessary to plan in time. Including waiting on, and taking detours around, the motions of other people and quadcopters.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2018
Series
Conference on Decision and Control (CDC), ISSN 2576-2370 ; 2018
Keywords
Motion Planning, Optimal Control, Autonomous System, UAV, Dynamic Obstacle Avoidance, Robotics
National Category
Control Engineering
Identifiers
urn:nbn:se:liu:diva-152131 (URN)10.1109/CDC.2018.8618964 (DOI)9781538613955 (ISBN)9781538613948 (ISBN)9781538613962 (ISBN)
Conference
2018 IEEE 57th Annual Conference on Decision and Control (CDC),17-19 December, Miami, Florida, USA
Funder
VINNOVAKnut and Alice Wallenberg FoundationSwedish Foundation for Strategic Research ELLIIT - The Linköping‐Lund Initiative on IT and Mobile CommunicationsSwedish Research CouncilLinnaeus research environment CADICSCUGS (National Graduate School in Computer Science)
Note

This work was partially supported by FFI/VINNOVA, the Wallenberg Artificial Intelligence, Autonomous Systems and Software Program (WASP) funded by Knut and Alice Wallenberg Foundation, the Swedish Foundation for Strategic Research (SSF) project Symbicloud, the ELLIIT Excellence Center at Linköping-Lund for Information Technology, Swedish Research Council (VR) Linnaeus Center CADICS, and the National Graduate School in Computer Science, Sweden (CUGS).

Available from: 2018-10-18 Created: 2018-10-18 Last updated: 2020-03-26Bibliographically approved
3. Deep Learning Quadcopter Control via Risk-Aware Active Learning
Open this publication in new window or tab >>Deep Learning Quadcopter Control via Risk-Aware Active Learning
2017 (English)In: Proceedings of The Thirty-first AAAI Conference on Artificial Intelligence (AAAI) / [ed] Satinder Singh and Shaul Markovitch, AAAI Press, 2017, Vol. 5, p. 3812-3818Conference paper, Published paper (Refereed)
Abstract [en]

Modern optimization-based approaches to control increasingly allow automatic generation of complex behavior from only a model and an objective. Recent years has seen growing interest in fast solvers to also allow real-time operation on robots, but the computational cost of such trajectory optimization remains prohibitive for many applications. In this paper we examine a novel deep neural network approximation and validate it on a safe navigation problem with a real nano-quadcopter. As the risk of costly failures is a major concern with real robots, we propose a risk-aware resampling technique. Contrary to prior work this active learning approach is easy to use with existing solvers for trajectory optimization, as well as deep learning. We demonstrate the efficacy of the approach on a difficult collision avoidance problem with non-cooperative moving obstacles. Our findings indicate that the resulting neural network approximations are least 50 times faster than the trajectory optimizer while still satisfying the safety requirements. We demonstrate the potential of the approach by implementing a synthesized deep neural network policy on the nano-quadcopter microcontroller.

Place, publisher, year, edition, pages
AAAI Press, 2017
Series
Proceedings of the AAAI Conference on Artificial Intelligence, ISSN 2159-5399, E-ISSN 2374-3468 ; 5
National Category
Computer Vision and Robotics (Autonomous Systems) Computer Sciences
Identifiers
urn:nbn:se:liu:diva-132800 (URN)978-1-57735-784-1 (ISBN)
Conference
Thirty-First AAAI Conference on Artificial Intelligence (AAAI), 2017, San Francisco, February 4–9.
Projects
ELLIITCADICSNFFP6SYMBICLOUDCUGS
Funder
Linnaeus research environment CADICSELLIIT - The Linköping‐Lund Initiative on IT and Mobile CommunicationsEU, FP7, Seventh Framework ProgrammeCUGS (National Graduate School in Computer Science)Swedish Foundation for Strategic Research
Available from: 2016-11-25 Created: 2016-11-25 Last updated: 2020-03-26Bibliographically approved
4. Deep RL for Autonomous Robots: Limitations and Safety Challenges
Open this publication in new window or tab >>Deep RL for Autonomous Robots: Limitations and Safety Challenges
2019 (English)Conference paper, Published paper (Refereed)
Abstract [en]

With the rise of deep reinforcement learning, there has also been a string of successes on continuous control problems using physics simulators. This has lead to some optimism regarding use in autonomous robots and vehicles. However, to successful apply such techniques to the real world requires a firm grasp of their limitations. As recent work has raised questions of how diverse these simulation benchmarks really are, we here instead analyze a popular deep RL approach on toy examples from robot obstacle avoidance. We find that these converge very slowly, if at all, to safe policies. We identify convergence issues on stochastic environments and local minima as problems that warrant more attention for safety-critical control applications.

National Category
Computer Sciences
Identifiers
urn:nbn:se:liu:diva-164581 (URN)
Conference
European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)Swedish Foundation for Strategic Research ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Available from: 2020-03-26 Created: 2020-03-26 Last updated: 2020-04-06
5. Model-Based Reinforcement Learning in Continuous Environments Using Real-Time Constrained Optimization
Open this publication in new window or tab >>Model-Based Reinforcement Learning in Continuous Environments Using Real-Time Constrained Optimization
2015 (English)In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI) / [ed] Blai Bonet and Sven Koenig, AAAI Press, 2015, p. 2497-2503Conference paper, Published paper (Refereed)
Abstract [en]

Reinforcement learning for robot control tasks in continuous environments is a challenging problem due to the dimensionality of the state and action spaces, time and resource costs for learning with a real robot as well as constraints imposed for its safe operation. In this paper we propose a model-based reinforcement learning approach for continuous environments with constraints. The approach combines model-based reinforcement learning with recent advances in approximate optimal control. This results in a bounded-rationality agent that makes decisions in real-time by efficiently solving a sequence of constrained optimization problems on learned sparse Gaussian process models. Such a combination has several advantages. No high-dimensional policy needs to be computed or stored while the learning problem often reduces to a set of lower-dimensional models of the dynamics. In addition, hard constraints can easily be included and objectives can also be changed in real-time to allow for multiple or dynamic tasks. The efficacy of the approach is demonstrated on both an extended cart pole domain and a challenging quadcopter navigation task using real data.

Place, publisher, year, edition, pages
AAAI Press, 2015
Keywords
Reinforcement Learning, Gaussian Processes, Optimization, Robotics
National Category
Computer Sciences Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-113385 (URN)978-1-57735-698-1 (ISBN)
Conference
Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), January 25-30, 2015, Austin, Texas, USA.
Funder
Linnaeus research environment CADICSeLLIIT - The Linköping‐Lund Initiative on IT and Mobile CommunicationsSwedish Foundation for Strategic Research VINNOVAEU, FP7, Seventh Framework Programme
Available from: 2015-01-16 Created: 2015-01-16 Last updated: 2020-03-26Bibliographically approved
6. Real-Time Robotic Search using Structural Spatial Point Processes
Open this publication in new window or tab >>Real-Time Robotic Search using Structural Spatial Point Processes
Show others...
2019 (English)Conference paper, Published paper (Refereed)
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:liu:diva-159698 (URN)
Conference
Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI 2019), Tel Aviv, Israel, July 22-25, 2019
Available from: 2019-08-19 Created: 2019-08-19 Last updated: 2020-06-01Bibliographically approved

Open Access in DiVA

fulltext(8200 kB)112 downloads
File information
File name FULLTEXT01.pdfFile size 8200 kBChecksum SHA-512
3591ede282ad408008a1de3c3757a44201f454391e7c6ee39b50a56c0b2e3fca91ffc0f94eb085b57fe9b69f7401d57c75caf5324ce5b8a6459b1a594b4d7bd8
Type fulltextMimetype application/pdf
Order online >>

Other links

Publisher's full text

Search in DiVA

By author/editor
Andersson, Olov
By organisation
Artificial Intelligence and Integrated Computer SystemsFaculty of Science & Engineering
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 112 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 1135 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf