Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Real-time Detection and Tracking of Moving Objects Using Deep Learning and Multi-threaded Kalman Filtering: A joint solution of 3D object detection and tracking for Autonomous Driving
Umeå University, Faculty of Science and Technology, Department of Applied Physics and Electronics.
2019 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

Perception for autonomous drive systems is the most essential function for safe and reliable driving. LiDAR sensors can be used for perception and are vying for being crowned as an essential element in this task. In this thesis, we present a novel real-time solution for detection and tracking of moving objects which utilizes deep learning based 3D object detection. Moreover, we present a joint solution which utilizes the predictability of Kalman Filters to infer object properties and semantics to the object detection algorithm, resulting in a closed loop of object detection and object tracking.On one hand, we present YOLO++, a 3D object detection network on point clouds only. A network that expands YOLOv3, the latest contribution to standard real-time object detection for three-channel images. Our object detection solution is fast. It processes images at 20 frames per second. Our experiments on the KITTI benchmark suite show that we achieve state-of-the-art efficiency but with a mediocre accuracy for car detection, which is comparable to the result of Tiny-YOLOv3 on the COCO dataset. The main advantage with YOLO++ is that it allows for fast detection of objects with rotated bounding boxes, something which Tiny-YOLOv3 can not do. YOLO++ also performs regression of the bounding box in all directions, allowing for 3D bounding boxes to be extracted from a bird's eye view perspective. On the other hand, we present a Multi-threaded Object Tracking (MTKF) solution for multiple object tracking. Each unique observation is associated to a thread with a novel concurrent data association process. Each of the threads contain an Extended Kalman Filter that is used for predicting and estimating an associated object's state over time. Furthermore, a LiDAR odometry algorithm was used to obtain absolute information about the movement of objects, since the movement of objects are inherently relative to the sensor perceiving them. We obtain 33 state updates per second with an equal amount of threads to the number of cores in our main workstation.Even if the joint solution has not been tested on a system with enough computational power, it is ready for deployment. Using YOLO++ in combination with MTKF, our real-time constraint of 10 frames per second is satisfied by a large margin. Finally, we show that our system can take advantage of the predicted semantic information from the Kalman Filters in order to enhance the inference process in our object detection architecture.

Place, publisher, year, edition, pages
2019. , p. 95
Keywords [en]
Master Thesis, Engineering, Autonomous Driving, Object Detection, Object Tracking, Deep Learning, Kalman Filter, Computer Vision, LiDAR
National Category
Robotics
Identifiers
URN: urn:nbn:se:umu:diva-160180OAI: oai:DiVA.org:umu-160180DiVA, id: diva2:1324807
External cooperation
Volvo Car Corporation
Educational program
Master's Programme in Robotics and Control
Presentation
2019-06-11, TB316, Department of Applied Physics, Umeå university, Umeå, 10:00 (English)
Supervisors
Examiners
Available from: 2019-06-19 Created: 2019-06-14 Last updated: 2019-06-19Bibliographically approved

Open Access in DiVA

fulltext(5064 kB)345 downloads
File information
File name FULLTEXT01.pdfFile size 5064 kBChecksum SHA-512
7f8a5e498e6d8ac6e258badfb5e9904421288fecdb305606b033905373efdc8bf8abf2d1aaeb21eeec8d3b15ad50a8c2f2ac67cd22ac9b462339f14c3b22a9de
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Söderlund, Henrik
By organisation
Department of Applied Physics and Electronics
Robotics

Search outside of DiVA

GoogleGoogle Scholar
Total: 345 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 923 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf