Change search
ReferencesLink to record
Permanent link

Direct link
Hand-held Augmented Reality for Facility Maintenance
Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Computerized Image Analysis and Human-Computer Interaction. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Visual Information and Interaction. University of Gävle.
2016 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Buildings and public infrastructures are crucial to our societies in that they provide habitations, workplaces, commodities and services indispensible to our daily life. As vital parts of facility management, operations and maintenance (O&M) ensure a facility to continuously function as intended, which take up the longest time in a facility’s life cycle and demand great expense. Therefore, computers and information technology have been actively adopted to automate traditional maintenance methods and processes, making O&M faster and more reliable.

Augmented reality (AR) offers a new approach towards human-computer interaction through directly displaying information related to real objects that people are currently perceiving. People’s sensory perceptions are enhanced (augmented) with information of interest naturally without deliberately turning to computers. Hence, AR has been proved to be able to further improve O&M task performance.

The research motif of this thesis is user evaluations of AR applications in the context of facility maintenance. The studies look into invisible target designation tasks assisted by developed AR tools in both indoor and outdoor scenarios. The focus is to examine user task performance, which is influenced by both AR system performance and human perceptive, cognitive and motoric factors.

Target designation tasks for facility maintenance entail a visualization-interaction dilemma. Two AR systems built upon consumer-level hand-held devices using an off-the-shelf AR software development toolkit are evaluated indoors with two disparate solutions to the dilemma – remote laser pointing and the third person perspective (TPP). In the study with remote laser pointing, the parallax effect associated with AR “X-ray vision” visualization is also an emphasis.

A third hand-held AR system developed in this thesis overlays infrared information on façade video, which is evaluated outdoors. Since in an outdoor environment marker-based tracking is less desirable, an infrared/visible image registration method is developed and adopted by the system to align infrared information correctly with the façade in the video. This system relies on the TPP to overcome the aforementioned dilemma.

Place, publisher, year, edition, pages
Uppsala: Acta Universitatis Upsaliensis, 2016. , 82 p.
Series
Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology, ISSN 1651-6214 ; 1412
Keyword [en]
Augmented reality, Façade, Image registration, Thermal infrared imaging, Facility management, Third person perspective, Target designation, Precision study, Experiment
National Category
Other Computer and Information Science Human Computer Interaction
Research subject
Computerized Image Processing
Identifiers
URN: urn:nbn:se:uu:diva-301363ISBN: 978-91-554-9669-2OAI: oai:DiVA.org:uu-301363DiVA: diva2:954225
Public defence
2016-10-07, Room 2446, Lägerhyddsvägen 2, House 2, Uppsala, 13:15 (English)
Opponent
Supervisors
Available from: 2016-09-15 Created: 2016-08-22 Last updated: 2016-09-22
List of papers
1. Detection of Line Features in Digital Images of Building Structures
Open this publication in new window or tab >>Detection of Line Features in Digital Images of Building Structures
2012 (English)In: Proceedings of IADIS International Conference Computer Graphics, Visualization, Computer Vision and Image Processing 2012 (CGVCVIP 2012), 2012, 163-167 p.Conference paper (Refereed)
National Category
Other Computer and Information Science Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:uu:diva-185560 (URN)
Conference
IADIS CGVCVIP 2012
Available from: 2012-11-26 Created: 2012-11-26 Last updated: 2016-08-29
2. Detection of Façade Regions in Street View Images from Split-and-Merge of Perspective Patches
Open this publication in new window or tab >>Detection of Façade Regions in Street View Images from Split-and-Merge of Perspective Patches
2014 (English)In: Journal of Image and Graphics, ISSN 2301-3699, Vol. 2, no 1, 8-14 p.Article in journal (Refereed) Published
Abstract [en]

Identification of building façades from digital images is one of the central problems in mobile augmented reality (MAR) applications in the built environment. Directly analyzing the whole image can increase the difficulty of façade identification due to the presence of image portions which are not façade. This paper presents an automatic approach to façade region detection given a single street view image as a pre-processing step to subsequent steps of façade identification. We devise a coarse façade region detection method based on the observation that façades are image regions with repetitive patterns containing a large amount of vertical and horizontal line segments. Firstly, scan lines are constructed from vanishing points and center points of image line segments. Hue profiles along these lines are then analyzed and used to decompose the image into rectilinear patches with similar repetitive patterns. Finally, patches are merged into larger coherent regions and the main building façade region is chosen based on the occurrence of horizontal and vertical line segments within each of the merged regions. A validation of our method showed that on average façade regions are detected in conformity with manually segmented images as ground truth.

Keyword
façade region detection, street view image, vanishing point, mobile augmented reality
National Category
Computer Vision and Robotics (Autonomous Systems) Human Computer Interaction
Research subject
Computerized Image Processing; Human-Computer Interaction
Identifiers
urn:nbn:se:uu:diva-238622 (URN)
Available from: 2014-12-15 Created: 2014-12-15 Last updated: 2016-08-29
3. Infrared-visible image registration for augmented reality-based thermographic building diagnostics
Open this publication in new window or tab >>Infrared-visible image registration for augmented reality-based thermographic building diagnostics
2015 (English)In: Visualization in Engineering, ISSN 2213-7459, Vol. 3, 16:1-15 p., 16Article in journal (Refereed) Published
Abstract [en]

Background

In virtue of their capability to measure temperature, thermal infrared cameras have been widely used in building diagnostics for detecting heat loss, air leakage, water damage etc. However, the lack of visual details in thermal infrared images makes the complement of visible images a necessity. Therefore, it is often useful to register images of these two modalities for further inspection of architectures. Augmented reality (AR) technology, which supplements the real world with virtual objects, offers an ideal tool for presenting the combined results of thermal infrared and visible images. This paper addresses the problem of registering thermal infrared and visible façade images, which is essential towards developing an AR-based building diagnostics application.

Methods

A novel quadrilateral feature is devised for this task, which models the shapes of commonly present façade elements, such as windows. The features result from grouping edge line segments with the help of image perspective information, namely, vanishing points. Our method adopts a forward selection algorithm to determine feature correspondences needed for estimating the transformation model. During the formation of the feature correspondence set, the correctness of selected feature correspondences at each step is verified by the quality of the resulting registration, which is based on the ratio of areas between the transformed features and the reference features.

Results and conclusions

Quantitative evaluation of our method shows that registration errors are lower than errors reported in similar studies and registration performance is usable for most tasks in thermographic inspection of building façades.

Place, publisher, year, edition, pages
Springer, 2015
Keyword
Multimodality image registration, Augmented reality, Thermal infrared imaging, Façade
National Category
Computer Vision and Robotics (Autonomous Systems)
Research subject
Computerized Image Processing
Identifiers
urn:nbn:se:uu:diva-266889 (URN)10.1186/s40327-015-0028-0 (DOI)
Available from: 2015-11-09 Created: 2015-11-13 Last updated: 2016-08-29
4. On the precision of third person perspective augmented reality for target designation tasks
Open this publication in new window or tab >>On the precision of third person perspective augmented reality for target designation tasks
2016 (English)In: Multimedia tools and applications, ISSN 1380-7501, E-ISSN 1573-7721Article in journal (Refereed) Accepted
National Category
Human Computer Interaction Other Computer and Information Science
Identifiers
urn:nbn:se:uu:diva-301359 (URN)
Available from: 2016-08-22 Created: 2016-08-22 Last updated: 2016-08-29
5. Precision study on augmented reality-based visual guidance for facility management tasks
Open this publication in new window or tab >>Precision study on augmented reality-based visual guidance for facility management tasks
(English)Article in journal (Refereed) Submitted
National Category
Human Computer Interaction Other Computer and Information Science
Identifiers
urn:nbn:se:uu:diva-301360 (URN)
Available from: 2016-08-22 Created: 2016-08-22 Last updated: 2016-08-29
6. Augmented reality-based building diagnostics using natural feature registration and third person perspective
Open this publication in new window or tab >>Augmented reality-based building diagnostics using natural feature registration and third person perspective
(English)Manuscript (preprint) (Other academic)
National Category
Human Computer Interaction Other Computer and Information Science
Identifiers
urn:nbn:se:uu:diva-301361 (URN)
Available from: 2016-08-22 Created: 2016-08-22 Last updated: 2016-08-29

Open Access in DiVA

fulltext(1853 kB)96 downloads
File information
File name FULLTEXT01.pdfFile size 1853 kBChecksum SHA-512
be5c45a964959ca8430f68fd0374bff0588bede392af4f8f1374dc9a25356ede249de36f38ce5bd9a427871348340d3391c90b3d34e38d6e464d922edf8c8441
Type fulltextMimetype application/pdf
Buy this publication >>

Search in DiVA

By author/editor
Liu, Fei
By organisation
Computerized Image Analysis and Human-Computer InteractionDivision of Visual Information and Interaction
Other Computer and Information ScienceHuman Computer Interaction

Search outside of DiVA

GoogleGoogle Scholar
Total: 96 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 246 hits
ReferencesLink to record
Permanent link

Direct link