Change search
ReferencesLink to record
Permanent link

Direct link
Feature-Based Image Processing for Rendering, Compression, and Visual Search
KTH, School of Electrical Engineering (EES), Communication Theory.
2015 (English)Doctoral thesis, monograph (Other academic)
Abstract [en]

Visual communication, vivid, meaningful, and creative, permits a way to express information visually. The communication media, by images, graphs and videos, passes informative color and shape to human perception sensors. But when we look close, we wonder: are we merely a passive receiver? Or can we actively select what we would like? Can our eyes only sense the visual images? Or can we enjoy a comprehensive immersive experience of the real world? To discover wonders, we have to explore the essentials and under wraps of visual communication.

The work described in this dissertation develops the techniques of visual communication, including rendering, compression and visual search. We leave the conventional pixel-by-pixel image processing behind to explore the opportunities of sparse feature-based image processing. Thus, in this dissertation, a new objective is proposed: to seek a methodology to improve the performance of visual communication by using geometric information carried by the image features. To motivate it, we investigate two systems of visual communication, namely free viewpoint coding and rendering, and mobile visual search. The first system is based on the delivery and presentation of multi-view videos. We demonstrate how to use the image features for efficient video coding and high quality virtual view rendering. To further boost the importance of image features, we discuss the second system, the mobile visual search system, which is only based on the transmission of image features. We illustrate how to achieve reliable identification by using sparse image features.

The system of free-viewpoint coding and rendering encodes and delivers the video content to the end-user and allows interactively choosing and rendering a virtual viewpoint in real time. We propose a content-adaptive coding and rendering method to separate the dynamic and static video content items, and apply content-adaptive coding and rendering to each of them. The content-adaptive scheme comprises the extraction of static and dynamic content, the video coding engines, and a synthesis unit for virtual view rendering. We address the problem of using the image features for rate-distortion optimal video coding and high quality geometry model-based rendering. For the video coding engine, we study a feature-based motion compensation scheme and an optimal rate allocation model. For the component of free viewpoint rendering, we study a hypothesis-driven free viewpoint rendering approach based on 3D model hypotheses.

For the second system of mobile visual search, we propose a geometry-based search, namely mobile 3D visual search. The end-to-end scheme uses a client-server model for visual communication. The client extracts and encodes the features of the query. The server holds the feature database derived from the multi-view imagery, as well as the feature matching engine. We address the problem of rate-constrained identification by using multi-view image features. For the client, we propose a rate-constrained feature coding method to efficiently encode the query features. For the server side, we propose a double hierarchy to structure the database for indexing the database features. Moreover, we develop an algorithm that accomplishes 3D geometry-based matching and ranking by utilizing 3D geometric information and 2D texture information jointly.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2015. , xiv, 136 p.
TRITA-EE, ISSN 1653-5146 ; 2015:106
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Electrical Engineering
URN: urn:nbn:se:kth:diva-177994ISBN: 978-91-7595-791-3OAI: diva2:875251
Public defence
2015-12-21, F3, Lindstedtsvägen 26, Stockholm, 13:15 (English)

QC 20151201

Available from: 2015-12-01 Created: 2015-12-01 Last updated: 2015-12-01Bibliographically approved

Open Access in DiVA

Thesis(116059 kB)9835 downloads
File information
File name FULLTEXT01.pdfFile size 116059 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Li, Haopeng
By organisation
Communication Theory
Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 9835 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 334 hits
ReferencesLink to record
Permanent link

Direct link