Change search
ReferencesLink to record
Permanent link

Direct link
Spatio-Temporal Scale-Space Theory
KTH, School of Computer Science and Communication (CSC), Computer Vision and Active Perception, CVAP.
2011 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis addresses two important topics in developing a systematic space-time geometric approach to real-time, low-level motion vision. The first one concerns measuring of image flow, while the second one focuses on how to find low level features.

We argue for studying motion vision in terms of space-time geometry rather than in terms of two (or a few) consecutive image frames. The use of Galilean Geometry and Galilean similarity geometry for this  purpose is motivated and relevant geometrical background is reviewed.

In order to measure the visual signal in a way that respects the geometry of the situation and the causal nature of time, we argue that a time causal Galilean spatio-temporal scale-space is needed. The scale-space axioms are chosen so that they generalize popular axiomatizations of spatial scale-space to spatio-temporal  geometries.

To be able to derive the scale-space, an infinitesimal framework for scale-spaces that respects a more general class of Lie groups (compared to previous theory) is developed and applied.

Perhaps surprisingly, we find that with the chosen axiomatization, a time causal Galilean scale-space is not possible as an evolution process on space and time. However, it is possible on space and memory. We argue that this actually is a more accurate and realistic model of motion vision.

While the derivation of the time causal Galilean spatio-temporal scale-spaces requires some exotic mathematics, the end result is as simple as one possibly could hope for and a natural extension of  spatial scale-spaces. The unique infinitesimally generated scale-space is an ordinary diffusion equation with drift on memory and a diffusion equation on space. The drift is used for velocity  adaption, the "velocity adaption" part of Galilean geometry (the Galilean boost) and the temporal scale-space acts as memory.

Lifting the restriction of infinitesimally generated scale spaces, we arrive at a new family of scale-spaces. These are generated by a family of fractional differential evolution equations that generalize the ordinary diffusion equation. The same type of evolution equations have recently become popular in research in e.g. financial and physical modeling.

The second major topic in this thesis is extraction of features from an image flow. A set of low-level features can be derived by classifying basic Galilean differential invariants. We proceed to derive invariants for two main cases: when the spatio-temporal  gradient cuts the image plane and when it is tangent to the image plane. The former case corresponds to isophote curve motion and the later to creation and disappearance of image structure, a case that is not well captured by the theory of optical flow.

The Galilean differential invariants that are derived are equivalent with curl, divergence, deformation and acceleration. These  invariants are normally calculated in terms of optical flow, but here they are instead calculated directly from the the  spatio-temporal image.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology , 2011. , viii, 86 p.
Series
Trita-CSC-A, ISSN 1653-5723 ; 2011:11
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
URN: urn:nbn:se:kth:diva-33686ISBN: 978-91-7501-024-3OAI: oai:DiVA.org:kth-33686DiVA: diva2:417099
Public defence
2011-06-10, sal D3, Lindstedtsvägen 5, KTH, Stockholm, 10:00 (English)
Opponent
Supervisors
Note
QC 20110518Available from: 2011-05-18 Created: 2011-05-15 Last updated: 2011-05-18Bibliographically approved
List of papers
1. Scale-space with causal time direction
Open this publication in new window or tab >>Scale-space with causal time direction
1996 (English)In: : ECCV'96 (Cambridge, U.K.) published in Springer Lecture Notes in Computer Science, vol 1064, Berlin / Heidelberg: Springer , 1996, Vol. 1064, 229-240 p.Conference paper (Refereed)
Abstract [en]

This article presents a theory for multi-scale representation of temporal data. Assuming that a real-time vision system should represent the incoming data at different time scales, an additional causality constraint arises compared to traditional scale-space theory—we can only use what has occurred in the past for computing representations at coarser time scales. Based on a previously developed scale-space theory in terms of noncreation of local maxima with increasing scale, a complete classification is given of the scale-space kernels that satisfy this property of non-creation of structure and respect the time direction as causal. It is shown that the cases of continuous and discrete time are inherently different.

Place, publisher, year, edition, pages
Berlin / Heidelberg: Springer, 1996
Series
, Lecture Notes in Computer Science, 1064
Keyword
scale-space, time, scale, motion, causality, Poisson kernel, Gaussian kernel, smoothing, visual front-end, multi-scale representation, computer vision, signal processing
National Category
Computer and Information Science Computer Vision and Robotics (Autonomous Systems) Signal Processing
Identifiers
urn:nbn:se:kth:diva-33682 (URN)10.1007/BFb0015539 (DOI)
Conference
4th European Conference on Computer Vision, (Cambridge, England), April 14-18, 1996
Note

QC 20130405

Available from: 2013-04-05 Created: 2011-05-15 Last updated: 2013-04-05Bibliographically approved
2. Temporal Scale Spaces
Open this publication in new window or tab >>Temporal Scale Spaces
2005 (English)In: International Journal of Computer Vision, ISSN 0920-5691, E-ISSN 1573-1405, Vol. 64, no 2, 97-106 p.Article in journal (Refereed) Published
Abstract [en]

In this paper we discuss how to define a scale space suitable for temporal measurements. We argue that such a temporal scale space should possess the properties of: temporal causality, linearity, continuity, positivity, recursitivity as well as translational and scaling covariance. It is shown that these requirements imply a one parameter family of convolution kernels. Furthermore it is shown that these measurements can be realized in a time recursive way, with the current data as input and the temporal scale space as state, i.e. there is no need for storing earlier input. This family of measurement processes contains the diffusion equation on the half line (that represents the temporal scale) with the input signal as boundary condition on the temporal axis. The diffusion equation is unique among the measurement processes in the sense that it is preserves positivity (in the scale domain) and is locally generated. A numerical scheme is developed and relations to other approaches are discussed.

Place, publisher, year, edition, pages
Netherlands: Springer, 2005
Keyword
scale-space, temporal measurement, stable density functions, fractional diffusion
National Category
Computer Science
Identifiers
urn:nbn:se:kth:diva-33683 (URN)10.1007/s11263-005-1837-8 (DOI)000231696700002 ()2-s2.0-24944448316 (ScopusID)
Note
QC 20110517Available from: 2011-05-15 Created: 2011-05-15 Last updated: 2011-05-18Bibliographically approved
3. Spatio-temporal Scale-Spaces
Open this publication in new window or tab >>Spatio-temporal Scale-Spaces
2007 (English)In: Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349, Vol. 4485, 326-337 p.Article in journal (Refereed) Published
Abstract [en]

A family of spatio-temporal scale-spaces suitable for a moving observer is developed. The scale-spaces are required to be time causal for being usable for real time measurements, and to be velocity adapted , i.e. to have Galilean covariance to avoid favoring any particular motion. Furthermore standard scale-space axioms: linearity, positivity, continuity, translation invariance, scaling covariance in space and time, rotational invariance in space and recursivity are used. An infinitesimal criterion for scale-spaces is developed, which simplifies calculations and makes it possible to define scale spaces on bounded regions. We show that there are no temporally causal Galilean scale-spaces that are semigroups acting on space and time, but that there are such scale-spaces that are semigroups acting on space and memory (where the memory is the scale-space). The temporally causal scale-space is a time-recursive process using current input and the scale-space as state, i.e. there is no need for storing earlier input. The diffusion equation acting on the memory with the input signal as boundary condition, is a member of this family of scale spaces and is special in the sense that its generator is local.

Place, publisher, year, edition, pages
Berlin / Heidelberg: Springer, 2007
National Category
Computer Science
Identifiers
urn:nbn:se:kth:diva-33685 (URN)10.1007/978-3-540-72823-8_28 (DOI)000247066200028 ()2-s2.0-37249080358 (ScopusID)
Conference
1st International Conference on Scale Space and Variational Methods in Computer Vision, SSVM 2007; Ischia; 30 May 2007 through 2 June 2007
Note

QC 20110517

Available from: 2011-05-15 Created: 2011-05-15 Last updated: 2013-11-26Bibliographically approved
4. Galilean Differential Geometry of Moving Images
Open this publication in new window or tab >>Galilean Differential Geometry of Moving Images
2004 (English)In: Computer Vision - ECCV 2004 / [ed] Pajdla, Tomás and Matas, Jirí, Berlin / Heidelberg: Springer , 2004, Vol. 3024, 97-101 p.Chapter in book (Refereed)
Abstract [en]

In this paper we develop a systematic theory about local structure of moving images in terms of Galilean differential invariants. We argue that Galilean invariants are useful for studying moving images as they disregard constant motion that typically depends on the motion of the observer or the observed object, and only describe relative motion that might capture surface shape and motion boundaries. The set of Galilean invariants for moving images also contains the Euclidean invariants for (still) images. Complete sets of Galilean invariants are derived for two main cases: when the spatio-temporal gradient cuts the image plane and when it is tangent to the image plane. The former case correspond to isophote curve motion and the later to creation and disappearance of image structure, a case that is not well captured by the theory of optical flow. The derived invariants are shown to be describable in terms of acceleration, divergence, rotation and deformation of image structure. The described theory is completely based on bottom up computation from local spatio-temporal image information.

Place, publisher, year, edition, pages
Berlin / Heidelberg: Springer, 2004
Series
, Lecture Notes in Computer Science, 3024
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-33684 (URN)10.1007/978-3-540-24673-2_40 (DOI)
Note
QC 20110518Available from: 2011-05-15 Created: 2011-05-15 Last updated: 2011-05-18Bibliographically approved

Open Access in DiVA

fulltext(707 kB)617 downloads
File information
File name FULLTEXT01.pdfFile size 707 kBChecksum SHA-512
37f7976d725ffd8af004427da6340e3529eeb92f36a883c22493e9c58fee9038b6902bd53c970d8f46c18e656b3b3e881d60b4421c626d841454878fbb2fb2e9
Type fulltextMimetype application/pdf
spikblad(181 kB)9 downloads
File information
File name SPIKBLAD01.pdfFile size 181 kBChecksum SHA-512
2b12b0f56cbf8017530e577804ba03234eaec455f943672a8dd531eb5a3f60fff343662d892993e2ecf68070876b16b2038cc4ebe8566eaac9294b37285051c8
Type spikbladMimetype application/pdf

Search in DiVA

By author/editor
Fagerström, Daniel
By organisation
Computer Vision and Active Perception, CVAP
Computer Vision and Robotics (Autonomous Systems)

Search outside of DiVA

GoogleGoogle Scholar
Total: 617 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 412 hits
ReferencesLink to record
Permanent link

Direct link