In this paper, we introduce a novel framework for low-level image processing and analysis. First, we process images with very simple, difference-based filter functions. Second, we fit the 2-parameter Weibull distribution to the filtered output. This maps each image to the 2D Weibull manifold. Third, we exploit the information geometry of this manifold and solve low-level image processing tasks as minimisation problems on point sets. For a proof-of-concept example, we examine the image autofocusing task. We propose appropriate cost functions together with a simple implicitly-constrained manifold optimisation algorithm and show that our framework compares very favourably against common autofocus methods from literature. In particular, our approach exhibits the best overall performance in terms of combined speed and accuracy

State-of-the-art object detectors typically use shape information as a low level feature representation to capture the local structure of an object. This paper shows that early fusion of shape and color, as is popular in image classification, leads to a significant drop in performance for object detection. Moreover, such approaches also yields suboptimal results for object categories with varying importance of color and shape. In this paper we propose the use of color attributes as an explicit color representation for object detection. Color attributes are compact, computationally efficient, and when combined with traditional shape features provide state-of-the-art results for object detection. Our method is tested on the PASCAL VOC 2007 and 2009 datasets and results clearly show that our method improves over state-of-the-art techniques despite its simplicity. We also introduce a new dataset consisting of cartoon character images in which color plays a pivotal role. On this dataset, our approach yields a significant gain of 14% in mean AP over conventional state-of-the-art methods.

We propose a novel method for iterative learning of point correspondences between image sequences. Points moving on surfaces in 3D space are projected into two images. Given a point in either view, the considered problem is to determine the corresponding location in the other view. The geometry and distortions of the projections are unknown as is the shape of the surface. Given several pairs of point-sets but no access to the 3D scene, correspondence mappings can be found by excessive global optimization or by the fundamental matrix if a perspective projective model is assumed. However, an iterative solution on sequences of point-set pairs with general imaging geometry is preferable. We derive such a method that optimizes the mapping based on Neyman's chi-square divergence between the densities representing the uncertainties of the estimated and the actual locations. The densities are represented as channel vectors computed with a basis function approach. The mapping between these vectors is updated with each new pair of images such that fast convergence and high accuracy are achieved. The resulting algorithm runs in real-time and is superior to state-of-the-art methods in terms of convergence and accuracy in a number of experiments.

This work proposes an approach to tracking by regression that uses no hard-coded models and no offline learning stage. The Linear Predictor (LP) tracker has been shown to be highly computationally efficient, resulting in fast tracking. Regression tracking techniques tend to require offline learning to learn suitable regression functions. This work removes the need for offline learning and therefore increases the applicability of the technique. The online-LP tracker can simply be seeded with an initial target location, akin to the ubiquitous Lucas-Kanade algorithm that tracks by registering an image template via minimisation. A fundamental issue for all trackers is the representation of the target appearance and how this representation is able to adapt to changes in target appearance over time. The two proposed methods, LP-SMAT and LP-MED, demonstrate the ability to adapt to large appearance variations by incrementally building an appearance model that identifies modes or aspects of the target appearance and associates these aspects to the Linear Predictor trackers to which they are best suited. Experiments comparing and evaluating regression and registration techniques are presented along with performance evaluations favourably comparing the proposed tracker and appearance model learning methods to other state of the art simultaneous modelling and tracking approaches.

In this article we describe a set of canonical transformations of the image spaces that make the description of three-view geometry very simple. The transformations depend on the three-view geometry and the canonically transformed trifocal tensor T' takes the form of a sparse array where 17 elements in well-defined positions are zero, it has a linear relation to the camera matrices and to two of the fundamental matrices, a third order relation to the third fundamental matrix, a second order relation to the other two trifocal tensors, and first order relations to the 10 three-view all-point matching constraints. In this canonical form, it is also simple to determine if the corresponding camera configuration is degenerate or co-linear. An important property of the three canonical transformations of the images spaces is that they are in SO(3). The 9 parameters needed to determine these transformations and the 9 parameters that determine the elements of T' together provide a minimal parameterization of the tensor. It does not have problems with multiple maps or multiple solutions that other parameterizations have, and is therefore simple to use. It also provides an implicit representation of the trifocal internal constraints: the sparse canonical representation of the trifocal tensor can be determined if and only if it is consistent with its internal constraints. In the non-ideal case, the canonical transformation can be determined by solving a minimization problem and a simple algorithm for determining the solution is provided. This allows us to extend the standard linear method for estimation of the trifocal tensor to include a constraint enforcement as a final step, similar to the constraint enforcement of the fundamental matrix.

Experimental evaluation of this extended linear estimation method shows that it significantly reduces the geometric error of the resulting tensor, but on average the algebraic estimation method is even better. For a small percentage of cases, however, the extended linear method gives a smaller geometric error, implying that it can be used as a complement to the algebraic method for these cases.

In this paper, we present a novel scheme for anisotropic diffusion driven by the image autocorrelation function. We show the equivalence of this scheme to a special case of iterated adaptive filtering. By determining the diffusion tensor field from an autocorrelation estimate, we obtain an evolution equation that is computed from a scalar product of diffusion tensor and the image Hessian. We propose further a set of filters to approximate the Hessian on a minimized spatial support. On standard benchmarks, the resulting method performs favorable in many cases, in particular at low noise levels. In a GPU implementation, video real-time performance is easily achieved.

Recent years have seen advances in the estimation of full 6 degree-of-freedom object pose from a single 2D image. These advances have often been presented as a result of, or together with, a new local image feature type. This paper examines how the pose accuracy and recognition robustness for such a system varies with choice of feature type. This is done by evaluating a full 6 degree-of-freedom pose estimation system for 17 different combinations of local descriptors and detectors. The evaluation is done on data sets with photos of challenging 3D objects with simple and complex backgrounds and varying illumination conditions. We examine the performance of the system under varying levels of object occlusion and we find that many features allow considerable object occlusion. From the experiments we can conclude that duplet features, that use pairs of interest points, improve pose estimation accuracy, compared to single point features. Interestingly, we can also show that many features previously used for recognition and wide-baseline stereo are unsuitable for pose estimation, one notable example are the affine covariant features that have been proven quite successful in other applications. The data sets and their ground truths are available on the web to allow future comparison with novel algorithms.

The major goal of the COSPAL project is to develop an artificial cognitive system architecture, with the ability to autonomously extend its capabilities. Exploratory learning is one strategy that allows an extension of competences as provided by the environment of the system. Whereas classical learning methods aim at best for a parametric generalization, i.e., concluding from a number of samples of a problem class to the problem class itself, exploration aims at applying acquired competences to a new problem class, and to apply generalization on a conceptual level, resulting in new models. Incremental or online learning is a crucial requirement to perform exploratory learning. In the COSPAL project, we mainly investigate reinforcement-type learning methods for exploratory learning, and in this paper we focus on the organization of cognitive systems for efficient operation. Learning is used over the entire system. It is organized in the form of four nested loops, where the outermost loop reflects the user-reinforcement-feedback loop, the intermediate two loops switch between different solution modes at symbolic respectively sub-symbolic level, and the innermost loop performs the acquired competences in terms of perception-action cycles. We present a system diagram which explains this process in more detail. We discuss the learning strategy in terms of learning scenarios provided by the user. This interaction between user (teacher) and system is a major difference to classical robotics systems, where the system designer places his world model into the system. We believe that this is the key to extendable robust system behavior and successful interaction of humans and artificial cognitive systems. We furthermore address the issue of bootstrapping the system, and, in particular, the visual recognition module. We give some more in-depth details about our recognition method and how feedback from higher levels is implemented. The described system is however work in progress and no final results are available yet. The available preliminary results that we have achieved so far, clearly point towards a successful proof of the architecture concept.

In this paper, we present a visual servoing method based on a learned mapping between feature space and control space. Using a suitable recognition algorithm, we present and evaluate a complete method that simultaneously learns the appearance and control of a low-cost robotic arm. The recognition part is trained using an action precedes perception approach. The novelty of this paper, apart from the visual servoing method per se, is the combination of visual servoing with gripper recognition. We show that we can achieve high precision positioning without knowing in advance what the robotic arm looks like or how it is controlled.

Intrinsic dimensionality is a concept introduced by statistics and later used in image processing to measure the dimensionality of a data set. In this paper, we introduce a continuous representation of the intrinsic dimension of an image patch in terms of its local spectrum or, equivalently, its gradient field. By making use of a cone structure and barycentric co-ordinates, we can associate three confidences to the three different ideal cases of intrinsic dimensions corresponding to homogeneous image patches, edge-like structures and junctions. The main novelty of our approach is the representation of confidences as prior probabilities which can be used within a probabilistic framework. To show the potential of our continuous representation, we highlight applications in various contexts such as image structure classification, feature detection and localisation, visual scene statistics and optic flow evaluation.

In this paper we propose a new approach to real-time view-based pose recognition and interpolation. Pose recognition is particularly useful for identifying camera views in databases, video sequences, video streams, and live recordings. All of these applications require a fast pose recognition process, in many cases video real-time. It should further be possible to extend the database with new material, i.e., to update the recognition system online. The method that we propose is based on P-channels, a special kind of information representation which combines advantages of histograms and local linear models. Our approach is motivated by its similarity to information representation in biological systems but its main advantage is its robustness against common distortions such as clutter and occlusion. The recognition algorithm consists of three steps: (1) low-level image features for color and local orientation are extracted in each point of the image; (2) these features are encoded into P-channels by combining similar features within local image regions; (3) the query P-channels are compared to a set of prototype P-channels in a database using a least-squares approach. The algorithm is applied in two scene registration experiments with fisheye camera data, one for pose interpolation from synthetic images and one for finding the nearest view in a set of real images. The method compares favorable to SIFT-based methods, in particular concerning interpolation. The method can be used for initializing pose-tracking systems, either when starting the tracking or when the tracking has failed and the system needs to re-initialize. Due to its real-time performance, the method can also be embedded directly into the tracking system, allowing a sensor fusion unit choosing dynamically between the frame-by-frame tracking and the pose recognition.

This paper brings together a novel information representation model for use in signal processing and computer vision problems, with a particular algorithmic development of the Landweber iterative algorithm. The information representation model allows a representation of multiple values for a variable as well as an expression for confidence. Both properties are important for effective computation using multi-level models, where a choice between models will be implementable as part of the optimization process. It is shown that in this way the algorithm can deal with a class of high-dimensional, sparse, and constrained least-squares problems, which arise in various computer vision learning tasks, such as object recognition and object pose estimation. While the algorithm has been applied to the solution of such problems, it has so far been used heuristically. In this paper we describe the properties and some of the peculiarities of the channel representation and optimization, and put them on firm mathematical ground. We consider the optimization a convexly constrained weighted least-squares problem and propose for its solution a projected Landweber method which employs oblique projections onto the closed convex constraint set. We formulate the problem, present the algorithm and work out its convergence properties, including a rate-of-convergence result. The results are put in perspective with currently available projected Landweber methods. An application to supervised learning is described, and the method is evaluated in an experiment involving function approximation, as well as application to transient signals. © 2006 Elsevier Ltd. All rights reserved.

A runtime system for implementation of image processing operations is presented. It is designed for working in a flexible and distributed environment related to the software architecture of a newly developed UAV system. The software architecture can be characterized at a coarse scale as a layered system, with a deliberative layer at the top, a reactive layer in the middle, and a processing layer at the bottom. At a finer scale each of the three levels is decomposed into sets of modules which communicate using CORBA, allowing system development and deployment on the UAV to be made in a highly flexible way. Image processing takes place in a dedicated module located in the process layer, and is the main focus of the paper. This module has been designed as a runtime system for data flow graphs, allowing various processing operations to be created online and on demand by the higher levels of the system. The runtime system is implemented in Java, which allows development and deployment to be made on a wide range of hardware/software configurations. Optimizations for particular hardware platforms have been made using Java's native interface.

 In medical helical cone-beam CT, it is common that the region-of-interest (ROI) is contained inside the helix cylinder, while the complete object is long and extends outside the top and the bottom of the cylinder. This is the Long Object Problem. Analytical reconstruction methods for helical cone-beam CT have been designed to handle this problem. It has been shown that a moderate amount of over-scanning is sufficient for reconstruction of a certain ROI. The over-scanning projection rays travel both through the ROI as well as outside the ROI. This is unfortunate for iterative methods since it seems impossible to compute accurate values for the projection rays which travel partly inside and partly outside the ROI. Therefore, it seems that the useful ROI will diminish for every iteration step. We propose the following solution to the problem. Firstly, we reconstruct volume regions also outside the ROI. These volume regions will certainly be incompletely reconstructed, but our experimental results show that they serve well for projection generation. This is rather counter-intuitive and contradictory to our initial assumptions. Secondly, we use careful extrapolation and masking of projection data. This is not a general necessity, but needed for the chosen iterative algorithm, which includes rebinning and iterative filtered backprojection. Our idea here was to use an approximate reconstruction method which gives cone-beam artifacts and then improve the reconstructed result by iterative filtered backprojection. The experimental results seem very encouraging. The cone-beam artifacts can indeed be removed. Even voxels close to the boundary of the ROI are as well enhanced by the iterative loop as those in the middle of the ROI.

Tensors have become a popular tool for representation of local orientation and can be used also for estimation of velocity. A number of computational approaches have been presented for tensor estimation which, however, are difficult to analyze or compare since there has been no common framework in which analysis or comparisons can be made. In this article, we propose such a framework based on second-order filters and show how it applies to three different methods for tensor estimation. The framework contains a few conditions on the filters which are sufficient to obtain correctly oriented rank one tensors for the case of simple signals. It also allows the derivation of explicit expressions for the variation of the tensor across oriented structures which, e.g., can be used to formulate conditions for phase invariance. (c) 2005 Elsevier B.V. All rights reserved.

In this paper we present a novel method to implement the monogenic scale space on a rectangular domain. The monogenic scale space is a vector valued scale space based on the Poisson scale space, which establishes a sophisticated alternative to the Gaussian scale space. Previous implementations of the monogenic scale space are Fourier transform based, and therefore suffer from the implicit periodicity in case of finite domains. The features of the monogenic scale space, including local amplitude, local phase, local orientation, local frequency, and phase congruency, are much easier to interpret in terms of image features evolving through scale than in the Gaussian case. Furthermore, applying results from harmonic analysis, relations between the features are obtained which improve the understanding of image analysis. As applications, we present a very simple but still accurate approach to image reconstruction from local amplitude and local phase and a method for extracting the evolution of lines and edges through scale.

A method for unrestricted recognition of three-dimensional objects was developed. By unrestricted, we imply that the recognition will be done independently of object position, scale, orientation, and pose against a structured background. It does not assume any preceding segmentation or allow a reasonable degree of occlusion. The method uses a hierarchy of triplet feature invariants, which are at each level defined by a learning procedure. In the feedback learning procedure, percepts are mapped on system states corresponding to manipulation parameters of the object. The method uses a learning architecture with channel information representation. This article discusses how objects can be represented. We propose a structure to deal with object and contextual properties in a transparent manner.

In this paper we address the topics of scale-space and phase-based image processing in a unifying framework. In contrast to the common opinion, the Gaussian kernel is not the unique choice for a linear scale-space. Instead, we chose the Poisson kernel since it is closely related to the monogenic signal, a 2D generalization of the analytic signal, where the Riesz transform replaces the Hilbert transform. The Riesz transform itself yields the flux of the Poisson scale-space and the combination of flux and scale-space, the monogenic scale-space, provides the local features phase-vector and attenuation in scale-space. Under certain assumptions, the latter two again form a monogenic scale-space which gives deeper insight to low-level image processing. In particular, we discuss edge detection by a new approach to phase congruency and its relation to amplitude based methods, reconstruction from local amplitude and local phase, and the evaluation of the local frequency.

This paper introduces a two-dimensional generalization of the analytic signal. This novel approach is based on the Riesz transform, which is used instead of the Hilbert transform. The combination of a 2D signal with the Riesz transformed one yields a sophisticated 2D analytic signal, the monogenic signal. The approach is derived analytically from irrotational and solenoidal vector fields. Based on local amplitude and local phase, an appropriate local signal representation is presented which preserves the split of identity, i.e., the invariance – equivariance property of signal decomposition. This is one of the central properties of the 1D analytic signal that decomposes a signal into structural and energetic information. We show that further properties of the analytic signal concerning symmetry, energy, allpass transfer function, and orthogonality are also preserved, and we compare this to the behavior of other approaches for a 2D analytic signal. As a central topic of this paper, a geometric phase interpretation is introduced which is based on the relation between the 1D analytic signal and the 2D monogenic signal established by the Radon transform. Possible applications of this relationship are sketched and references to other applications of the monogenic signal are given. This report is a revised version of the technical report 2009 [7], and therefore supercedes it.

There is no indication that it will ever be possible to find some simple trick that miraculously solves most problems in vision. It turns out that the processing system must be able to implement a model structure, the complexity of which is directly related to the structural complexity of the problem under consideration in the external world. It has become increasingly apparent that Vision cannot be treated in isolation from the response generation, because a very high degree of integration is required between different levels of percepts and corresponding response primitives. The response to be produced at a given instance is as much dependent upon the state of the system, as the percepts impinging upon the system. In addition, it has become apparent that many classical aspects of perception, such as geometry, probably do not belong to the percept domain of a Vision system, but to the response domain. This article will focus on what are considered crucial problems in Vision for robotics for the future, rather than on the classical solutions today. It will discuss hierarchical architectures for combination of percept and response primitives. It will discuss the concept of combined percept-response invariances as important structural elements for Vision. It will be maintained that learning is essential to obtain the necessary flexibility and adaptivity. In consequence, it will be argued that invariances for the purpose of Vision are not abstractly geometrical, but derived from the percept-response interaction with the environment. The issue of information representation becomes extremely important in distributed structures of the types foreseen, where uncertainty of information has to be stated for update of models and associated data. The question of object representation is central to the paper. Equivalence is established between the representations of response, geometry and time. Finally an integrated percept-response structure is proposed for flexible response control.

This paper deals with a new framework in analyzing the formal mathematical correspondence between quantum mechanics and time-frequency representations of a signal. It is also shown that joint time-frequency distributions have a close link with Heisenberg uncertainty relations if the observables are taken as fuzzy entities. This result contradicts the arguments of Cohen [IEEE Proc. 77(7):941 (1989)] regarding the time-frequency distributions and the uncertainty relation. It is postulated that these mechanisms will be of crucial importance in highly fragmented computation structures, such as neural networks, as they may exhibit a strong mutual interaction between data and operator.

In this paper, we discuss certain issues regarding robot vision. The main theme will be the importance of the choice of information representation. We will see the implications at different parts of a robot vision structure. We deal with aspects of pre-attentive versus attentive vision, control mechanisms for low level focus of attention, and representation of motion as the orientation of hyperplanes in multdimensional time-space. Issues of scale will be touched upon, and finally, a depth-from stereo algorithm based on guadrature filter phase is presented.

A framework for computer-aided analysis of mammograms is described. General computer vision algorithms are combined with application specific procedures in a hierarchical fashion. The system is under development and is currently limited to detection of a few types of suspicious areas. The image features are extracted by using feature extraction methods where wavelet techniques are utilized. A low-pass pyramid representation of the image is convolved with a number of quadrature filters. The filter outputs are combined according to simple local Fourier domain models into parameters describing the local neighborhood with respect to the model. This produces estimates for each pixel describing local size, orientation, Fourier phase, and shape with confidence measures associated to each parameter. Tentative object descriptions are then extracted from the pixel-based features by application-specific procedures with knowledge of relevant structures in mammograms. The orientation, relative brightness and shape of the object are obtained by selection of the pixel feature estimates which best describe the object. The list of object descriptions is examined by procedures, where each procedure corresponds to a specific type of suspicious area, e.g. clusters of microcalcifications.

A new low-level vision primitive based on logarithmic spirals is presented for various image processing tasks. The detection of such primitives is equivalent to detection of lines and edges in another coordinate system which has been used to model the mapping of the visual field to the striate cortex. Algorithms detecting the proposed primitives and pointing out a matched subclass are presented along with necessary theory. As a result, if the local structure is describable by the proposed primitives then a certainty parameter based on a well-defined mismatch (error) function will indicate this. Moreover, the best fit of a subclass of the proposed primitives in the least squares sense will be computed. The resulting images are unthresholded. They are computed by means of simple convolutions and pixelwise arithmetic operations which make the algorithms suitable for real time image processing applications. Since the resulting images contain information about the local structure, they can be used as feature images in applications like remote sensing, texture analysis, and object recognition. Experimental results on the latter including synthetic as well as natural images are presented along with noise sensitivity tests. The results exhibit good detection properties for the subclasses of the modeled primitives along with uniform and reliable behavior of the corresponding certainty measures.

Phototransduction in rod cells is likely to involve an intracellular messenger system that links the absorption of light by rhodopsin to a change in membrane conductance. The direct effect of guanosine 3',5'-monophosphate (cGMP) on excised patches of rod outer segment membrane strongly supports a role for cGMP as an intracellular messenger in phototransduction. It is reported here that magnesium and calcium directly affect the conductance of excised patches of rod membrane in the absence of cGMP and that magnesium, applied to intact rod cells, blocks a component of the cellular light response. The divalent cation-suppressed conductance in excised patches showed outward rectification and cation-selective permeability resembling those of the light-suppressed conductance measured from the intact rod cell. The divalent cation-suppressed conductance was partly blocked by a concentration of the pharmacological agent L-cis-diltiazem that blocked all of the cGMP-activated conductance. Divalent cations may act, together with cGMP, as an intracellular messenger system that mediates the light response of the rod photoreceptor cell.

A new device for generation of electromagnetic fields at extra low frequencies, to be used in fracture treatment, is described. The device involves a coil and a battery powered noise-generator. An alternating magnetic field of 4 × 10⁻⁴T (4 Gauss) (RMS value) with a frequency range 1–1000 Hz is generated. Results from a controlled randomized study of fresh fractures have shown significant differences (p < 0.01) between the treated group and the control group. The results are encouraging and motivate further investigations with this method.

A new form of image estimator, which takes account of linear features, is derived using a signal equivalent formulation. The estimator is shown to be a nonstationary linear combination of three stationary estimators. The relation of the estimator to human visual physiology is discussed. A method for estimating the nonstationary control information is described and shown to be effective when the estimation is made from noisy data. A suboptimal approach which is computationally less demanding is presented and used in the restoration of a variety of images corrupted by additive white noise. The results show that the method can improve the quality of noisy images even when the signal-to-noise ratio is very low.

The mathematical basis is described of a new radiographic method by which an arbitrarily thick layer of the patient may be reconstructed. The reconstruction is performed from at least 60 images of the volume under examination. Each of these images, which have to be in digital form, is subjected to a special filtration process of its spatial frequencies. The combination of all the images will form the resulting image of the layer--the ectomogram. The method has been analysed and tested in experiments simulated with a computer.

The problem of finding a general, parallel, and hierarchical operator for picture processing is considered. An operator is defined which at different levels can detect and describe structure as opposed to uniformity within local regions, whatever structure and uniformity may imply at a particular level. The operator performs a mapping from one complex field to another. The important characteristic of this approach is the use of complex fields which allows a global-to-local feedback. In the transformation process the image is simplified. A Fourier implementation of the operator is described and a new transform is defined. The operators become increasingly global on higher levels in order to include adjacent high-level features. A hierarchical structure of such transformations gives a sequential description of structure over increasingly larger regions of the image. The processed information at different levels can be used as input to a classifier. Examples are given of processing results.

The advent of new stains for chromosomes has increased the possibilities that useful automated chromosome analysis can be implemented. The search for appropriate descriptors to use in this process is an important task. Data compression using integrated intensity and density profiles has already shown itself to be valuable. A method is proposed in this paper to describe these profiles as a sum of distribution functions. Every distribution function can be described by a triplet stating peak height, position, and width and it appears that these parameters are directly related to physical processes. The importance of such parameters in statistical chromosome analysis is emphasized. A classification experiment is described in which 240 chromosomes 1 to 22, X and Y have been classified with an accuracy of 96%.

Signal Processing for Computer Vision is a unique and thorough treatment of the signal processing aspects of filters and operators for low-level computer vision.

Computer vision has progressed considerably over recent years. From methods only applicable to simple images, it has developed to deal with increasingly complex scenes, volumes and time sequences. A substantial part of this book deals with the problem of designing models that can be used for several purposes within computer vision. These partial models have some general properties of invariance generation and generality in model generation.

Signal Processing for Computer Vision is the first book to give a unified treatment of representation and filtering of higher order data, such as vectors and tensors in multidimensional space. Included is a systematic organisation for the implementation of complex models in a hierarchical modular structure and novel material on adaptive filtering using tensor data representation.

Signal Processing for Computer Vision is intended for final year undergraduate and graduate students as well as engineers and researchers in the field of computer vision and image processing.

Linear filter systems are used in low-level image processing to analyze the visual properties of small image patches. We show first how to use the theory of group representations to construct filter systems that are both steerable and are minimum mean squared error solutions. The underlying groups are the dihedral groups and the permutation groups and the resulting filter systems define a transform which has many properties in common with the well-known discrete Fourier transform. We also show that the theory of extreme value distributions provides a framework to investigate the statistical properties of the vast majority of the computed filter responses. These distributions are completely characterized by only three parameters and in applications involving huge numbers of such distributions, they provide very compact and efficient descriptors of the visual properties of the images. We compare these descriptors with more conventional methods based on histograms and show how they can be used for re-ranking (finding typical images in a class of images) and classification.

Traffic sign recognition is important for the development of driver assistance systems and fully autonomous vehicles. Even though GPS navigator systems works well for most of the time, there will always be situations when they fail. In these cases, robust vision based systems are required. Traffic signs are designed to have distinct colored fields separated by sharp boundaries. We propose to use locally segmented contours combined with an implicit star-shaped object model as prototypes for the different sign classes. The contours are described by Fourier descriptors. Matching of a query image to the sign prototype database is done by exhaustive search. This is done efficiently by using the correlation based matching scheme for Fourier descriptors and a fast cascaded matching scheme for enforcing the spatial requirements. We demonstrated on a publicly available database state of the art performance.

Linear scale-space theory is the fundamental building block for many approaches to image processing like pyramids or scale-selection. However, linear smoothing does not preserve image structures very well and thus non-linear techniques are mostly applied for image enhancement. A different perspective is given in the framework of channel-smoothing, where the feature domain is not considered as a linear space, but it is decomposed into local basis functions. One major drawback is the larger memory requirement for this type of representation, which is avoided if the channel representation is subsampled in the spatial domain. This general type of feature representation is called channel-coded feature map (CCFM) in the literature and a special case using linear channels is the SIFT descriptor. For computing CCFMs the spatial resolution and the feature resolution need to be selected. In this paper, we focus on the spatio-featural scale-space from a scale-selection perspective. We propose a coupled scheme for selecting the spatial and the featural scales. The scheme is based on an analysis of lower bounds for the product of uncertainties, which is summarized in a theorem about a spatio-featural uncertainty relation. As a practical application of the derived theory, we reconstruct images from CCFMs with resolutions according to our theory. The results are very similar to the results of non-linear evolution schemes, but our algorithm has the fundamental advantage of being non-iterative. Any level of smoothing can be achieved with about the same computational effort.

This Chapter is devoted to the cooperation of multiple UAVs for environment perception. First, probabilistic methods for multi-UAV cooperative perception are analyzed. Then, the problem of multi-UAV detection, localization and tracking is described, and local image processing techniques are presented. Then, the Chapter shows two approaches based on the Information Filter and on evidence grid representations.

One problem in appearance-based pose estimation is the need for many training examples, i.e. images of the object in a large number of known poses. Some invariance can be obtained by considering translations, rotations and scale changes in the image plane, but the remaining degrees of freedom are often handled simply by sampling the pose space densely enough. This work presents a method for accurate interpolation between training views using local linear models. As a view representation local soft orientation histograms are used. The derivative of this representation with respect to the image plane transformations is computed, and a Gauss-Newton optimization is used to optimize all pose parameters simultaneously, resulting in an accurate estimate.

The purpose of cognitive systems is to produce a response to appropriate percepts. The response may be a direct physical action which may change the state of the system. It may be delayed in the form of a reconfiguration of internal models in relation to the interpreted context of the system. Or it may be to generate in a subsequent step a generalized symbolic representation which will allow its intentions of actions to be communicated to some other system. As important as the percepts, is the dependence upon context. A fundamental property of cognitive vision systems is that they shall be extendable. This requires that systems both acquire and store information about the environment autonomously on their own terms. The distributed organization foreseen for processing and for memory to allow learning, implies that later acquired information has to be stored in relation to earlier. The semantic character of the information which this requires, implies a storage with respect to similarity, and the availability of a metric. The paper discusses organization aspects of such systems, and proposes an architecture for cognitive systems. In this architecture, which consists of two major parts, the first part of the system, step by step performs a mapping from percepts onto actions or states. The central mechanism is the perception-action feedback cycle. In a learning phase, action precedes perception. The reason for this reversed causal direction is that action space is much less complex than percept space. It is easy to distinguish which percepts change as a function of an action or a state change. Percepts shall be mapped directly onto states or responses or functions involving these. The second part of the architecture deals with more invariant representations, of symbolic representations, which are derived mainly from system states and action states. Through active exploration of the environment, e.g. using perceptionaction learning, a system builds up concept spaces, defining the phenomena it can deal with. Information can subsequently be acquired by the system within these concept spaces without interaction, by extrapolation using passive observation or communication such as language. This structure has been implemented for the learning of object properties and view parameters in a fairly unrestricted setting, to be used for subsequent recognition purposes.

This paper presents novel results from an ongoing feasibility study of fully 3D X-ray scanning of Pinus Sylvestris (Scots Pine) logs. Logs are assumed to be translated through two identical and static cone beam systems with the beams rotated 90degrees relative to eachother, providing a dual set of 2D-projections. For reasons of both cost and speed, each 2D-detector in these two systems consists of a limited number of line detectors. The quality of the reconstructed images is far from perfect, due to sparse detector data and missing projection angles. In spite of this we show that by employing a shape- and direction discriminative technique based on second derivatives, we are able to enhance knot-like features in these data. In the enhanced images it is then possible to detect and localize the pith for each whorl of knots, and subsequently also to perform a full segmentation of the knots in the heartwood.

Perceptual experiments indicate that corners and curvature are very important features in the process of recognition. This paper presents a new method to efficiently detect rotational symmetries, which describe complex curvature such as corners, circles, star- and spiral patterns. The method is designed to give selective and sparse responses. It works in three steps, first extract local orientation from a gray-scale or color image, second correlate the orientation image with rotational symmetry filters and third let the filter responses inhibit each other in order to get more selective responses, The correlations can be made efficient by separating the 2D-filters into a small number of 1D-filters. These symmetries can serve as feature points at a high abstraction level for use in hierarchical matching structures for 3D-estimation, object recognition, etc.

We apply the 3D-orientation tensor representation to construct an object tracking algorithm. 2D-line normal flow is estimated by computing the eigenvector associated with the largest eigenvalue of 3D (two spatial dimensions plus time) tensors with a planar structure. Object's true 2D velocity is computed by averaging tensors with consistent normal flows, generating a 3D line represention that corresponds to a 2D point in motion. Flow induced by camera rotation is compensated for by ignoring points with velocity consistent with the ego-rotation. A region-of-interest growing process based on motion consistency generates estimates of object size and position.  Introduction The literature on optical flow estimation is wast. Descriptions and performance studies of a number of different techniques are given in  and the monographs by Fleet  and Jahne. We will only briefly describe the particular methods used in the present study. Details on the tensor field represention a...

This chapter deals with texture analysis, an important application of the methods described in earlier chapters. It introduces ideas from preattentive vision, which gives clues for the extraction of texture primitives. There is also a discussion on how to handle features whose significance varies with spatial position.

This chapter discusses techniques for processing of higher order data such as vector and tensor fields. As abstraction implies a more complex descriptor, developing methods for processing of higher order data is an essential part of any hierarchical or layered approach to vision. The chapter focuses on models for extracting local symmetries and discontinuities in higher order fields.

This chapter considers what the important properties are for an information representation to behave well in various transformations. There is an extended discussion on the necessity to separate between class membership and certainty of a signal.

This chapter deals with the concept of phase and phase representation in multiple dimensions. Phase is an important concept, which emerges in several contexts in vision. The chapter provides a detailed treatment of phase properties in various situations and deals with how to maintain continuity in phase representation. An example is given of how local phase in difference scales can be used for disparity estimation. The chapter contains original material on the representation of phase for signals in three dimensions and higher dimensions.

This chapter presents a method for obtaining an optimal n-dimensional set of filter coefficients for any given frequency response. An optimality criterion is defined that enables different frequencies to be given individual weights. Appropriate forms of frequency weight functions are discussed and a number of optimization examples are given.

This chapter gives an introductory treatment of operations andrepresentations for low-level features in multi-dimensional spaces. Animportant issue is how to combine contributions from several filtersto provide robust statements in accordance with certain low-levelmodels. This chapter gives an introduction to the problems ofunambiguous mappings in multi-dimensional spaces.

This chapter establishes the motivation anduse of hierarchical operation structures to provide a systematicorganization for the implementation of complicated models. The chaptergives an intuitive treatment of most aspects that are considered inthe later chapters.

The problem of estimating depth information from two or more images of a scene is one which has received considerable attention over the years and a wide variety of methods have been proposed to solve it [Barnard and Fichsler, 1982; Fleck, 1991]. Methods based on correlation and methods using some form of feature matching between the images have found most widespread use. Of these, the latter have attracted increasing attention since the work of Marr [Marr, 1982], in which the features are zero-crossings on varying scales. These methods share an underlying basis of spatial domain operations.

In recent years, however, increasing interest has been shown in computational models of vision based primarily on a localized frequency domain representation - the Gabor representation [Gabor, 1946; Adelson and Bergen, 1985], first suggested in the context of computer vision by Granlund [Granlund, 1978].

The Application Visualization System software from Advanced Visual Systems Inc is an interactive visualization environment for scientists, engineers and technical professionals. This report contains a short overview of the AVS software packages and a discussion about its general performance. The software package has actively been used at the Computer Vision Laboratory, Linköping University, during the last three years. The AVS package has been used in many applications. Examples are generating images from a virtual environment, simulation of a controllable robot with a stereo camera head and visualization of multidimensional data structures. Lately we also have used AVS for handling communication between different processes which may be distributed on different machines. AVS was primarily developed as a tool for visualization of complex data sets. However, another important aspect of the software is that it can be used as an advanced workbench for controlling networks of Unix processes (including external ones on different machine types) using simple visual programming.

In this work we derive a novel density driven diffusion scheme for image enhancement. Our approach, called D3, is a semi-local method that uses an initial structure-preserving oversegmentation step of the input image.  Because of this, each segment will approximately conform to a homogeneous region in the image, allowing us to easily estimate parameters of the underlying stochastic process thus achieving adaptive non-linear filtering. Our method is capable of producing competitive results when compared to state-of-the-art methods such as non-local means, BM3D and tensor driven diffusion on both color and grayscale images.

Robust estimation of the relative pose between two camerasis a fundamental part of Structure and Motion methods.For calibrated cameras, the five point method togetherwith a robust estimator such as RANSAC gives the best resultin most cases. The current state-of-the-art method forsolving the relative pose problem from five points is due toNist´er [9], because it is faster than other methods and in theRANSAC scheme one can improve precision by increasingthe number of iterations.In this paper, we propose a new iterative method, whichis based on Powell’s Dog Leg algorithm. The new methodhas the same precision and is approximately twice as fast asNist´er’s algorithm. The proposed method is easily extendedto more than five points while retaining a efficient error metrics.This makes it also very suitable as an refinement step.The proposed algorithm is systematically evaluated onthree types of datasets with known ground truth.

A method for online global pose estimation of aerial images by alignment with a georeferenced 3D model is presented.Motion stereo is used to reconstruct a dense local height patch from an image pair. The global pose is inferred from the 3D transform between the local height patch and the model.For efficiency, the sought 3D similarity transform is found by least-squares minimizations of three 2D subproblems.The method does not require any landmarks or reference points in the 3D model, but an approximate initialization of the global pose, in our case provided by onboard navigation sensors, is assumed.Real aerial images from helicopter and aircraft flights are used to evaluate the method. The results show that the accuracy of the position and orientation estimates is significantly improved compared to the initialization and our method is more robust than competing methods on similar datasets.The proposed matching error computed between the transformed patch and the map clearly indicates whether a reliable pose estimate has been obtained.

Many RGB-D sensors, e.g. the Microsoft Kinect, use rolling shutter cameras. Such cameras produce geometrically distorted images when the sensor is moving. To mitigate these rolling shutter distortions we propose a method that uses an attached gyroscope to rectify the depth scans. We also present a simple scheme to calibrate the relative pose and time synchronization between the gyro and a rolling shutter RGB-D sensor. We examine the effectiveness of our rectification scheme by coupling it with the the Kinect Fusion algorithm. By comparing Kinect Fusion models obtained from raw sensor scans and from rectified scans, we demonstrate improvement for three classes of sensor motion: panning motions causes slant distortions, and tilt motions cause vertically elongated or compressed objects. For wobble we also observe a loss of detail, compared to the reconstruction using rectified depth scans. As our method relies on gyroscope readings, the amount of computations required is negligible compared to the cost of running Kinect Fusion.

Complementary information, when combined in the right way, is capable of improving clustering and segmentation problems. In this paper, we show how it is possible to enhance motion segmentation accuracy with a very simple and inexpensive combination of complementary information, which comes from the column and row spaces of the same measurement matrix. We test our approach on the Hopkins155 dataset where it outperforms all other state-of-the-art methods.

This work addresses the problem of fast, online segmentationof moving objects in video. We pose this as a discriminative onlinesemi-supervised appearance learning task, where supervising labelsare autonomously generated by a motion segmentation algorithm. Thecomputational complexity of the approach is signicantly reduced byperforming learning and classication on oversegmented image regions(superpixels), rather than per pixel. In addition, we further exploit thesparse trajectories from the motion segmentation to obtain a simplemodel that encodes the spatial properties and location of objects at eachframe. Fusing these complementary cues produces good object segmentationsat very low computational cost. In contrast to previous work,the proposed approach (1) performs segmentation on-the-y (allowingfor applications where data arrives sequentially), (2) has no prior modelof object types or `objectness', and (3) operates at signicantly reducedcomputational cost. The approach and its ability to learn, disambiguateand segment the moving objects in the scene is evaluated on a numberof benchmark video sequences.

The case when a partial differential equation (PDE) can be considered as an Euler-Lagrange (E-L) equation of an energy functional, consisting of a data term and a smoothness term is investigated. We show the necessary conditions for a PDE to be the E-L equation for a corresponding functional. This energy functional is applied to a color image denoising problem and it is shown that the method compares favorably to current state-of-the-art color image denoising techniques.

This paper introduces a bundle adjustment (BA) method that obtains accurate structure and motion from rolling shutter (RS) video sequences: RSBA. When a classical BA algorithm processes a rolling shutter video, the resultant camera trajectory is brittle, and complete failures are not uncommon. We exploit the temporal continuity of the camera motion to define residuals of image point trajectories with respect to the camera trajectory. We compare the camera trajectories from RSBA to those from classical BA, and from classical BA on rectified videos. The comparisons are done on real video sequences from an iPhone 4, with ground truth obtained from a global shutter camera, rigidly mounted to the iPhone 4. Compared to classical BA, the rolling shutter model requires just six extra parameters. It also degrades the sparsity of the system Jacobian slightly, but as we demonstrate, the increase in computation time is moderate. Decisive advantages are that RSBA succeeds in cases where competing methods diverge, and consistently produces more accurate results.

Structured light range sensors, such as the Microsoft Kinect, have recently become popular as perception devices for computer vision and robotic systems. These sensors use CMOS imaging chips with electronic rolling shutters (ERS). When using such a sensor on a moving platform, both the image, and the depth map, will exhibit geometric distortions. We introduce an algorithm that can suppress such distortions, by rectifying the 3D point clouds from the range sensor. This is done by first estimating the time continuous 3D camera trajectory, and then transforming the 3D points to where they would have been, if the camera had been stationary. To ensure that image and range data are synchronous, the camera trajectory is computed from KLT tracks on the structured-light frames, after suppressing the structured-light pattern. We evaluate our rectification, by measuring angles between the visible sides of a cube, before and after rectification. We also measure how much better the 3D point clouds can be aligned after rectification. The obtained improvement is also related to the actual rotational velocity, measured using a MEMS gyroscope.

Techniques from the theory of partial differential equations are often used to design filter methods that are locally adapted to the image structure. These techniques are usually used in the investigation of gray-value images. The extension to color images is non-trivial, where the choice of an appropriate color space is crucial.  The RGB color space is often used although it is known that the space of human color perception is best described in terms of non-euclidean geometry, which is fundamentally different from the structure of the RGB space. Instead of the standard RGB space, we use a simple color transformation based on the theory of finite groups. It is shown that this transformation reduces the color artifacts originating from the diffusion processes on RGB images. The developed algorithm is evaluated on a set of images, and it is shown that our approach exhibits fewer color artifacts compared to a state-of-the-art technique. Also, our approach preserves details in the image for a larger number of iterations.

Quantitative tissue classification using dual-energy CT has the potential to improve accuracy in radiation therapy dose planning as it provides more information about material composition of scanned objects than the currently used methods based on single-energy CT. One problem that hinders successful application of both single-and dualenergy CT is the presence of beam hardening and scatter artifacts in reconstructed data. Current pre-and post-correction methods used for image reconstruction often bias CT numbers and thus limit their applicability for quantitative tissue classification. Here we demonstrate simulation studies with a novel iterative algorithm that decomposes every soft tissue voxel into three base materials: water, protein and adipose. The results demonstrate that beam hardening artifacts can effectively be removed and accurate estimation of mass fractions of all base materials can be achieved. In the future, the algorithm may be developed further to include segmentation of soft and bone tissue and subsequent bone decomposition, extension from 2-D to 3-D and scatter correction.

Segmentation is an important preprocessing step in many applications. Compared to colour segmentation, fusion of colour and depth greatly improves the segmentation result. Such a fusion is easy to do by stacking measurements in different value dimensions, but there are better ways. In this paper we perform fusion using the channel representation, and demonstrate how a state-of-the-art segmentation algorithm can be modified to use channel values as inputs. We evaluate segmentation results on data collected using the Microsoft Kinect peripheral for Xbox 360, using the superparamagnetic clustering algorithm. Our experiments show that depth gradients are more useful than depth values for segmentation, and that channel coding both colour and depth gradients makes tuned parameter settings generalise better to novel images.

A new approach to track bicycles from imagery sensor data is proposed. It is based on detecting ellipsoids in the images, and treat these pair-wise using a dynamic bicycle model. One important application area is in automotive collision avoidance systems, where no dedicated systems for bicyclists yet exist and where very few theoretical studies have been published.

Possible conflicts can be predicted from the position and velocity state in the model, but also from the steering wheel articulation and roll angle that indicate yaw changes before the velocity vector changes. An algorithm is proposed which consists of an ellipsoid detection and estimation algorithm and a particle filter.

A simulation study of three critical single target scenarios is presented, and the algorithm is shown to produce excellent state estimates. An experiment using a stationary camera and the particle filter for state estimation is performed and has shown encouraging results.

Techniques from the theory of partial differential equations are often used to design filter methods that are locally adapted to the image structure. These techniques are usually used in the investigation of gray-value images. The extension to color images is non-trivial, where the choice of an appropriate color space is crucial. The RGB color space is often used although it is known that the space of human color perception is best described in terms of non-euclidean geometry, which is fundamentally different from the structure of the RGB space. Instead of the standard RGB space, we use a simple color transformation based on the theory of finite groups. It is shown that this transformation reduces the color artifacts originating from the diffusion processes on RGB images. The developed algorithm is evaluated on a set of real-world images, and it is shown that our approach exhibits fewer color artifacts compared to state-of-the-art techniques. Also, our approach preserves details in the image for a larger number of iterations.

We study the problem of registering a sequence of scan lines (a strip) from an airborne push-broom imager to another sequence partly covering the same area. Such a registration has to compensate for deformations caused by attitude and speed changes in the aircraft. The registration is challenging, as both strips contain such deformations. Our algorithm estimates the 3D rotation of the camera for each scan line, by parametrising it as a linear spline with a number of knots evenly distributed in one of the strips. The rotations are estimated from correspondences between strips of the same area. Once the rotations are known, they can be compensated for, and each line of pixels can be transformed such that ground trace of the two strips are registered with respect to each other.

We have performed a field trial with an airborne push-broom hyperspectral sensor, making several flights over the same area and with known changes (e.g., moved vehicles) between the flights. Each flight results in a sequence of scan lines forming an image strip, and in order to detect changes between two flights, the two resulting image strips must be geometrically aligned and radiometrically corrected. The focus of this paper is the geometrical alignment, and we propose an image- and gyro-based method for geometric co-alignment (registration) of two image strips. The method is particularly useful when the sensor is not stabilized, thus reducing the need for expensive mechanical stabilization. The method works in several steps, including gyro-based rectification, global alignment using SIFT matching, and a local alignment using KLT tracking. Experimental results are shown but not quantified, as ground truth is, by the nature of the trial, lacking.

We propose a method for segmenting an arbitrary number of moving objects using the geometry of 6 points in 2D images to infer motion consistency. This geometry allows us to determine whether or not observations of 6 points over several frames are consistent with a rigid 3D motion. The matching between observations of the 6 points and an estimated model of their configuration in 3D space, is quantified in terms of a geometric error derived from distances between the points and 6 corresponding lines in the image. This leads to a simple motion inconsistency score, based on the geometric errors of 6points that in the ideal case should be zero when the motion of the points can be explained by a rigid 3D motion. Initial point clusters are determined in the spatial domain and merged in motion trajectory domain based on this score. Each point is then assigned to the cluster, which gives the lowest score.Our algorithm has been tested with real image sequences from the Hopkins155 database with very good results, competing withthe state of the art methods, particularly for degenerate motion sequences. In contrast to the motion segmentation methods basedon multi-body factorization, that assume an affine camera model, the proposed method allows the mapping from 3D space to the 2D image to be fully projective.

A common computer vision task is navigation and mapping. Many indoor navigation tasks require depth knowledge of flat, unstructured surfaces (walls, floor, ceiling). With passive illumination only, this is an ill-posed problem. Inspired by small children using a torchlight, we use a spotlight for active illumination. Using our torchlight approach, depth and orientation estimation of unstructured, flat surfaces boils down to estimation of ellipse parameters. The extraction of ellipses is very robust and requires little computational effort.

A single-camera gaze tracker has been created,based on previous implementations by Shih and Liu[5] and Hennessey et al. [3]. The method used is basedon controlled infrared illumination. The implementedsystem has been evaluated on both synthetic and realimage data and found to be capable of estimatinggaze point with an accuracy of approximately 1° visualangle.

In this work we examine in detail the use of optimisation algorithms on deformable template matching problems. We start with the examination of simple, direct-search methods and move on to more complicated evolutionary approaches. Our goal is twofold: first, evaluate a number of methods examined under different template matching settings and introduce the use of certain, novel evolutionary optimisation algorithms to computer vision, and second, explore and analyse any additional advantages of using a hybrid approach over existing methods. We show that in computer vision tasks, evolutionary strategies provide very good choices for optimisation. Our experiments have also indicated that we can improve the convergence speed and results of existing algorithms by using a hybrid approach.

Linear scale-space theory is the fundamental building block for many approaches to image processing like pyramids or scale-selection. However, linear smoothing does not preserve image structures very well and thus non-linear techniques are mostly applied for image enhancement. A different perspective is given in the framework of channel-smoothing, where the feature domain is not considered as a linear space, but it is decomposed into local basis functions. One major drawback is the larger memory requirement for this type of representation, which is avoided if the channel representation is subsampled in the spatial domain. This general type of feature representation is called channel-coded feature map (CCFM) in the literature and a special case using linear channels is the SIFT descriptor. For computing CCFMs the spatial resolution and the feature resolution need to be selected.

In this paper, we focus on the spatio-featural scale-space from a scale-selection perspective. We propose a coupled scheme for selecting the spatial and the featural scales. The scheme is based on an analysis of lower bounds for the product of uncertainties, which is summarized in a theorem about a spatio-featural uncertainty relation. As a practical application of the derived theory, we reconstruct images from CCFMs with resolutions according to our theory. The results are very similar to the results of non-linear evolution schemes, but our algorithm has the fundamental advantage of being non-iterative. Any level of smoothing can be achieved with about the same computational effort.

This paper describes a system that efficiently uses the KLT tracker together with a calibrated 5-point solver for structure-from-motion (SfM). Our system uses a GPU to perform tracking, and the CPU for SfM.

In this setup, it is advantageous to run the tracker both forwards and backwards in time, to detect incorrectly tracked points. We introduce a modification to the point selection inside the RANSAC step of the 5-point solver, and demonstrate how this speeds up the algorithm. Our evaluations are done using both real camera sequences, and data from a state-of-the art rendering engine with associated ground-truth.

Radial sampling of k-space is known to simultaneously provide both high spatial and high temporal resolution. Recently, an optimal radial profile time order based on the Golden Ratio was presented in [1]. We have adopted and modified the idea, with a focus onhigher temporal resolution without sacrificing any image quality.

[1] Winkelmann et. al.: An optimal radial profile order based on the golden ratio for time-resolved MRI, IEEE Trans. Med. Im, Vol.26, No.1,2007.

This paper describes a system for structure-and-motion estimation for real-time navigation and obstacle avoidance. We demonstrate it technique to increase the efficiency of the 5-point solution to the relative pose problem. This is achieved by a novel sampling scheme, where We add a distance constraint on the sampled points inside the RANSAC loop. before calculating the 5-point solution. Our setup uses the KLT tracker to establish point correspondences across tone in live video We also demonstrate how an early outlier rejection in the tracker improves performance in scenes with plenty of occlusions. This outlier rejection scheme is well Slated to implementation on graphics hardware. We evaluate the proposed algorithms using real camera sequences with fine-tuned bundle adjusted data as ground truth. To strenghten oar results we also evaluate using sequences generated by a state-of-the-art rendering software. On average we are able to reduce the number of RANSAC iterations by half and thereby double the speed.

This work presents a novel object tracking approach, where the motion model is learned from sets of frame-wise detections with unknown associations. We employ a higher-order Markov model on position space instead of a first-order Markov model on a high-dimensional state-space of object dynamics. Compared to the latter, our approach allows the use of marginal rather than joint distributions, which results in a significant reduction of computation complexity. Densities are represented using a grid-based approach, where the rectangular windows are replaced with estimated smooth Parzen windows sampled at the grid points. This method performs as accurately as particle filter methods with the additional advantage that the prediction and update steps can be learned from empirical data. Our method is compared against standard techniques on image sequences obtained from an RC car following scenario. We show that our approach performs best in most of the sequences. Other potential applications are surveillance from cheap or uncalibrated cameras and image sequence analysis.

Recent years have seen advances in the estimation of full 6 degree-of-freedom object pose from a single 2D image. These advances have often been presented as a result of, or together with, a new local image descriptor. This paper examines how the performance for such a system varies with choice of local descriptor. This is done by comparing the performance of a full 6 degree-of-freedom pose estimation system for fourteen types of local descriptors. The evaluation is done on a database with photos of complex objects with simple and complex backgrounds and varying lighting conditions. From the experiments we can conclude that duplet features, that use pairs of interest points, improve pose estimation accuracy, and that affine covariant features do not work well in current pose estimation frameworks. The data sets and their ground truth is available on the web to allow future comparison with novel algorithms.

Triangulation of a 3D point from two or more views can be solved inseveral ways depending on how perturbations in the image coordinatesare dealt with.  A common approach is optimal triangulation which minimizes the total L₂ reprojection error in the images,corresponding to finding a maximum likelihood estimate of the 3Dpoint assuming independent Gaussian noise in the image spaces.Computational approaches for optimal triangulation have beenpublished for the stereo case and, recently, also for the three-viewcase.  In short, they solve an independent optimization problem foreach 3D point, using relatively complex computations such as findingroots of high order polynomials or matrix decompositions.  Thispaper discuss three-view triangulation and reports the followingresults: (1) the 3D point can be computed as multi-linear mapping(tensor) applied on the homogeneous image coordinates, (2)  the set of triangulation tensors forms a 7-dimensional space determined by the camera matrices, (3) given a set of corresponding 3D/2Dcalibration points, the 3D residual L₁ errors can be optimized over the elements in the 7-dimensional space, (4) using theresulting tensor as initial value, the error can be further reducedby tuning the tensor in a two-step iterative process, (5) the 3Dresidual L₁ error for a set of evaluation points which lieclose to the calibration set is comparable to the three-view optimalmethod.  In summary, three-view triangulation can be done by firstperforming an optimization of the triangulation tensor and once this is done, triangulation can be made with 3D residual error at thesame level as the optimal method, but at a much lower computationalcost.  This makes the proposed method attractive for real-time three-view triangulation of large data sets provided that thenecessary calibration process can be performed.

We present a novel local descriptor for range data that can describe one or more planes or lines in a local region. It is possible to recover the geometry of the described local region and extract the size, position and orientation of each local plane or line-like structure from the descriptor. This gives the descriptor a property that other popular local descriptors for range data, such as spin images or point signatures, does not have. The estimation of the descriptor is dependant on estimation of surface normals but does not depend on the specific normal estimation method used. It is shown that is possible to extract how many planar surface regions the descriptor represents and that this could be used as a point-of-interest detector.

One major goal of the COSPAL project is to develop an artificial cognitive system architecture with the capability of exploratory learning. Exploratory learning is a strategy that allows to apply generalization on a conceptual level, resulting in an extension of competences. Whereas classical learning methods aim at best possible generalization, i.e., concluding from a number of samples of a problem class to the problem class itself, exploration aims at applying acquired competences to a new problem class. Incremental or online learning is an inherent requirement to perform exploratory learning.

Exploratory learning requires new theoretic tools and new algorithms. In the COSPAL project, we mainly investigate reinforcement-type learning methods for exploratory learning and in this paper we focus on its algorithmic aspect. Learning is performed in terms of four nested loops, where the outermost loop reflects the user-reinforcement-feedback loop, the intermediate two loops switch between different solution modes at symbolic respectively sub-symbolic level, and the innermost loop performs the acquired competences in terms of perception-action cycles. We present a system diagram which explains this process in more detail.

We discuss the learning strategy in terms of learning scenarios provided by the user. This interaction between user ('teacher') and system is a major difference to most existing systems where the system designer places his world model into the system. We believe that this is the key to extendable robust system behavior and successful interaction of humans and artificial cognitive systems.

We furthermore address the issue of bootstrapping the system, and, in particular, the visual recognition module. We give some more in-depth details about our recognition method and how feedback from higher levels is implemented. The described system is however work in progress and no final results are available yet. The available preliminary results that we have achieved so far, clearly point towards a successful proof of the architecture concept.

This paper explores the possibility to use a single low-resolution FIR camera for detection of pedestrians in the near zone in front of a vehicle. A low resolution sensor reduces the cost of the system, as well as the amount of data that needs to be processed in each frame.

We present a system that makes use of hot-spots and image positions of a near constant bearing to detect potential pedestrians. These detections provide seeds for an energy minimization algorithm that fits a pedestrian model to the detection. Since false alarms are hard to tolerate, the pedestrian model is then tracked, and the distance-to-collision (DTC) is measured by integrating size change measurements at sub-pixel accuracy, and the car velocity. The system should only engage braking for detections on a collision course, with a reliably measured DTC.

Preliminary experiments on a number of recorded near collision sequences indicate that our method may be useful for ranges up to about 10m using an 80x60 sensor, and somewhat more using a 160x120 sensor. We also analyze the robustness of the evaluated algorithm with respect to dead pixels, a potential problem for low-resolution sensors.

Although not so often expressed as a modeling problem neither projection nor back-projection can be designed without certain insights in the physics of CT. However, most of this insight is left aside, since it is generally believed that only the most simplified models can be included in the innermost timeconsuming loop in projection and back-projection. We propose that any linear projection procedure should model three functions: The irradiation function, the footprint/basis function, and the gantry rotation function. We demonstrate how a moderately advanced modeling of these three functions can be brought together in an interpolation procedure and yield a surprisingly efficient inner loop interpolation. To this end we i) carefully select a locus of interpolation path through image and projection data spaces and ii) execute multiple convolution as integration by parts implemented by table-look-up.

The GPU is the main processing unit on a graphics card. A modern GPU typically provides more than ten times the computational power of an ordinary PC processor. This is a result of the high demands for speed and image quality in computer games. This paper investigates the possibility of exploiting this computational power for tracking points in image sequences. Tracking points is used in many computer vision tasks, such as tracking moving objects, structure from motion, face tracking etc. The algorithm was successfully implemented on the GPU and a large speed up was achieved.

In order to insert a virtual object into a TV image, the graphics system needs to know precisely how the camera is moving, so that the virtual object can be rendered in the correct place in every frame. Nowadays this can be achieved relatively easily in postproduction, or in a studio equipped with a special tracking system. However, for live shooting on location, or in a studio that is not specially equipped, installing such a system can be difficult or uneconomic. To overcome these limitations, the MATRIS project is developing a real-time system for measuring the movement of a camera. The system uses image analysis to track naturally occurring features in the scene, and data from an inertial sensor. No additional sensors, special markers, or camera mounts are required. This paper gives an overview of the system and presents some results.  

We have developed a system which integrates the information output from several pose estimation algorithms and from several views of the scene. It is tested in a real setup with a robotic manipulator. It is shown that integrating pose estimates from several algorithms increases the overall performance of the pose estimation accuracy as well as the robustness as compared to using only a single algorithm. It is shown that increased robustness can be achieved by using pose estimation algorithms based on complementary features, so called algorithmic multi-cue integration (AMC). Furthermore it is also shown that increased accuracy can be achieved by integrating pose estimation results from different views of the scene, so-called temporal multi-cue integration (TMC). Temporal multi-cue integration is the most interesting aspect of this paper.

Traditionally, quadrature filters and derivatives have been considered as alternative approaches to low-level image analysis. In this paper we show that there actually exist close connections: We define the quadrature-based boundary tensor and the derivative-based gradient energy tenser which exhibit very similar behavior. We analyse the reason for this and determine how to minimize the difference. These insights lead to a simple and very efficient integrated feature detection algorithm.

The channel representation allows the construction of soft histograms, where peaks can be detected with a much higher accuracy than in regular hard-binned histograms. This is critical in e.g. reducing the number of bins of generalized Hough transform methods. When applying the maximum entropy method to the channel representation, a minimum-information reconstruction of the underlying continuous probability distribution is obtained. The maximum entropy reconstruction is compared to simpler linear methods in some simulated situations. Experimental results show that mode estimation of the maximum entropy reconstruction outperforms the linear methods in terms of quantization error and discrimination threshold. Finding the maximum entropy reconstruction is however computationally more expensive.

In this paper, we combine the well-established technique of Wiener filtering with an efficient method for robust smoothing: channel smoothing. The main parameters to choose in channel smoothing are the number of channels and the averaging filter. Whereas the number of channels has a natural lower bound given by the noise level and should for the sake of speed be as small as possible, the averaging filter is a less obvious choice. Based on the linear behavior of channel smoothing for inlier noise, we derive a Wiener filter applicable for averaging the channels of an image. We show in some experiments that our method compares favorable with established methods.

This largelytutorial treatise presents a Fourier based model for 2D-projection,the latter being a most important ingredient in any iterativereconstruction method. For sampled images the model requires anassumed basis function, which implicitly defines the necessarywindow and interpolation functions. We unravel the basis and windowfunctions for some projection techniques described as procedures.Circular symmetric basis functions make it simple to findinterpolation coefficients but require well tuned interpolationfunctions to avoid aliasing. We find it unnecessary to distinguishbetween voxel and ray driven projection. These two techniquesconcern only the innermost loop and both can be applied to anyinterpolation function, and to projection and back-projectionalike. 

 In this paper we discuss the benefits of writing code for a specific processor and exploiting all its capabilities. We shows that in some situations it is possible to significantly reduce the time consumption by using SSE2, a Single Instruction Multiple Data (SIMD) extension available in new Pentium processors. Speed of the Harris operator is used for evaluation. All experiments are run on a Pentium 4 and the results are compared between ordinary C-code and code using SSE2. The purpose is not only to achieve a significant speed-up of the code, but also to benefit from SSE2 code with the least possible programming effort.

 To program a robot to solve a simple shape-sorter puzzle is trivial. To devise a Cognitive System Architecture, which allows the system to find out by itself how to go about a solution, is less than trivial. The development of such an architecture is one of the aims of the COSPAL project, leading to new techniques in vision based Artificial Cognitive Systems, which allow the development of robust systems for real dynamic environments. The systems developed under the project itself remain however in simplified scenarios, likewise the shape-sorter problem described in the present paper. The key property of the described system is its robustness. Since we apply association strategies of local features, the system behaves robustly under a wide range of distortions, as occlusion, colour and intensity changes. The segmentation step which is applied in many systems known from literature is replaced with local associations and view-based hypothesis validation. The hypotheses used in our system are based on the anticipated state of the visual percepts. This state replaces explicit modeling of shapes. The current state is chosen by a voting system and verified against the true visual percepts. The anticipated state is obtained from the association to the manipulator actions, where reinforcement learning replaces the explicit calculation of actions. These three differences to classical schemes allow the design of a much more generic and flexible system with a high level of robustness. On the technical side, the channel representation of information and associative learning in terms of the channel learning architecture are essential ingredients for the system. It is the properties of locality, smoothness, and non-negativity which make these techniques suitable for this kind of application. The paper gives brief descriptions of how different system parts have been implemented and show some examples from our tests.

This paper describes a method for extracting point features from an image, corresponding to corners and crossings of lines. The method is based on a local estimation of a 6 x 6 tensor which describes the parameters of a pair of line segments. By considering the rank of the tensor, it is possible to find points of interest. These points can then be further analyzed to provide detailed information about the configuration of the segments. The proposed method is intended for features which can be used for estimation of position and pose of 3D objects, e.g., for the purpose of grasping.

 Epipolar geometry is the geometry situation of two cameras depicting the same scene. For un-calibrated cameras epipolar geometry is compactly described by the fundamental matrix. Estimation of the fundamental matrix is trivial if we have a set of corresponding points in the two images. Corresponding points are often found using e.g. the Harris interest point detector, but there are several advantages with using richer features instead. In this paper we will use blob features. Blobs are homogeneous regions which are compactly described by their colour, area, centroid and inertia matrix. Using blobs to establish correspondences is fast, and the extra information besides position, allows us to reject false matches more accurately.

 The paper describes a method for extracting point features from an image, corresponding to corners and crossings of lines. The method is based on a fourth order tensor representation which can describe the parameters of a local pair of line segments. By considering the rank of the tensor, it is possible to find points which correspond to corners, crossings or junctions. These points can then be further analyzed to provide detailed information about the configuration of the segments. The proposed method is intended for features which can be used for estimation of position and pose of 3D objects, e.g., for the purpose of grasping.

This paper presents a proposal (not entirely new) for combining analytical and algebraic reconstruction techniques. Such a combination bears the promise to improve the image quality of fast but non-exact, reconstruction of the filtered backprojection type. The difference between the present proposal and traditional ART is that we compute a full error image with FBP, applied to projection differences, to update the solution in each iteration step. The main road-block seems to be the same that has been an obstacle for many ART-algorithms in CT applications, namely that the forward projections are subjected to ailiasing, which tend to override the intended benefits of the updating loop. We present an analysis of this problem and indicate some possible solutions.

This paper presents experiments on using integer arithmetic with the channel representation. Integer arithmetic allows reduction of memory requirements, and allows efficient implementations using machine code vector instructions, integer-only CPUs, or dedicated programmable hardware such as FPGAs possible. We demonstrate the effects of discretisation on a non-iterative robust estimation technique called channel smoothing, but the results are also valid for other applications.

We present a two-dimensional information representation, where small but overlapping Gaussian kernels are used to encode the data in a matrix. Apart from points we apply this to constraints that restrict the solution to a linear subspace. A localised decoding scheme accurately extracts multiple solutions together with an estimate of the covariances. We employ the method in optical flow computations to determine multiple velocities occurring at motion discontinuities.

Channel smoothing is an alternative to diffusion filtering for robust estimation of image features. Its main advantages are speed, stability with respect to parameter changes, and a simple implementation. However, channel smoothing becomes instable in certain situations, typically for elongated, periodic patterns like for instance fingerprints. As for the diffusion filtering an anisotropic extension is required in these cases. In this paper we introduce a new method for anisotropic channel smoothing which is comparable to coherence enhancing diffusion, but faster and easier to implement. Anisotropic channel smoothing implements an orientation adaptive non-linear filtering scheme as a special case of adaptive channel filtering. The smoothing algorithm is applied to several fingerprint images and the results are compared to those of coherence enhancing diffusion.

We consider alpha scale spaces, a parameterized class (alpha is an element of (0, 1]) of scale space representations beyond the well-established Gaussian scale space, which are generated by the alpha-th power of the minus Laplace operator on a bounded domain using the Neumann boundary condition. The Neumann boundary condition ensures that there is no grey-value flux through the boundary. Thereby no artificial grey-values from outside the image affect the evolution proces, which is the case for the alpha scale spaces on an unbounded domain. Moreover, the connection between the a scale spaces which is not trivial in the unbounded domain case, becomes straightforward: The generator of the Gaussian semigroup extends to a compact, self-adjoint operator on the Hilbert space L-2(Omega) and therefore it has a complete countable set of eigen functions. Taking the alpha-th power of the Gaussian generator simply boils down to taking the alpha-th power of the corresponding eigenvalues. Consequently, all alpha scale spaces have exactly the same eigen-modes and can be implemented simultaneously as scale dependent Fourier series. The only difference between them is the (relative) contribution of each eigen-mode to the evolution proces. By introducing the notion of (non-dimensional) relative scale in each a scale space, we are able to compare the various alpha scale spaces. The case alpha = 0.5, where the generator equals the square root of the minus Laplace operator leads to Poisson scale space, which is at least as interesting as Gaussian scale space and can be extended to a (Clifford) analytic scale space.

Disparity estimation is a fundamental problem of computer vision. Besides other approaches, disparity estimation from phase information is a quite wide-spread technique. In the present paper, we have considered the influence of the involved quadrature filters and we have replaced them with filters based on the monogenic signal. The implemented algorithm makes use of a scale-pyramid and applies channel encoding for the representation and fusion of the estimated data. The performed experiments show a significant improvement of the results.

Quadrature filters are a well known method of low-level computer vision for estimating certain properties of the signal, as there are local amplitude and local phase. However, 2D quadrature filters suffer from being not rotation invariant. Furthermore, they do not allow to detect truly 2D features as corners and junctions unless they are combined to form the structure tensor. The present paper deals with a new 2D generalization of quadrature filters which is rotation invariant and allows to analyze intrinsically 2D signals. Hence, the new approach can be considered as the union of properties of quadrature filters and of the structure tensor. The proposed method first estimates the local orientation of the signal which is then used for steering some basis filter responses. Certain linear combination of these filter responses are derived which allow to estimate the local isotropy and two perpendicular phases of the signal. The phase model is based on the assumption of an angular band-limitation in the signal. As an application, a simple and efficient point-of-interest operator is presented and it is compared to the Plessey detector.

We introduce a compact coding of image information which explicitely separates geometric information (orientation) and structural information (phase and color). We investigate the importance of these factors for stereo matching on a large data set. From these investigation we can conclude that it is their combination that gives the best results. Concrete weights for their relative importance are measured.

One important problem within the WITAS project is detection of moving objects in aerial images. This paper presents an original method to estimate the displacement between two frames, based on multiscale local polynomial expansions of the images. When the displacement field has been computed, a plane + parallax approach is used to separate moving objects from the camera egomotion.

This paper presents a novel disparity estimation algorithm based on local polynomial expansion of the images in a stereo pair. Being a spin-off from work on two-frame motion estimation, it is primarily intended as a proof of concept for some of the underlying ideas. It may, however, be useful on its own as well, since it is very simple and fast. The accuracy still remains to be determined.

In [Farnebäck00] we presented a new velocity estimation algorithm, using orientation tensors and parametric motion models to provide both fast and accurate results. One of the tradeoffs between accuracy and speed was that no attempts were made to obtain regions of coherent motion when estimating the parametric models. In this paper we show how this can be improved by doing a simultaneous segmentation of the motion field. The resulting algorithm is slower than the previous one, but more accurate. This is shown by evaluation on the well-known Yosemite sequence, where already the previous algorithm showed an accuracy which was substantially better than for earlier published methods. This result has now been improved further.

We present a novel method that finds edges between certain image features, e.g. gray-levels, and disregards edges between other features. The method uses a channel representation of the features and performs normalized convolution using the channel values as certainties. This means that areas with certain features can be disregarded by the edge filter. The method provides an important new tool for finding tissue specific edges in medical images, as demonstrated by an MR-image example.

The purpose of this paper is to provide a broad overview of the WITAS Unmanned Aerial Vehicle Project. The WITAS UAV project is an ambitious, long-term basic research project with the goal of developing technologies and functionalities necessary for the successful deployment of a fully autonomous UAV operating over diverse geographical terrain containing road and traffic networks. Theproject is multi-disciplinary in nature, requiring many different research competences, and covering a broad spectrum of basic research issues, many of which relate to current topics in artificial intelligence. A number of topics considered are knowledge representation issues, active vision systems and their integration with deliberative/reactive architectures, helicopter modeling and control, ground operator dialogue systems, actual physical platforms, and a number of simulation techniques.

This paper presents a general strategy for automated generation of efficient representations in vision. The approach is highly task oriented and what constitutes the relevant information is defined by a set of examples. The examples are pairs of situations that are dependent through the chosen feature but are otherwise independent. Particularly important concepts in the work are mutual information and canonical correlation. How visual operators and representations can be generated from examples are presented for a number of features, e.g. local orientation, disparity and motion. Interesting similarities to biological vision functions are observed. The results clearly demonstrates the potential of combining advanced filtering techniques and learning strategies based on canonical correlation analysis (CCA).

This contest involved the running and evaluation of computer vision and pattern recognition techniques on different data sets with known groundtruth. The contest included three areas; binary shape recognition, symbol recognition and image flow estimation. A package was made available for each area. Each package contained either real images with manual groundtruth or programs to generate data sets of ideal as well as noisy images with known groundtruth. They also contained programs to evaluate the results of an algorithm according to the given groundtruth. These evaluation criteria included the generation of confusion matrices, computation of the misdetection and false alarm rates and other performance measures suitable for the problems. This paper summarizes the data generation for each area and experimental results for a total of six participating algorithms.

Frame representations (e.g. wavelets) and subspace projections are important tools in many image processing applications. A unified framework for frames and subspace bases, as well as bases and subspace frames, is developed for finite dimensional vector spaces. Dual (subspace) bases and frames are constructed and the theory is generalized to weighted norms and seminorms. It is demonstrated how the framework applies to the cubic facet model, to normalized convolution, and to projection onto second degree polynomials.

This paper presents a general strategy for designing efficient visual operators. The approach is highly task oriented and what constitutes the relevant information is defined by a set of examples. The examples are pairs of images displaying a strong dependence in the chosen feature but are otherwise independent. Particularly important concepts in the work are mutual information and canonical correlation. Visual operators learned from examples are presented, e.g. local shift invariant orientation operators and image content invariant disparity operators. Interesting similarities to biological vision functions are observed.

This paper presents a general approach for obtaining optimal filters as well as filter sequences. A filter is termed optimal when it minimizes a chosen distance measure with respect to an ideal filter. The method allows specification of the metric via simultaneous weighting functions in multiple domains, e.g. the spatio-temporal space and the Fourier space. It is shown how convolution kernels for efficient spatio-temporal filtering can be implemented in practical situations. The method is based on applying a set of jointly optimized filter kernels in sequence. The optimization of sequential filters is performed using a novel recursive optimization technique. A number of optimization examples are given that demonstrate the role of key parameters such as: number of kernel coefficients, number of filters in sequence, spatio-temporal and Fourier space metrics. In multidimensional filtering applications the method potentially outperforms both standard convolution and FFT based approaches by two-digit numbers.