Methods and Techniques for Multi-source Data Analysis and Fusion

This work has been inspired by the recent trend in remote sensing and environmental data acquisition. Remote sensing techniques allow us to measure information about an object without touching it. In the last decades remote sensing via satellites has been used in various applications such as Earth observation, weather and storm predictive analysis, atmospheric monitoring, climate change, human-environment interactions. Sensors on airborne and satellite platforms have been recording signals from space for many years, giving rise to a huge amount of data. Some data are processed on-board but others are treated and post-processed in ground stations. Signal and image processing are widely applied on data coming from satellites to extract meaningful information for the aforementioned tasks. Satellites and ground stations communicate with each others by using several transceivers and techniques. After acquisition, data has to be processed and correlated to generate forecasts and detect unusual phenomena. Human attention works very well when detecting salient patches in an image, and this skill is frequently used to segment areas of interest in satellite images. Saliency is the quality that makes certain regions of an image stand out from the rest of the visual field and grab our attention. Due to the relevance of visual saliency in remote imaging and its applicability to many other scientific and psychological research areas, the main part of this work focuses on the study of theoretical saliency models, the research of novel methods for saliency detection in images and the characterization of human attention on sub-object details. As an extension of these, investigated the effects of Colour Vision Deficiencies (CVDs) on attention and new techniques to alter image colours depending on saliency values have been investigated, in order to restore fixation point coherence between CVD affected and normal observers. In this work we propose two remote sensing systems, composed by low cost hardware components and portable software solutions, designed to receive meteorological satellite signals. We aim at sampling and processing of the modulated signal entirely in software enabled by Software Defined Radios (SDR) and CPU computational speed overcoming hardware limitations such as high receiver noise and low ADC resolution. For this purpose we developed two software-hardware integrated systems able to perform the following steps: satellite pass prediction, time scheduling, signal demodulation, image cropping and filtering. Then, cloud detection is performed by using two well known clustering algorithms, Otsu and K-means. Although we employed low cost components, we obtained good results in terms of signal demodulation, synchronization and image reconstruction. In order to emulate human attention to detect and segment salient patches in images (such as satellite images) we have developed two saliency algorithms based on Keypoint Density Maps. Our algorithms work by analyzing the spatial density of keypoints detected in images converted to perceptually uniform color spaces (CIE L*a*b* and CIE L*u*v*); both scale-aware and multi-scale approaches have been implemented in our methods. Furthermore, we compared our methods against the most important and popular saliency detection methods in the state of the art. It is shown that our approaches achieve good results in all experiments. As a reference for performance evaluation, we collected a dataset of eye fixations on objects and sub-object details using eye-tracking technology; the dataset is called Eye-Tracking Through Objects (ETTO) and it is publicly available. As already said, we developed a technique to use the above-mentioned saliency algorithms to re-color images for individuals affected by color vision deficiencies. Color vision deficiencies affect visual perception of colors and, more generally, color images. We eye-tracked the human fixations in first three seconds of observation of color images to build real fixation point maps, then we detected the main differences between the aforementioned human visual systems related to color vision deficiencies by analyzing real fixation maps among people with and without color vision deficiencies. In this work we provide a method to enhance color regions of the image by using a detailed color mapping of the segmented salient regions of the given image. The segmentation is performed by using the saliency difference between the original input image and the corresponding color blind altered image. A second eye-tracking of color blind people with the images enhanced by using re-coloring of segmented salient regions reveals that the real fixation points are then more coherent (up to 10%) with the normal visual system. The eye-tracking data collected during our experiments are in a publicly available dataset called Eye-Tracking of Colour Vision Deficiencies (EToCVD).