Ebook: Neural Nets WIRN10

Welcome to the proceedings of the annual meeting of the Italian Neural Network Society (SIREN). As usual, here you will find fresh scientific news from the Italian community plus contributions by some foreign friends who joined us in Vietri sul Mare (Italy).

After more than 20 years, what motivates us to gather in this annual workshop is a genuine willingness to exchange our latest thoughts on neural networks and related fields. This publication provides an opportunity to read the proceedings, a process which may perhaps have additional value, after reading similar documents from conferences worldwide on the same topics.

All the papers have been peer reviewed and all the authors contributed with original results. Accordingly, the work of the reviewers has been mainly to suggest improvements to the authors and to remedy some possible misprints. We do not claim low acceptance rates as an indication of quality; rather, we constantly promote quality in our community, with the result that our acceptance rate is consistently high.

As for the content, we have a number of works on methodological and implementation topics, which have been discussed in the regular sessions of the conference and which we have grouped into the chapters: Models, Signal Processing and other Applications. Then, we have two chapters referring to special sessions devoted to current focuses in the field. This year they concern “The dynamics of biological networks” and “Nonlinear systems for multimodal human-machine interaction”. This latter session consists of a special activity of the European Cooperation in Science and Technology (COST). The Editors would like to take this opportunity to thank the invited speakers, the contributors and the referees as well as the organizers of both regular and special sessions for their contribution to the realization of this volume. Above all, we hope to meet them all again next year.

Bruno Apolloni

Simone Bassis

Anna Esposito

Francesco Carlo Morabito

We develop a hybrid machine learning architecture, the Influence Relevance Voter (IRV), where an initial geometry- or kernel- based step is followed by a feature-based step to derive the final prediction. While other implementations of the general idea are possible, we use a k-Nearest-Neighbor approach to implement the first step, and a Neural Network approach to implement the second step for a classification problem. In this version of the IRV, the rank and similarities of the k nearest neighbors of an input are used to compute their individual relevances. Relevances are combined multiplicatively with the class membership values to produce influences. Finally the influences of all the neighbors are aggregated to produce the final probabilistic prediction. IRVs have several advantages: they can be trained fast, they are easily interpretable and modifiable, and they are not prone to overfitting since they rely on extensive weight sharing across neighbors. The IRV approach is applied to the problem of predicting whether a given compound is active or not with respect to a particular biochemical assay in drug discovery and shown to perform well in comparison to other predictors. In addition, we also introduce and demonstrate a new approach, the Concentrated ROC (CROC), for assessing prediction performance in situations ranging from drug discovery to information retrieval, where ROC curves are not adequate, because only a very small subset of the top ranked positives is practically useful. The CROC approach uses a change of coordinates to smoothly magnify the relevant portion of the ROC curve.

In this paper regression problems, in which the output is continuous or discrete, are considered. In particular, the Switching Neural Network approach, which has been introduced for classification, is properly extended to deal with regression tasks. The resulting model, named SNN-reg, presents multiple advantages, involving both the quality of the obtained solution and the computational effort needed for its generation. Moreover, SNN-reg allows a regression function to be swritten in terms of a set of intelligible rules, which can be interpreted by the user.

Extreme Learning Machine (ELM) is an approach recently proposed in the literature for Neural Network (NN) training. It has been shown to be much faster than the traditional gradient-based learning algorithms, with many variants, extensions and applications also investigated in the last few years. Among them, the ELM paradigm has been applied to train Time-Variant Neural Networks (TV-NN), through which the training time can be greatly reduced w.r.t. common Back-Propagation (BP). However, this approach may require more hidden nodes and the right type of basis function (through which time-dependence is introduced in TV-NN weight parameters) to attain good generalization. In this paper, we propose a hybrid learning algorithm, which applies differential evolutionary (to determine related input weights) and group selection method (to determine the type of basis function). Experimental results show that the proposed method allows achieving more compact networks yet with better generalization performance at the cost of larger training time; moreover the algorithm behavior is anyway significantly more performing than the BP one.

Within the framework of Algorithmic Inference, we recall a linear regression analysis tool based on the identification of the joint probability distribution of the regression coefficients compatible with the sampled data and aimed at finding out the independent components of this distribution. On this distribution we implement specific Independent Component Analysis (ICA) procedures to obtain the parameter independent components giving rise to suitable confidence regions also when the noise term is far from being independent and identically Gaussian.

The unsupervised analysis of gene expression data plays a very important role in Genetics experiments. That is why a lot of clustering and biclustering techniques have been proposed. Our choice of biclustering methods is motivated by the accuracy in the obtained results and the possibility to find not only rows or columns that provide a partition of the dataset but also rows and columns together. Unfortunately, the experimental data yet contains many inaccuracy and errors, therefore the main task of mathematicians is to find algorithms that permit to analyze this data with maximal precision. In this work, a new biclustering algorithm that permits to find biclusters with an error less than a predefined threshold is presented. The comparison with other known biclustering algorithms is shown.

Assigning functional classes to unknown genes or proteins on diverse large-scale data is a key task in biological systems, and it needs the integration of different data sources and the analysis of functional hierarchies. In this paper we present a method based on Hopfield neural networks which is a variant of a precedent semi-supervised approach that transfers protein functions from annotated to unannotated proteins. Unlike this approach, our method preserves the prior information and takes into account the imbalance between positive and negative examples. To obtain more reliable inferences, we use different evidence sources, and integrate them in a Functional Linkage Network (FLN). Preliminary results show the effectiveness of our approach.

This paper introduces a trading system where decisions are driven by an algorithm belonging to the class of Artificial Immune Systems (AIS). In practice, the system we have built operates according to a two-steps procedure, where, in the first stage, the Negative Selection Algorithm (NSA) runs on historical values of the financial timeseries, while in the second phase, at each time t the outcomes of the NSA are merged into a decision support system that uses them to suggest an active trading position (long, short or standby, i.e.: buy, sell, or doing nothing) at time t + 1. The effectiveness of the procedure is examined using intraday data from the FOReign EXchange market (FOREX), and the results are evaluated mainly under the financial profile by means of typical indicators of financial performances. At the present time, the results suggest that the procedure can be proficiently used especially during downward periods of the market (descending prices), to hedge investors from the probability of higher drawdown.

In the current literature, there are several papers which have considered the modelling and forecasting of population mortality using the Lee-Carter framework. According to Booth (2006), the Lee-Carter-based approach is widely considered because it produces fairly realistic life expectancy forecasts, which are used as reference values for other modelling methods. In recent years, there have been several extensions of the standard LC method, retaining some of its basic properties, but adding additional statistical features too. In 2006, Renshaw and Haberman developed a special adaptation of the LC method. They transformed the basic LC model into a more general framework in order to analyse the relationship between age and time and their joint impact on the mortality rates. This transformation gave birth to the so-called age-period-cohort (APC) log-bilinear generalized linear models (GLM) with Poisson error structures. In this paper, we take into consideration a family of generalised log-linear models of the LC type structure with Poisson errors that includes the basic LC model too. In this framework, we implement a specialised iterative regression methodology based on Poisson likelihood maximization process. In particular, we make use of the approach proposed and illustrated in Renshaw and Haberman (2006), which generalises the basic LC modelling framework to develop a tailored iterative process for updating the parameter estimates. In order to assess the goodness of fit of the regression, we provide a range of residual analyses with corresponding target fitted values. Diagnostic plots are provided to show the results.

In this paper a blind source separation algorithm in convolutive environment is presented. In order to avoid the classical permutation ambiguity in the frequency domain solution, a geometrical constraint is considered. Moreover a beam-former algorithm is integrated with the proposed solution: in this way the directivity pattern of the proposed architecture can take into account the residual permutation at low frequencies and the scaling inconsistency. Several experimental results are shown to demonstrate the effectiveness of the proposed method.

In this paper a pre-filtering and a post-filtering approach to blind source separation in reverberant environment is presented. The preprocessing consists in the use of common acoustical poles that can simplify the recovering network, giving some a priori information on the environment. In particular the autoregressive part of a transfer function in a closed environment is common for all positions. After pre-filtering conventional BSS algorithm in frequency domain is applied to get estimates of original sources. In addition an adaptive noise canceler is used as post-filter in order to enhance the quality of the separation. Some experimental results demonstrate the effectiveness of the proposed approach.

Electroencephalographic (EEG) recordings are often contaminated by artifacts, i.e. signals that have no-cerebral origin and therefore distort the EEG analysis. In this paper we propose a wavelet analysis useful to detect impulsive artifacts, like spike and eye blink; in particular we use two threshold's methods based on the discrete wavelet transform (DWT) and on the stationary wavelet transform (SWT), respectively. Both methods are equivalent in the identification of eye blink artifact, but we find a different behavior in the spikes detection. Using the DWT we observed that the spike detection is decimation-sensitive, in other words the even or odd decimation corrupts the artifact identification step. On the other hand, the translational invariance property of SWT allows to overcome this limitation improving the wavelet analysis for the EEG artifacts detection.

In classification problems, lack of knowledge of the prior distribution may make the application of Bayes' rule inadequate. Uniform or arbitrary priors may often provide classification answers that, even in simple examples, may end up contradicting our common sense about the problem. Entropic priors, determined via the application of the maximum entropy principle, seem to provide a much better answer and can be easily derived and applied to classification tasks when no more than the likelihood functions are available. In this paper we present an example in which the use of the entropic priors is compared to the results of the application of Dempster-Shafer theory.

In this paper we employ the Kohonen's Self Organizing Map (SOM) as a strategy for an unsupervised analysis of two IKONOS multispectral images of different dates. The main object is the development of an automatic multi-temporal analysis methodology of the land use modifications through change detection techniques using remotely sensed data. In order to obtain an accurate segmentation of changes we introduce as input for the network, in addition to spectral data, some texture measures, which give an essential contribution to the classification of changes in man-made structures. Furthermore we introduce a classical statistical method based on the image differencing and we evaluate the classification performances of the proposed approaches. We propose the results obtained with different combinations of the multi-temporal input data and compare them with prior knowledge of the scene analyzed.

We present a non-destructive magnetic imaging technique developed for the evaluation of hot-rolled stainless steel. Starting from measurements of magnetic field carried out by means of a Hall probe array, magnetic images of sample surfaces are attained. In order to enhance the signal to noise ratio and to detect defects, we have implemented some image processing protocols. In particular we report the contextual application of Independent Component Analysis and Wiener filter to the image deconvolution task focusing on the advantages that such approach assures.

MicroRNAs are a hot topic of research in molecular biology. Their role is however still to be elucidated in full. We present a data-oriented approach to the study of microRNA-gene interactions, building heavily on methods of document analysis. The paper aims at illustrating an approach rather than specific methods of analysis, however some sample results are also presented, which show how latent information can be exploited to suggest directions for laboratory experiments, thus avoiding unnecessary expense in time and resources.

In this paper we present how, using a careful definition of a state function, long animation sequences can be created joining clips from a database. Each next clip is chosen in real-time by a controller optimizing a cost function on the state function; this allows the user interact in real-time with the digital character. We analyze here two possible cost functions, one that is based on the evaluation of the compatibility of the next clip and one based on reinforcement learning in which the global policy of the controller is evaluated.

In a previous work [1] the design of a reader device for blind people was presented from the algorithmic standpoint. In this paper we will discuss the study, design and development of its evolution toward an embedded appliance. Two hardware platforms will be analyzed, considering their pros and cons respect to the target application, based on image processing algorithms and Artificial Neural Networks (ANNs). The comparison will take into account the available computational power, the power consumption, the ease of interfacing with other devices and the overall development cycle.

A feasibility study will be outlined and, in the end, one platform will be selected as the base for the proposed reader device for blind people. Run-time performance results of the software running on such platform will be reported.

Pulsed Eddy Current (PEC) is a new emerging Non Destructive Evaluation technique for sub-surface defect detection. It provides new challenges to signal analysis and interpretation approach applied to the inspection evaluation. For instance, PEC could suffer from noise and be not sufficient to extract more information about the defects. This paper aims to approach the challenge of flaw identification in PECs. Due to non-Gaussianity of PEC measurements, we applied Independent Component Analysis (ICA) in extracting information from PEC responses. We considered three different approaches implementing ICA, in order to project the response signals of various defects into the Independent Components (ICs) feature space. Then, useful ICs of each algorithm were used as features for machine learning algorithms, in order to solve the inverse problem of pattern classification. Since the nongaussianity of the OEC measurements, we retained ICs with highest kurtosis. The considered different kinds of defects were: metal loss, sub-surface cracks, surface defects and slants. We compared the performances of our implemented algorithms with results available in scientific literature. We obtained improvements in reliability of the pattern classification algorithm, as well as in reducing the computational load, obtaining a classification error of 8.54% over 3063 testing patterns.

Spatial forecasting of physical environmental parameters like temperature and humidity, can be realized by soft sensors based on neural networks. The paper is focused on the use of an original neural network model named E-αNet to realize a soft sensors system. E-αNet introduces the concept of “automatic learning” of the activation functions, reducing the complexity of the net in terms of number of hidden units and improving the learning capability. A comparison among different architecture models led from statistical and metrological points of view, shows how E-αNet produces interesting results in a real world application (a non invasive monitoring of the conservation state of old monument).

Among the Web opportunities, e-Commerce processes have increased in relevance requiring the development of complex tools to support all the parties involved therein. This paper proposes a neural network hybrid recommender system able to provide customers, associated with XML-based personal agents within a multi-agent system called MARF, with suggestions about flights purchases. MARF agents continuously monitor customers' interests and preferences in their commercial Web activities, by constructing and automatically maintaining their profiles. In order to highlight the benefits provided by the proposed flight recommender, some experimental results carried out by exploiting a MARF prototype are presented.

This study presents a neural-based algorithm for the automatic detection of landslides on Stromboli volcano (Italy). It has been shown that landslides are an important short-term precursor of effusive eruptions of Stromboli. In particular, an increase in the occurrence rate of landslides was observed a few hours before the beginning of the February 2007 effusive eruption. Automating the process of detection of these signals can help analysts and represents a useful tool for the monitoring of the stability of the Sciara del Fuoco flank of Stromboli volcano. A multi-layer perceptron neural network is here applied to continuously discriminate landslides from other signals recorded at Stromboli (e.g., explosion quakes, tremor signals), and its output is used by an automatic system for the detection task. To correctly represent the seismic data, coefficients are extracted from both the frequency domain, using the linear predictive coding technique, and the time domain, using temporal waveform parameterization. The network training and testing was carried out using a dataset of 537 signals, from 267 landslides and 270 records that included explosion quakes and tremor signals. The classification results were 99.5% predictive for the best net performance, and 98.7% when the performance was averaged over different net configurations. Thus, this detection system was effective when tested on the 2007 effusive eruption period. However, continuing investigations into different time intervals are needed, to further define and optimize the algorithm.