Tesis doctoral de Enrique Romero Merino
1. An algorithm for sequential approximation with fnns, named sequential approximation with optimal coefficients and interacting frequencies (saocif). most of the sequential approximations proposed in the literature select the new frequencies (the non-linear weights) guided by the approximation of the residue of the partial approximation. We propose a sequential algorithm where the new frequency is selected taking into account its interactions with the previously selected ones. The interactions are discovered by means of their optimal coefficients (the linear weights). A number of heuristics can be used to select the new frequencies. The aim is that the same level of approximation may be achieved with less hidden units than if we only try to match the residue as best as possible. In terms of the bias/variance decomposition, it will be possible to obtain simpler models with the same bias. The idea behind saocif can be extended to approximation in hilbert spaces, maintaining orthogonal-like properties. In this case, the importance of the interacting frequencies lies in the expectation of increasing the rate of approximation. Experimental results show that the idea of interacting frequencies allows to construct better approximations than matching the residue. 2. A study and comparison of different criteria to perform feature selection (fs) with multi-layer perceptrons (mlps) and the sequential backward selection (sbs) procedure within the wrapper approach. Fs procedures control the bias/variance decomposition by means of the input dimension, establishing a clear connection with the curse of dimensionality. Several critical decision points are studied and compared. First, the stopping criterion. second, the data set where the value of the loss function is measured. finally, we also compare two ways of computing the saliency (i.E., The relative importance) of a feature: either first train a network and then remove temporarily every fe
Datos académicos de la tesis doctoral «Learning with feed-forward neural networks: three schemes to deal with the bias/variance trade-off«
- Título de la tesis: Learning with feed-forward neural networks: three schemes to deal with the bias/variance trade-off
- Autor: Enrique Romero Merino
- Universidad: Politécnica de catalunya
- Fecha de lectura de la tesis: 30/11/2004
Dirección y tribunal
- Director de la tesis
- René Alquézar Mancho
- Tribunal
- Presidente del tribunal: José Luis Balcazar navarro
- gustavo Deco (vocal)
- sarunas Raudys (vocal)
- cecilio Angulo bahon (vocal)