Randomized Neural Networks, Kernel Ridge Regression and Non-iterative Learning

This page presents our research centered around randomized neural networks, kernel ridge regression and related topics in the area of non-iterative learning.

1.    L. Zhang, P. N. Suganthan, "A Comprehensive Evaluation of Random Vector Functional Link Networks," Information Sciences, DOI: 10.1016/j.ins.2015.09.025.  Software Available here.

In the context of classification, the above work shows that the direct links in RVFL improve the performance of the single layer feedforward Network (SLFN) when randomized weights are used between inputs and hidden neurons, suggesting that direct-links regularize the effects of randomization. In other words, instead of using only randomized features, mixing the randomized features with the original features performs better. Further, randomization range can also be tuned to improve performance.121 UCI datasets were used in our study. Our experimental framework is identical to the one used in the following publication:
M. Fernández-Delgado, E. Cernadas, S. Barro, D. Amorim, "Do we Need Hundreds of Classifiers to Solve Real World Classification Problems?", JMLR, 15(Oct):3133−3181, 2014.

2.     Y. Ren, P.  N. Suganthan, N. Srikanth, G. Amaratunga, "Random Vector Functional Link Network for Short-term Electricity Load Demand Forecasting", Information Sciences, Software Available here.   Also from: https://github.com/ron1818/PhD_code/
In the context of time series forecasting, the above work shows that the direct links in RVFL improve the performance in a statistically significant manner. The direct links can be compared to the time delayed line in the finite impulse response filter (FIR). Randomization range can also be tuned to improve performance. RVFL with direct input-output links slightly outperformed SVR also. 18 time series datasets were used in this study.


3.      L. Zhang, P. N. Suganthan, "A Survey of Randomized Algorithms for Training Neural Networks," Information Sciences, DoI: 10.1016/j.ins.2016.01.039, accepted, 2016.

 Promising Future Research Direction: There is an increase in research activities using randomized neural networks. Researchers are encouraged to further investigate the role of direct links in RVFL and other randomized neural network configurations.


 Related Special Issues and Special Sessions:

1.      Applied Soft Computing Special Issue on Non-iterative Learning methods. Expected publication in early 2018.

2.      IJCNN 2018 special session on non-iterative learning.

3.      Cognitive Computation journal special issue on non-iterative learning.