Extreme Learning Machines (ELM) - Learning without iterative tuning
Neural networks (NN) and support vector machines (SVM) play key roles in machine learning and data analysis. Feedforward neural networks and support vector machines are usually considered different learning techniques in computational intelligence community. Both popular learning techniques face some challenging issues such as: intensive human intervene, slow learning speed, poor learning scalability.
It is clear that the learning speed of feedforward neural networks is in general far slower than required and it has been a major bottleneck in their applications for past decades. Two key reasons behind may be: 1) the slow gradient-based learning algorithms are extensively used to train neural networks, and 2) all the parameters of the networks are tuned iteratively by using such learning algorithms. On the other hand, due to their outstanding classification capability, support vector machine and its variants such as least square support vector machine (LS-SVM) have been widely used in binary classification applications. The conventional SVM and LS-SVM cannot be used in regression and multi-class classification applications directly although different SVM/LS-SVM variants have been proposed to handle such cases.
ELM works for the “generalized” single-hidden layer feedforward networks (SLFNs) but the hidden layer (or called feature mapping) in ELM need not be tuned. Such SLFNs include but are not limited to support vector machine, polynomial network, RBF networks, and the conventional (both single-hidden-layer and multi-hidden-layer) feedforward neural networks. Different from the tenet in neural networks that all the hidden nodes in SLFNs need to be tuned, ELM learning theory shows that the hidden nodes of generalized feedforward networks needn’t be tuned and these hidden nodes can be randomly generated. All the hidden node parameters are independent from the target functions or the training datasets. All the parameters of ELMs can be analytically determined instead of being tuned.
According to ELM theory:
The hidden node parameters are not only independent of the training data but also of each other.
Unlike conventional learning methods which MUST see the training data before generating the hidden node parameters, ELM could generate the hidden node parameters before seeing the training data.
ELM was originally proposed for standard single hidden layer feedforward neural networks (with random hidden nodes (random features)), and has recently been extended to kernel learning as well:
ELM is efficient in:
ELM has been successfully used in the following applications:
Due to the demand on ELM solutions, ELM may help drive R&D in the following areas and make some applications which seem impossible in the past become true in the future:
The BMW-NTU Joint Future Mobility Research Lab has been recently setup. Several PhD scholarships and Project leaders positions are avaiable. Applicants can submit their application to D-ERIAN@ntu.edu.sg
Two research positions on ELM research (one for realtime clustering and one for Internet data analytics) are available. Students who wish to pursue PhD degree or researchers who are interested in the two positions are welcome to write to firstname.lastname@example.org with detail CV. Only shortlisted will be informed for further discussion.
Beijing, October 15 - 17 2013
Hefei, China 20-23 October 2013
Organzied by The USTC-Birmingham Joint Research Institute in Intelligent Computation and Its Applications
Free counter Last update :