Ivor Wai-Hung TSANG Assistant Professor School of Computer Engineering Nanyang Technological University Singapore 639798 Email: IvorTsang (at) ntu (dot) edu (dot) sg Ivor (dot) Tsang (at) gmail (dot) com

Opening for a Research Assistant Position in Natural Language Processing and Text Mining. Please send me a copy of your full CV

Call for Papers: Asian Conference on Machine Learning 2012, Singapore

The software of Feature Generating Machine for extremely high dimensional feature selection is now available at here, and some lecture notes are available at Machine Learning Summer School 2011

Dr Ivor Wai-Hung Tsang is an Assistant Professor in the School of Computer Engineering of Nanyang Technological University. He is also the Deputy Director of the Centre for Computational Intelligence. He received his Ph.D. degree in Computer Science from the Hong Kong University of Science and Technology (HKUST) in 2007. His research focuses on support vector machines, very high dimensional feature selection, large scale machine learning, transfer learning, and their applications to data mining and pattern recognitions. He has more than 80 research papers published in refereed international journals and conference proceedings, including Journal of Machine Learning Research (JMLR), IEEE Transactions on Neural Networks and Learning Systems (TNNLS), IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Neural Computation, NIPS, ICML, UAI, AISTATS, SIGKDD, IJCAI, AAAI, ICCV, CVPR, etc. He has given invited lectures and talks on large-scale machine learning and extremely high dimensional feature selection at Machine Learning Summer School (MLSS), and many world-class universities and research institutes.

In 2009 Dr Tsang was conferred the 2008 Natural Science Award (Class II) by Ministry of Education, China, which recognizes his contributions to kernel methods. Besides this, he received the prestigious IEEE Transactions on Neural Networks Outstanding 2004 Paper Award in 2006. His co-authored paper concerning transfer learning for visual event recognition won the Best Student Paper Award at the 23rd IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2010). His co-authored work on outlier detection clinched the Best Paper Award at the 23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2011), and also the 2011 Best Student Paper (Gold Prize) from PREMIA, Singapore. His work on speech adaptation earned him the Best Paper Award from the IEEE Hong Kong Chapter of Signal Processing Postgraduate Forum in 2006. He was also awarded the Microsoft Fellowship in 2005.

"Theory without Practice is empty; but Practice without Theory is blind" - Immanuel Kant

Machine learning and Data mining:

Feature selection, Sparse representation and Sparse coding
Transfer learning, Domain adaptation, Dataset shift, Multitask learning and Distribution matching
Learning from ambiguity, Multiple instance learning, Outlier detection, Clustering, Unsupervised and Semi-supervised learning
Core vector machines, Coreset approximation, Combinatorial optimization, Large-scale Learning and Prediction
Kernel learning, Output-Kernel learning and Beyond

Applications:

Internet vision, Web-Scale Multimedia retrieval and annotation
Computer vision, Scene and Object recognition
Text mining, Word segmentation and Information extraction

Co-PI, A*STAR TSRP – "Large Scale Hybrid Storage System"
Co-PI, A*STAR TSRP – "User and Domain-Driven Data Analytics as a Service Framework (UDDDASF)"
Co-PI, Rolls Royce Project – "Large-Scale Data Management"
NTU-PI, DSO – "Towards Automatic Template Extraction with Minimal Human Supervision"
PI, NTU and A*STAR IHPC Joint Project – "Large Scale Domain Adaptation Machines: Information Integration, Revolution and Transfer"
Co-PI, NRF IDM – "Next Generation Annotation Techniques for Consumer Photos and Videos"
PI, MOE AcRF Tier 1 – "Transferable Kernel Machines: Knowledge Integration, Extraction and Transfer"
PI, NTU Startup Grant

Yiteng Zhai, Mingkui Tan, Ivor W. Tsang, Yew-Soon Ong. Discovering Support and Affiliated Features from Very High Dimensions. To appear in the 29th International Conference on Machine Learning (ICML), 2012.

Qi Mao, Ivor W. Tsang. Multiple Template Learning for Structured Prediction. arXiv CoRR 1103.0890 (link)

Qi Mao, Ivor W. Tsang. Optimizing Performance Measures for Feature Selection. Proceedings of the IEEE International Conference on Data Mining (IEEE ICDM 2011), Vancouver, Canada, Dec 2011. (PDF) (More results on Multi-Instance Learning via Embedded Feature Selection can be found at arXiv)
A two-layer cutting plane algorithm together with a primal MKL algorithm are proposed for optimizing performance measures for feature selection where the problem has exponential size of both feature groups and label configurations for a given dataset

Mingkui Tan, Li Wang, Ivor W. Tsang. Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets. Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, Israel, June 2010. (PDF) (Software)
A linear-time SVM-type feature selection algorithm is proposed for large-scale and extremely high dimensional datasets, a very small subset of non-monotonic features can be identified from 3 Million features for suspicious URLs prediction

Mingkui Tan, Ivor W. Tsang, Li Wang, Xinming Zhang. Convex Matching Pursuit for Large-scale Sparse Coding and Subset Selections. To appear in the 26th AAAI Conference on Artificial Intelligence (AAAI), 2012.

Shenghua Gao, Ivor W. Tsang, Liang-Tien Chia. Laplacian Sparse Coding, Hypergraph Laplacian Sparse Coding, and Applications. To appear in IEEE Transactions on Pattern Analysis and Machine Intelligence. (link)
Our Laplacian Sparse Coding achieves 89.78% accuracy on Scene15 dataset and 85.27% accuracy on UIUC-Sport dataset

Shenghua Gao, Liang-Tien Chia, Ivor W. Tsang. Multi-layer Group Sparse Coding - for Concurrent Image Classification and Annotation. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2011), Colorado Springs, Colorado, June 2011. (PDF)
How can image classification and image annotation help each other ?

Shenghua Gao, Ivor W. Tsang, Liang-Tien Chia. Kernel Sparse Representation for Image Classification and Face Recognition. Proceedings of the 11th European Conference on Computer Vision (ECCV 2010), Crete, Greece, September 2010. (PDF)

Shenghua Gao, Ivor W. Tsang, Liang-Tien Chia, Peilin Zhao. Local Features Are Not Lonely - Laplacian Sparse Coding for Image Classification. Proceedings of the 23rd IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2010), San Francisco, CA, June 2010. (PDF)
The local dependency among sparse codes is introduced to preserve local consistence of sparse representation

Jian Bo Yang, Qi Mao, Qiao Liang Xiang, Ivor W. Tsang, Kian Ming Adam Chai and Hai Leong Chieu. Domain Adaptation for Coreference Resolution: An Adaptive Ensemble Approach. To appear in the Conference on Empirical Methods in Natural Language Processing and Conference on Natural Language Learning (EMNLP-CoNLL), 2012.

Lixin Duan, Dong Xu, Ivor W. Tsang. Learning with Augmented Features for Heterogeneous Domain Adaptation. To appear in the 29th International Conference on Machine Learning (ICML), 2012.

Liang Feng, Yew-Soon Ong, Ivor W. Tsang, Ah-Hwee Tan. An Evolutionary Search Paradigm that Learns with Past Experiences. To appear in the IEEE World Congress on Computational Intelligence, Congress on Evolutionary Computation, 2012.
How to transfer knowledge for solving NP hard optimization problems ?

Lixin Duan, Dong Xu, Ivor W. Tsang, Jiebo Luo. Visual Event Recognition in Videos by Learning from Web Data. To appear in IEEE Transactions on Pattern Analysis and Machine Intelligence. (PDF)

Lixin Duan, Ivor W. Tsang, Dong Xu. Domain Transfer Multiple Kernel Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(3):465-479, March 2012. (PDF)

Lixin Duan, Dong Xu, Ivor W. Tsang. Domain Adaptation from Multiple Sources: A Domain-Dependent Regularization Approach. IEEE Transactions on Neural Networks and Learning Systems, 23(3):504 - 518, March 2012. (PDF)

Chun-Wei Seah, Ivor W. Tsang, Yew-Soon Ong. Healing Sample Selection Bias by Source Classifier Selection. Proceedings of the IEEE International Conference on Data Mining (IEEE ICDM 2011), pp. 577 - 586, Vancouver, Canada, Dec 2011. (PDF) [Full Paper, Acceptance rate = 13%]
How to handle the change of class distribution in training source data and test target data ? How to alleviate Negative Transfer ? How to efficiently handle large-scale transduction ? How to select and combine source classifiers for the target domain ?

Shukai Li, Ivor W. Tsang. Learning to Locate Relative Outliers. Proceedings of the Asian Conference on Machine Learning (ACML), Journal of Machine Learning Research W & CPs, Vol. 20, 47--62, Nov 2011. (PDF)
How to use a reference dataset and output-kernel learning to robustly locate the relative outliers (the changes) in the target domain ?

Yiming Liu, Dong Xu, Ivor W. Tsang, Jiebo Luo. Textual Query of Personal Photos Facilitated by Large-scale Web Data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5): 1022 - 1036, May 2011. (PDF)
Automatic Web Image Retrieval is introduced to collect loosely labeled web images for a textual query. Then a cross-domain relevant feedback using loosely labeled web images is proposed to retrieve personal photos

Sinno Jialin Pan, Ivor W. Tsang, James T. Kwok and Qiang Yang. Domain Adaptation via Transfer Component Analysis. IEEE Transactions on Neural Networks, 22(2): 199 - 210, Feb 2011. (PDF)

Bo Chen, Wai Lam, Ivor W. Tsang, Tak-Lam Wong. Location and Scatter Matching for Dataset Shift in Text Mining. Proceedings of the IEEE International Conference on Data Mining (IEEE ICDM 2010), Sydney, Australia, Dec 2010.(PDF)

Chun-Wei Seah, Ivor W. Tsang, Yew-Soon Ong, Kee-Khoon Lee. Predictive Distribution Matching SVM for Multi-Domain Learning. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2010), Barcelona, Spain, September 2010. (PDF)
We proposed Predictive Distribution Matching SVM (PDM-SVM), which iteratively constructs a graph for identifying the patterns of positive transfer and negative transfer from multiple sources, to address Negative Transfer in Domain Adaptation

Lixin Duan, Dong Xu, Ivor W. Tsang, Jiebo Luo. Visual Event Recognition in Videos by Learning from Web Data. Proceedings of the 23rd IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2010), San Francisco, CA, June 2010. (PDF) [Oral, Acceptance rate = 4.5%]
#This paper was awarded with the best student paper prize at the 23rd IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2010)

Brian Mak, Tsz-Chung Lai, Ivor W. Tsang, James T. Kwok. Maximum Penalized Likelihood Kernel Regression for Fast Adaptation. IEEE Transactions on Audio, Speech and Language Processing, 17(7): 1372-1381, September 2009. (PDF)
Maximum Penalized Likelihood Kernel Regression is proposed to adapt a new HMM from the structure of existing Hidden Markov Models (HMMs) for a new speaker with very few speech data

Lixin Duan, Ivor W. Tsang, Dong Xu, Tat-Seng Chua. Domain Adaptation from Multiple Sources via Auxiliary Classifiers. Proceedings of the 26th International Conference on Machine Learning (ICML 2009), Montreal, Quebec, June 2009. (PDF)
How to use target unlabeled data and auxiliary classifiers from multiple sources to learn the target classifier for domain adaptation ?

Bo Chen, Wai Lam, Ivor W. Tsang, Tak-Lam Wong. Extracting Discriminative Concepts for Domain Adaptation in Text Mining. Proceedings of the 15th ACM SIGKDD conference on Knowledge Discovery and Data Mining (KDD 2009), Paris, June 2009. (PDF)
Discriminative Subspace Extraction and Feature Propagation are proposed for cross-domain text mining

Lixin Duan, Ivor W. Tsang, Dong Xu, Stephen J. Maybank. Domain Transfer SVM for Video Concept Detection. Proceedings of the 22nd IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami Beach, Florida, USA, June 2009.(PDF)
Domain Transfer SVM combines both Multiple Kernel Learning (MKL) and Distribution Matching via MMD for Semi-Supervised Learning/Transduction

Wen Li, Lixin Duan, Ivor W. Tsang, Dong Xu. Batch Mode Adaptive Multiple Instance Learning for Computer Vision Tasks. To appear in the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2012).

Qi Mao, Ivor W. Tsang. Optimizing Performance Measures for Feature Selection. Proceedings of the IEEE International Conference on Data Mining (IEEE ICDM 2011), Vancouver, Canada, Dec 2011. (PDF) (More results on Multi-Instance Learning via Embedded Feature Selection can be found at arXiv)
A two-layer cutting plane algorithm together with a primal MKL algorithm are proposed for optimizing performance measures for feature selection where the problem has exponential size of both feature groups and label configurations for a given dataset

Lixin Duan, Wen Li, Ivor W. Tsang, Dong Xu. Improving Web Image Search by Bag-based Re-ranking. IEEE Transactions on Image Processing, 20(11):3280--3290, November 2011. (PDF)
The notion of different level of ambiguity in positive bags and negative bags are introduced. How to automatically create noisy label for unlabeled web data ? How to handle a large amount of label noise in web data ?

Wen Li, Lixin Duan, Dong Xu, Ivor W. Tsang. Text-based Image Retrieval using Progressive Multi-Instance Learning. Proceedings of the International Conference on Computer Vision (ICCV 2011), Barcelona, Spain, Nov 2011. (PDF)
How to select reliable positive bags progressively to improve the performance of MIL ?

Yu-Feng Li, James T. Kwok, Ivor W. Tsang, Zhi-Hua Zhou. A Convex Method for Locating Regions of Interest with Multi-Instance Learning. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2009), pp.15-30, Bled, Slovenia, September 2009. (PDF) (Software)
Instead of bag prediction only, two convex and scalable Key-Instance SVMs via output-kernel learning are proposed to identify the Key Instance (KI) in positive bags and to predict bags simultaneously. Instance level KI-SVM can accurately locate the region of KI inside a positive bag; while Bag level KI-SVM achieves better bag prediction performance

Chun-Wei Seah, Ivor W. Tsang, Yew-Soon Ong. Transductive Ordinal Regression. To appear in IEEE Transactions on Neural Networks and Learning Systems. (PDF)

Lin Chen, Ivor W. Tsang, Dong Xu. Laplacian Embedded Regression for Scalable Manifold Regularization. IEEE Transactions on Neural Networks and Learning Systems, 23(6):902 - 915, June 2012. (PDF)
How to solve large-scale Manifold Regularization ?

Jinfeng Zhuang, Ivor W. Tsang, Steven C. Hoi. A Family of Simple Non-Parametric Kernel Learning Algorithms. Journal of Machine Learning Research(JMLR), 12:1313-1347, 2011.(PDF)
A scalable algorithm is proposed to solve a family of large-scale SDP problems, including Colored Maximum Variance Unfolding, Minimum Volume Embedding, Structure Preserving Embedding and other non-parametric kernel learning problems

Qi Mao, Ivor W. Tsang. Parameter-Free Spectral Kernel Learning. Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010), pp. 350-357, Catalina Island, California, July 2010. (PDF) [Plenary Oral, Acceptance rate = 11.5%]
For a given Laplacian matrix, a parameter-free Spectral Kernel Learning is proposed to learn an ideal kernel with the closed-form solution for transduction

Jinfeng Zhuang, Ivor W. Tsang, Steven C. Hoi. SimpleNPKL: Simple Non-Parametric Kernel Learning. Proceedings of the 26th International Conference on Machine Learning (ICML 2009), Montreal, Quebec, June 2009. (PDF)
By exploiting the sparse structure, the Large-Scale Semi-Definite Programming (SDP) problem of Non-Parametric Kernel Learning with Pairwise Constraints can be solved by a very simple and scalable convex algorithm. A 5000-by-5000 non-parametric kernel matrix (25M variables) can be learned within a minute

Ivor W. Tsang, James T. Kwok. Large-scale sparsified manifold regularization. Proceedings of the Advances in Neural Information Processing Systems (NIPS), pp. 1401-1408, Vancouver, Canada, December 2006. (PDF) [Plenary Oral, Acceptance rate = 3%] (A preliminary version appeared in NIPS 2005 Workshop on Large Scale Kernel Machines)
The manifold regularizer is transformed as pairwise constraints, which avoids matrix inversion

James T. Kwok, Ivor W. Tsang. Learning with idealized kernels. Proceedings of the International Conference on Machine Learning (ICML 2003), pp.400-407, Washington, D.C., USA, August 2003. (PDF)
Combination of BOTH Similar and Dissimilar Side information is firstly introduced as Distance constraints for learning a kernel in semi-supervised setting

Qiaoliang Xiang, Qi Mao, Kian Ming Chai, Hai Leong Chieu, Ivor W. Tsang, Zhenddong Zhao. A Split-Merge Framework for Comparing Clusterings. To appear in the 29th International Conference on Machine Learning (ICML), 2012.

Feiping Nie, Zinan Zeng, Ivor W. Tsang, Dong Xu, Changshui Zhang. Spectral Embedded Clustering: A Framework for In-Sample and Out-of-Sample Spectral Clustering. IEEE Transactions on Neural Networks, 22(11): 1796 - 1808, Nov 2011. (PDF)
How to do Out-of-Sample Spectral Clustering ?

Wenliang Zhong, Weike Pan, James T. Kwok, Ivor W. Tsang. Incorporating the Loss Function into Discriminative Clustering of Structured Outputs. IEEE Transactions on Neural Networks, 21(10): 1564 - 1575, Oct 2010. (PDF)
How to combine HSIC and Structured output loss function for clustering ? How to improve K-means clustering ?

Yu-Feng Li, Ivor W. Tsang, James T. Kwok, Zhi-Hua Zhou. Tighter and Convex Maximum Margin Clustering. Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS) 2009, Journal of Machine Learning Research W & CPs, Vol. 5, pp. 344-351, Clearwater Beach, Florida, USA, April 2009. (PDF) (Software) [Plenary Oral, Acceptance rate = 10%]
The mixed integer program in MMC can be solved by a scalable and efficient multiple output-kernel learning algorithm, which achieves a tighter approximation than the SDP relaxation

Kai Zhang, Ivor W. Tsang, James T. Kwok. Maximum Margin Clustering Made Practical. IEEE Transactions on Neural Networks, 20(4): 583-596, April 2009. (PDF) (Software)
Why do local optimization methods for MMC/TSVM easily get stuck in local minima ?

Shukai Li, Ivor W. Tsang. Learning to Locate Relative Outliers. Proceedings of the Asian Conference on Machine Learning (ACML), Journal of Machine Learning Research W & CPs, Vol. 20, 47--62, Nov 2011. (PDF)
How to use a reference dataset and output-kernel learning to robustly locate the relative outliers (the changes) in the target domain ?

Ivor W. Tsang, James T. Kwok, Shutao Li. Learning the kernel in Mahalanobis one-class support vector machines. Proceedings of the International Joint Conference on Neural Networks (IJCNN'06), pp.1169- 1175, Vancouver, Canada, July 2006. (PDF)
Relative Margin is introduced in one-class Support Vector Machine to learn an optimal kernel for novelty detection

James T. Kwok, Ivor W. Tsang. The pre-image problem in kernel methods. IEEE Transactions on Neural Networks, 15(6):1517-1525, Nov 2004. (PDF) (Software)
#This paper was awarded with the IEEE Transactions on Neural Networks Outstanding 2004 Paper Award

Mingkui Tan, Ivor W. Tsang, Li Wang, Xinming Zhang. Convex Matching Pursuit for Large-scale Sparse Coding and Subset Selections. To appear in the 26th AAAI Conference on Artificial Intelligence (AAAI), 2012.

Lin Chen, Ivor W. Tsang, Dong Xu. Laplacian Embedded Regression for Scalable Manifold Regularization. IEEE Transactions on Neural Networks and Learning Systems, 23(6):902 - 915, June 2012. (PDF)
How to solve large-scale Manifold Regularization ?

Nan Li, Ivor W. Tsang, Zhi-Hua Zhou. Efficiently Learning Nonlinear Classifiers for Domain Specific Performance Measures. arXiv CoRR 1012.0930 (link)

Chun-Wei Seah, Ivor W. Tsang, Yew-Soon Ong. Healing Sample Selection Bias by Source Classifier Selection. Proceedings of the IEEE International Conference on Data Mining (IEEE ICDM 2011), pp. 577 - 586, Vancouver, Canada, Dec 2011. (PDF) [Full Paper, Acceptance rate = 13%]
How to handle the change of class distribution in training source data and test target data ? How to alleviate Negative Transfer? How to efficiently handle large-scale transduction ? How to select and combine source classifiers for the target domain ?

Jianbo Yang, Ivor W. Tsang. Hierarchical Maximum Margin Learning for Multi-Class Classification. Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (UAI 2011), p. 753-760, Barcelona, Spain, July 2011. (PDF)
We employ the large margin criterion to separate classes and learn a hierarchical decision tree (representing a discriminating order) via output-kernel learning for fast many-class prediction

Jinfeng Zhuang, Ivor W. Tsang, Steven C. Hoi. A Family of Simple Non-Parametric Kernel Learning Algorithms. Journal of Machine Learning Research(JMLR), 12:1313-1347, 2011.(PDF)
A scalable algorithm is proposed to solve a family of large-scale SDP problems, including Colored Maximum Variance Unfolding, Minimum Volume Embedding, Structure Preserving Embedding and other non-parametric kernel learning problems

Mingkui Tan, Li Wang, Ivor W. Tsang. Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets. Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, Israel, June 2010. (PDF) (Software)
A linear-time SVM-type feature selection algorithm is proposed for large-scale and extremely high dimensional datasets, a very small subset of non-monotonic features can be identified from 3 Million features for suspicious URLs prediction

Jinfeng Zhuang, Ivor W. Tsang, Steven C. Hoi. SimpleNPKL: Simple Non-Parametric Kernel Learning. Proceedings of the 26th International Conference on Machine Learning (ICML 2009), Montreal, Quebec, June 2009. (PDF)
By exploiting the sparse structure, the Large-Scale Semi-Definite Programming (SDP) problem of Non-Parametric Kernel Learning with Pairwise Constraints can be solved by a very simple and scalable convex algorithm. A 5000-by-5000 non-parametric kernel matrix (25M variables) can be learned within a minute

Yu-Feng Li, Ivor W. Tsang, James T. Kwok, Zhi-Hua Zhou. Tighter and Convex Maximum Margin Clustering. Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS) 2009, Journal of Machine Learning Research W & CPs, Vol. 5, pp. 344-351, Clearwater Beach, Florida, USA, April 2009. (PDF) (Software) [Plenary Oral, Acceptance rate = 10%]
The mixed integer program in MMC can be solved by a scalable and efficient multiple output-kernel learning algorithm, which achieves a tighter approximation than the SDP relaxation

Kai Zhang, Ivor W. Tsang, James T. Kwok. Improved Nystrom low rank approximation and error analysis. Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML), pp.1232-1239, Helsinki, Finland. July 2008.(PDF) (Software)
How to find landmark points from a theoretical perspective to improve Nystrom Algorithm ?

Ivor W. Tsang, Andras Kocsor, James T. Kwok. Large-Scale Maximum Margin Discriminant Analysis Using Core Vector Machines. IEEE Transactions on Neural Networks, 19(4): 610-624, April 2008. (PDF)

Ivor W. Tsang, Andras Kocsor, James T. Kwok. Simpler core vector machines with enclosing balls. Proceedings of the Twenty-Fourth International Conference on Machine Learning (ICML), pp.911-918, Corvallis, Oregon, USA, June 2007.(PDF) (Software)
SVM can be solved by the multi-scaled Enclosing Ball problem in Computational Geometry

Ivor W. Tsang, James T. Kwok. Large-scale sparsified manifold regularization. Proceedings of the Advances in Neural Information Processing Systems (NIPS), pp. 1401-1408, Vancouver, Canada, December 2006. (PDF) [Plenary Oral, Acceptance rate = 3%] (A preliminary version appeared in NIPS 2005 Workshop on Large Scale Kernel Machines)
The manifold regularizer is transformed as pairwise constraints, which avoids matrix inversion

Ivor W. Tsang, James T. Kwok, Jacek M. Zurada. Generalized core vector machines. IEEE Transactions on Neural Networks, 17(5): 1126- 1140, Sept 2006. (PDF) (Software)

Ivor W. Tsang, James T. Kwok, Pak-Ming Cheung. Core vector machines: Fast SVM training on very large data sets. Journal of Machine Learning Research, 6:363-392, 2005. (PDF) (Software)

Ivor W. Tsang, James T. Kwok, Kimo T. Lai. Core Vector Regression for Very Large Regression Problems. Proceedings of the Twentieth-Second International Conference on Machine Learning (ICML-2005), pp.913-920, Bonn, Germany, August 2005.(PDF) (Software)

Ivor W. Tsang, James T. Kwok, Pak-Ming Cheung. Very large SVM training using core vector machines. Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics (AISTATS 2005), Barbados, January 2005. (PDF)
SVM can be transformed equivalently as the Minimum Enclosing Ball problem in Computational Geometry. An approximation algorithm using coresets is firstly proposed in solving large-scale SVMs

Lixin Duan, Ivor W. Tsang, Dong Xu. Domain Transfer Multiple Kernel Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(3):465-479, March 2012. (PDF)

Qi Mao, Ivor W. Tsang. Multiple Template Learning for Structured Prediction. arXiv CoRR 1103.0890 (link)

Chun-Wei Seah, Ivor W. Tsang, Yew-Soon Ong. Healing Sample Selection Bias by Source Classifier Selection. Proceedings of the IEEE International Conference on Data Mining (IEEE ICDM 2011), pp. 577 - 586, Vancouver, Canada, Dec 2011. (PDF) [Full Paper, Acceptance rate = 13%]
How to handle the change of class distribution in training source data and test target data ? How to alleviate Negative Transfer? How to efficiently handle large-scale transduction ? How to select and combine source classifiers for the target domain ?

Qi Mao, Ivor W. Tsang. Optimizing Performance Measures for Feature Selection. Proceedings of the IEEE International Conference on Data Mining (IEEE ICDM 2011), Vancouver, Canada, Dec 2011. (PDF) (More results on Multi-Instance Learning via Embedded Feature Selection can be found at arXiv)
A two-layer cutting plane algorithm together with a primal MKL algorithm are proposed for optimizing performance measures for feature selection where the problem has exponential size of both feature groups and label configurations for a given dataset

Lixin Duan, Wen Li, Ivor W. Tsang, Dong Xu. Improving Web Image Search by Bag-based Re-ranking. IEEE Transactions on Image Processing, 20(11):3280--3290, November 2011. (PDF)
The notion of different level of ambiguity in positive bags and negative bags are introduced. How to automatically create noisy label for unlabeled web data ? How to handle a large amount of label noise in web data ?

Shukai Li, Ivor W. Tsang. Learning to Locate Relative Outliers. Proceedings of the Asian Conference on Machine Learning (ACML), Journal of Machine Learning Research W & CPs, Vol. 20, 47--62, Nov 2011. (PDF)
How to use a reference dataset and output-kernel learning to robustly locate the relative outliers (the changes) in the target domain ?

Jianbo Yang, Ivor W. Tsang. Hierarchical Maximum Margin Learning for Multi-Class Classification. Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (UAI 2011), p. 753-760, Barcelona, Spain, July 2011. (PDF)
We employ the large margin criterion to separate classes and learn a hierarchical decision tree (representing a discriminating order) via output-kernel learning for fast many-class prediction

Jinfeng Zhuang, Ivor W. Tsang, Steven C. Hoi. Two-Layer Multiple Kernel Learning. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS 2011), Journal of Machine Learning Research W & CPs, Vol. 15, Ft. Lauderdale, FL, USA, April 2011.(PDF)
Should we learn a deeper kernel ?

Jinfeng Zhuang, Ivor W. Tsang, Steven C. Hoi. A Family of Simple Non-Parametric Kernel Learning Algorithms. Journal of Machine Learning Research(JMLR), 12:1313-1347, 2011.(PDF)
A scalable algorithm is proposed to solve a family of large-scale SDP problems, including Colored Maximum Variance Unfolding, Minimum Volume Embedding, Structure Preserving Embedding and other non-parametric kernel learning problems

Qi Mao, Ivor W. Tsang. Parameter-Free Spectral Kernel Learning. Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010), pp. 350-357, Catalina Island, California, July 2010. (PDF) [Plenary Oral, Acceptance rate = 11.5%]
For a given Laplacian matrix, a parameter-free Spectral Kernel Learning is proposed to learn an ideal kernel with the closed-form solution for transduction

Mingkui Tan, Li Wang, Ivor W. Tsang. Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets. Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, Israel, June 2010. (PDF) (Software)
A linear-time SVM-type feature selection algorithm is proposed for large-scale and extremely high dimensional datasets, a very small subset of non-monotonic features can be identified from 3 Million features for suspicious URLs prediction

Yu-Feng Li, James T. Kwok, Ivor W. Tsang, Zhi-Hua Zhou. A Convex Method for Locating Regions of Interest with Multi-Instance Learning. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2009) , pp.15-30, Bled, Slovenia, September 2009. (PDF) (Software)
Instead of bag prediction only, two convex and scalable Key-Instance SVMs via output-kernel learning are proposed to identify the Key Instance (KI) in positive bags and to predict bags simultaneously. Instance level KI-SVM can accurately locate the region of KI inside a positive bag; while Bag level KI-SVM achieves better bag prediction performance

Jinfeng Zhuang, Ivor W. Tsang, Steven C. Hoi. SimpleNPKL: Simple Non-Parametric Kernel Learning. Proceedings of the 26th International Conference on Machine Learning (ICML 2009), Montreal, Quebec, June 2009. (PDF)
By exploiting the sparse structure, the Large-Scale Semi-Definite Programming (SDP) problem of Non-Parametric Kernel Learning with Pairwise Constraints can be solved by a very simple and scalable convex algorithm. A 5000-by-5000 non-parametric kernel matrix (25M variables) can be learned within a minute

Lixin Duan, Ivor W. Tsang, Dong Xu, Stephen J. Maybank. Domain Transfer SVM for Video Concept Detection. Proceedings of the 22rd IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami Beach, Florida, USA, June 2009.(PDF)
Domain Transfer SVM combines both Multiple Kernel Learning (MKL) and Distribution Matching via MMD for Semi-Supervised Learning/Transduction

Yu-Feng Li, Ivor W. Tsang, James T. Kwok, Zhi-Hua Zhou. Tighter and Convex Maximum Margin Clustering. Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS) 2009, Journal of Machine Learning Research W & CPs, Vol. 5, pp. 344-351, Clearwater Beach, Florida, USA, April 2009. (PDF) (Software) [Plenary Oral, Acceptance rate = 10%]
The mixed integer program in MMC can be solved by a scalable and efficient multiple output-kernel learning algorithm, which achieves a tighter approximation than the SDP relaxation

Ivor W. Tsang, James T. Kwok, Shutao Li. Learning the kernel in Mahalanobis one-class support vector machines. Proceedings of the International Joint Conference on Neural Networks (IJCNN'06), pp.1169- 1175, Vancouver, Canada, July 2006. (PDF)
Relative Margin is introduced in one-class Support Vector Machine to learn an optimal kernel for novelty detection

James T. Kwok, Ivor W. Tsang. Learning with idealized kernels. Proceedings of the International Conference on Machine Learning (ICML 2003), pp.400-407, Washington, D.C., USA, August 2003. (PDF)
Combination of BOTH Similar and Dissimilar Side information is firstly introduced as Distance constraints for learning a kernel in semi-supervised setting

Copyright Notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

IEEE Copyright Notice: © 200x IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from IEEE.

ACM Copyright Notice: Copyright © YYYY by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept., ACM, Inc., fax +1 (212) 869-0481, or permissions@acm.org. See ACM Copyright Policy for details.

Structured Feature Selection for Very High Dimensional Problems. Invited Lecture, Machine Learning Summer School (MLSS 2011), Singapore, June 2011.

Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets. Invited Talk, Baidu, Beijing, China, September 2010.

Learning Sparse SVM for Feature Selection on Very High Dimensional Datasets. Invited Talk, Institute for Pure and Applied Mathematics, University of California, Los Angeles , USA, July 2010.

Non-Parametric Kernel Learning: Algorithms and Applications. Seminar, Department of Mathematical Informatics, The University of Tokyo , Tokyo, Japan, July 2010.

Non-Parametric Kernel Learning: Algorithms and Applications. Invited Talk, Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology , Tokyo, Japan, July 2010.

SimpleNPKL: Simple Non-Parametric Kernel Learning. Special Lecture, Department of Computer Science, University of Alberta, Edmonton, Alberta, Canada, June 2009.

Large-Scale Maximum Margin Clustering. Invited Talk, DSO National laboratories, Singapore, April 2009.

Support Vector Machine Made Simpler. LAMDA Open Seminar, Department of Computer Science & Technology, Nanjing University, China, June 2008.

Machine learning on very large data sets. Machine Learning Lunch Seminar, Machine Learning Department, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, September 2007.

Machine learning on very large data sets. CS Seminar, School of Computing, National University of Singapore, Singapore, March 2007.

Kernel methods meet minimum enclosing balls. Colloquium Presentation, School of Computer Science, Simon Fraser University, Vancouver, Canada, December 2006.

Kernel methods meet minimum enclosing balls. Colloquium Presentation, Department Schoelkopf, Max Planck Institute for Biological Cybernetics, Tuebingen, Germany, September 2006.

Kernel methods meet minimum enclosing balls. Colloquium Presentation, Intelligent Data Analysis Group, Fraunhofer FIRST Institute, Berlin, Germany, September 2006.

Very large SVM training using core vector machines. Invited Talk, School of Mathematics, Statistics and Computer Science, University of New England, Armidale, Australia, January 2005.

Efficient hyperkernel learning using second-order cone programming. Invited Talk, School of Mathematics, Statistics and Computer Science, University of New England, Armidale, Australia, January 2005.

Very large SVM training using core vector machines. Invited Talk, School of Computer Science and Engineering, University of New South Wales, Sydney, Australia, January 2005.

The NIPS 2009 Workshop on Transfer Learning for Structured Data (TLSD-09)

International Workshop on Scalable Machine Learning and Applications (SMLA-10)

The 22nd International Joint Conference on Artificial Intelligence (IJCAI 2011)
The 3rd Asian Conference on Machine Learning (ACML 2011)

The 28th International Conference on Machine Learning (ICML 2011)
The 13th International Conference on Computer Vision (ICCV 2011)
The 25th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012)
The 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2011)
The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2012)
The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2011)
The 12th European Conference on Computer Vision (ECCV 2012)
The 20th ACM Conference on Information and Knowledge Management (CIKM 2011)
The 2nd Asian Conference on Machine Learning (ACML 2010)
The 1st Asian Conference on Machine Learning (ACML 2009)
The 15th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2011)
The 14th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2010)
The 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2009)
The 12th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2008)
The 11th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2007)
The 20th International Conference for Pattern Recognition (ICPR 2010)
IEEE International Conference on Multimedia & Expo (ICME 2010)
The 18th International Conference on Neural Information Processing (ICONIP 2011)

The 25th Annual Conference on Neural Information Processing Systems (NIPS 2011)
The 24th Annual Conference on Neural Information Processing Systems (NIPS 2010)
The 23rd Annual Conference on Neural Information Processing Systems (NIPS 2009)
The 26th International Conference on Machine Learning (ICML 2009)
The 12th IEEE International Conference on Computer Vision (ICCV 2009)
The 20th European Conference on Machine Learning and Principles and the 13th Practice of Knowledge Discovery in Databases (ECML PKDD 2010)

Journal of Machine Learning Research
Machine Learning
Artificial Intelligence
IEEE Transactions on Neural Networks and Learning Systems
IEEE Transactions on Pattern Analysis and Machine Intelligence
IEEE Transactions on Image Processing
IEEE Transactions on Knowledge and Data Engineering
IEEE Transactions on Systems, Man and Cybernetics (Part B)
Neural Networks
Pattern Recognition
Information Sciences

CPE 406/CSC 304 Artificial Intelligence and Intelligent Systems

CPE 406/CSC 304 Artificial Intelligence and Intelligent Systems
CSC 202 Algorithms
CE 1008/CZ 1008/CSC 103/CPE 103 Engineering Mathematics

C/C++ Programming
Machine Learning Softwares
Machine Learning Journals
Machine Learning Benchmark Datasets
Useful Courses and Links in Machine Learning
Related Conferences in Machine Learning