Hanwang Zhang

Hanwang Zhang

Ideas occur to us when they please, not when it pleases us.

---- by Max Weber

Assistant Professor

School of Computer Science and Engineering (SCSE)
Nanyang Technological University
Block N4 #02b-67, Nanyang Avenue, Singapore 639798

Email: hanwangzhang AT ntu.edu.sg (best way to contact me)


About me

I joined SCSE, Nanyang Technological University (NTU) as Assistant Professor in Jan 2018. I was doing a post-doc job at DVMM, whose boss is Prof. Shih-Fu Chang. Before this job, I worked with Prof. Tat-Seng Chua, who is also my Ph.D supervisor (2009 - 2014) and Assoc. Prof. Shuicheng Yan (2013 - 2014). And, before this, I was an undergraduate student of Zhejiang University (2005 - 2009). In particular, I was a member of Chu Kochen Honors College.

I was honored to be the valedictorian of NUS Commencement 2014 (Youtube Link, valediction script, and Best Ph.D Thesis)


Research

Multimedia Knowledge Graph
Towards connecting images, videos, and common sense via knowledge graph, which can be used for visual reasoning, scene understanding, and natural language description generation.

Multimedia Modeling for Large-scale Social Networks
Developing efficient algorithms for analyzing heterogeneous data on social networks such as users, images, videos, and their social relations.

Representation Learning for Multimedia
Deep learning algorithms for learning image/video feature representations, which can be real-valued vectors for subsequent statistical modeling or binary codes for fast similarity search.


Selected Publications

  • All
  • 2018
  • 2017
  • 2016
  • 2015
  • earlier
  • Low-shot Learning via Covariance-Preserving Adversarial Augmentation Network

    Hang Gao, Zheng Shou, Alireza Zareian, Hanwang Zhang, Shih-Fu Chang

    Thirty-second Conference on Neural Information Processing Systems. NIPS 2018

    [pdf and codes coming soon]

  • More is Better: Precise and Detailed Image Captioning using Online Positive Recall and Missing Concepts Mining

    Mingxing Zhang, Yang Yang, Hanwang Zhang, Yanli Ji, Heng-Tao Shen, Tat-Seng Chua

    IEEE Transactions on Image Processing. TIP 2018

  • Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features

    Xu Yang, Hanwang Zhang, Jianfei Cai

    15th European Conference on Computer Vision. ECCV 2018. Munich, Germany. Sep 2018

    [arxiv preprint]  [merged in vtranse]

  • Context-Aware Visual Policy Network for Sequence-Level Image Captioning  [oral]

    Daqing Liu, Zheng-Jun Zha, Hanwang Zhang, Yongdong Zhang, Feng Wu

    ACM International Conference on Multimedia. MM 2018. Seoul, Korea, October 2018

    [arxiv preprint]  [codes]

  • Discrete Factorization Machines for Fast Feature-based Recommendation

    Han Liu, Xiangnan He, Fuli Feng, Liqiang Nie, Rui Liu, Hanwang Zhang

    The 27th International Joint Conference on Artificial Intelligence. IJCAI 2018. Stockholm, Sweden, July, 2018

  • Multi-Level Policy and Reward Reinforcement Learning for Image Captioning

    An-An Liu, Ning Xu, Hanwang Zhang, Weizhi Nie, Yuting Su, Yongdong Zhang

    The 27th International Joint Conference on Artificial Intelligence. IJCAI 2018. Stockholm, Sweden, July, 2018

  • Self-Supervised Video Hashing with Hierarchical Binary Auto-encoder

    Jingkuan Song, Hanwang Zhang, Xiangpeng Li, Lianli Gao, Meng Wang, Richang Hong

    IEEE Transactions on Image Processing. TIP 2018

  • Attributed Social Network Embedding

    Lizi Liao, Xiangnan He, Hanwang Zhang, Tat-Seng Chua

    IEEE Transactions on Knowledge and Data Engineering. TKDE 2018

  • Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Network

    Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, Shih-Fu Chang

    IEEE International Conference on Computer Vision and Pattern Recognition. CVPR 2018. Salt Late City, USA. June 2018

    [arxiv preprint]  [codes]

  • Grounding Referring Expressions in Images by Variational Context

    Hanwang Zhang, Yulei Niu, Shih-Fu Chang

    IEEE International Conference on Computer Vision and Pattern Recognition. CVPR 2018. Salt Late City, USA. June 2018

    [arxiv preprint]  [codes]

  • Learning to Guide Decoding for Image Captioning

    Wenhao Jiang, Lin Ma, Xinpeng Chen, Hanwang Zhang, Wei Liu

    The Thirty-Second AAAI Conference on Artificial Intelligence. AAAI 2018. New Orleans, USA, Feb 2018

  • PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCN

    Hanwang Zhang, Zawlin Kyaw, Jinyang Yu, and Shih-Fu Chang

    International Conference on Computer Vision. ICCV 2017. Venice, Italy, October 2017

    [arxiv preprint]

  • Improving Event Extraction via Cross-Modal Integration

    Tongtao Zhang, Spencer Whitehead, Hanwang Zhang, Hongzhi Li, Joseph Ellis, Lifu Huang, Wei Liu, Heng Ji and Shih-Fu Chang

    ACM International Conference on Multimedia. MM 2017. Mountain View, CA USA, October 2017

  • Enhancing Micro-video Understanding by Harnessing External Sounds  [oral]

    Liqiang Nie, Xiang Wang, Jianglong Zhang, Xiangnan He, Hanwang Zhang, Richang Hong and Qi Tian

    ACM International Conference on Multimedia. MM 2017. Mountain View, CA USA, October 2017

  • Video Visual Relation Detection

    Xindi Shang, Tongwei Ren, Jingfan Guo, Hanwang Zhang and Tat-Seng Chua

    ACM International Conference on Multimedia. MM 2017. Mountain View, CA USA, October 2017

  • Video Question Answering via Gradually Refined Attention over Appearance and Motion

    Dejing Xu, Zhou Zhao, Jun Xiao, Fei Wu, Hanwang Zhang, Xiangnan He and Yueting Zhuang

    ACM International Conference on Multimedia. MM 2017. Mountain View, CA USA, October 2017

  • Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks

    Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu and Tat-Seng Chua

    The 26th International Joint Conference on Artificial Intelligence. IJCAI 2017. Melbourne, Australia, August 2017

  • Videos Captioning with Attention-based LSTM and Semantic Consistency

    Lianli Gao, Zhao Guo, Hanwang Zhang, Xing Xu, and Heng-Tao Shen

    IEEE Transactions on Multimedia. TMM 2017

  • VideoWhisper: Towards Discriminative Unsupervised Video Feature Learning with Attention Based Recurrent Neural Networks

    Na Zhao, Hanwang Zhang, Richang Hong, Meng Wang, Tat-Seng Chua

    IEEE Transactions on Multimedia. TMM 2017

  • Attentive Collaborative Filtering: Multimedia Recommendation with Feature- and Item-level Attention  [oral]

    Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, Chua Tat-Seng

    International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR 2017. Tokyo, Japan. August 2017

    [pdf]  [codes]

  • Visual Translation Embedding Network for Visual Relation Detection

    Hanwang Zhang, Zawlin Kyaw, Shih-Fu Chang, Tat-Seng Chua

    IEEE International Conference on Computer Vision and Pattern Recognition. CVPR 2017. Hawaii, USA. July 2017

    [arxiv preprint]  [codes & data]

  • SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning

    Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, Tat-Seng Chua

    IEEE International Conference on Computer Vision and Pattern Recognition. CVPR 2017. Hawaii, USA. July 2017

    [arxiv preprint]  [codes]

  • Matryoshka Peek: Towards Learning Fine-grained, Robust, Discriminative Features for Product Search

    Zawlin Kyaw, Shuhan Qi, Ke Gao, Hanwang Zhang, Luming Zhang, Jun Xiao, Xuan Wang, Tat-Seng Chua

    IEEE Transactions on Multimedia. TMM 2017

  • Neural Collaborative Filtering  [oral]

    Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, Tat-Seng Chua

    26th International World Wide Web Conference. WWW 2017. Perth, Australia, April 2017

    [pdf]  [codes]

  • I Know What You Want to Express: Sentence Element Inference by Incorporating External Knowledge Base

    Xiaochi Wei, Heyan Huang, Liqiang Nie, Hanwang Zhang, Xian-Ling Mao, Chua, Tat-Seng

    IEEE Transactions on Knowledge and Data Engineering. TKDE 2016

  • Micro Tells Macro: Predicting the Popularity of Micro-Videos via a Transductive Model  [oral]

    Jingyuan Chen, Xuemeng Song, Liqiang Nie, Xiang Wang, Hanwang Zhang, Tat-Seng Chua

    ACM International Conference on Multimedia. MM 2016. Amsterdam, The Netherlands, October 2016

  • Play and Rewind: Optimizing Binary Representations of Videos by Self-Supervised Temporal Hashing  [oral]

    Hanwang Zhang, Meng Wang, Richang Hong, Tat-Seng Chua

    ACM International Conference on Multimedia. MM 2016. Amsterdam, The Netherlands, October 2016

    [pdf]  [codes]

  • Learning from Collective Intelligence: Feature Learning Using Social Images and Tags

    Hanwang Zhang, Xindi Shang, Huanbo Luan, Meng Wang, Tat-Seng Chua.

    ACM Transactions on Multimedia Computing, Communications and Applications. TOMM (formerly known as TOMCCAP) 2016

    [pdf]

  • Event Classification in Microblog via Social Tracking

    Yue Gao, Hanwang Zhang, Xibin Zhao, Shuicheng Yan

    ACM Transactions on Intelligent Systems and Technology. TIST. 2016

  • Discrete Collaborative Filtering  [oral]  Best Paper Honorable Mention

    Hanwang Zhang, Fumin Shen, Wei Liu, Xiangnan He, Huanbo Luan, Chua Tat-Seng

    International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR 2016. Pisa, Italy. July 2016

    [pdf]  [codes]

  • Fast Matrix Factorization for Online Recommendation with Implicit Feedback  [oral]

    Xiangnan He, Hanwang Zhang, Min-Yen Kan, Tat-Seng Chua

    International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR 2016. Pisa, Italy. July 2016

  • Online Collaborative Learning for Open-Vocabulary Visual Classifiers

    Hanwang Zhang, Xindi Shang, Wenzhuo Yang, Huan Xu, Huanbo Luan, Tat-Seng Chua.

    IEEE International Conference on Computer Vision and Pattern Recognition. CVPR 2016. Las Vegas, USA. Jun 2016

    [pdf]  [codes]

  • Discrete Image Hashing Using Large Weakly Annotated Photo Collections

    Hanwang Zhang, Na Zhao, Xindi Shang, Huanbo Luan, Tat-Seng Chua.

    The Thirtieth AAAI Conference on Artificial Intelligence. AAAI 2016. Phoenix, Arizona, USA. Feb 2016.

  • Learning Image and User Features for Recommendation in Social Networks

    Xue Geng, Hanwang Zhang, Jingwen Bian, Tat-Seng Chua.

    IEEE International Conference on Computer Vision. ICCV 2015. Santiago, Chile. Nov 2015.

  • Learning Features from Large-Scale, Noisy and Social Image-Tag Collection

    Hanwang Zhang, Xindi Shang, Huanbo Luan, Yang Yang, Tat-Seng Chua.

    ACM International Conference on Multimedia. MM 2015. Brisbane, Australia. Oct 2015.

  • Visual Coding in a Semantic Hierarchy  [oral]

    Yang Yang, Hanwang Zhang, Mingxing Zhang, Fumin Shen, Xuelong Li.

    ACM International Conference on Multimedia. MM 2015. Brisbane, Australia. Oct 2015.

  • Deep Fusion of Multiple Semantic Cues for Complex Event Recognition

    Xishan Zhang, Hanwang Zhang, Yongdong Zhang, Yang Yang, Meng Wang, Huanbo Luan, Jintao Li, Tat-Seng Chua.

    IEEE Transactions on Image Processing. TIP 2015

  • Deep Aging Face Verification with Large Gaps

    Luoqi Liu, Xiong Chao, Hanwang Zhang, Zhiheng Niu, Meng Wang, Shuicheng Yan

    IEEE Transactions on Multimedia. TMM 2015

  • Multimedia Summarization for Social Events in Microblog Stream

    Jingwen Bian, Yang Yang, Hanwang Zhang, Yue Gao, Tat-Seng Chua

    IEEE Transactions on Multimedia. TMM 2015

  • Enhancing Video Event Recognition using Automatically Constructed Semantic-Visual Knowledge Base

    Xishan Zhang, Yang Yang, Yongdong Zhang, Huanbo Luan, Jintao Li, Hanwang Zhang, Tat-Seng Chua.

    IEEE Transactions on Multimedia. TMM 2015

  • Start from Scratch: Towards Automatically Identifying, Modeling, and Naming Visual Attributes  [oral]

    Hanwang Zhang, Yang Yang, Huanbo Luan, Shuicheng Yan, Tat-Seng Chua.

    ACM International Conference on Multimedia. MM 2014. Orlando, USA. Nov 2014.

    [pdf]  [slides]  [demo (ID:deep, PWD: deep123456)]  [models]  [codes]

  • Perception-Guided Multimodal Feature Fusion for Photo Aesthetics Assessment  [oral]

    Luming Zhang, Yue Gao, Chao Zhang, Hanwang Zhang, Qi Tian, Roger Zimmermann.

    ACM International Conference on Multimedia. MM 2014. Orlando, USA. Nov 2014.

  • One of a Kind: User Profiling by Social Curation  [oral]

    Xue Geng, Hanwang Zhang, Zheng Song, Yang Yang, Huanbo Luan, Tat-Seng Chua.

    ACM International Conference on Multimedia. MM 2014. Orlando, USA. Nov 2014.

    [pdf][slides]

  • Image Tagging with Social Assistance  [oral]

    Yang Yang, Yue Gao, Hanwang Zhang, Jie Shao and Tat-Seng Chua.

    ACM International Conference on Multimedia Retrieval. ICMR 2014. Glasgow, Scotland, Apr 2014.

  • Robust (Semi-) Nonnegative Graph Embedding

    Hanwang Zhang, Zheng-Jun Zha, Yang Yang, Shuicheng Yan, Tat-Seng Chua.

    IEEE Transactions on Image Processing. TIP 2014

    [pdf][codes]

  • Attribute-augmented Semantic Hierarchy: Towards a Unified Framework for Content-based Image Retrieval

    Hanwang Zhang, Zheng-Jun Zha, Yang Yang, Shuicheng Yan, Yue Gao, Tat-Seng Chua.

    ACM Transactions on Multimedia Computing, Communications and Applications. TOMM (formerly known as TOMCCAP) 2014

    [pdf]

  • Attribute-augmented Semantic Hierarchy  [oral]

    Hanwang Zhang, Zheng-Jun Zha, Yang Yang, Shuicheng Yan, Yue Gao, Tat-Seng Chua.

    ACM International Conference on Multimedia. MM 2013. Barcelona, Spain. Oct 2014.

    [pdf]  [slides]  Best Student Paper

  • Attribute Feedback  [oral]

    Hanwang Zhang, Zheng-Jun Zha, Shuicheng Yan, Jingwen Bian, Tat-Seng Chua.

    ACM International Conference on Multimedia. MM 2012. Nara, Japan. Oct 2012.

    [pdf]  [slides]  [demo]  Best Demo Runner-up

  • Robust Non-negative Graph Embedding: Towards Noisy Data, Unreliable Graphs, and Noisy Labels

    Hanwang Zhang, Zheng-Jun Zha, Shuicheng Yan, Meng Wang, Tat-Seng Chua.

    IEEE International Conference on Computer Vision and Pattern Recognition. CVPR 2012. Rhode Island, USA. June 2012.

    [pdf]  [codes]