Best Paper Runner Up in the 15th ACM International Conference on Web Search and Data Mining (WSDM)

The paper "The Datasets Dilemma: How Much Do We Really Know about Recommendation Datasets", by Chin Jin Yao, Chen Yile, and Cong Gao has won the Best Paper Award (Runner-Up) in the 15th ACM International Conference on Web Search and Data Mining (WSDM), 2022 (Acceptance rate of 15.8%, 80 out of 505 submissions). The conference was held virtually from February 21 to 25, 2022.  

WSDM (pronounced "wisdom") is one of the premier conferences on web-inspired research involving search and data mining. WSDM publishes original, high-quality papers related to search and data mining on the Web and the Social Web, with an emphasis on practical yet principled novel models of search and data mining, algorithm design and analysis, economic implications, and in-depth experimental analysis of accuracy and performance.

The awarded paper, titled “The Datasets Dilemma: How Much Do We Really Know about Recommendation Datasets”, takes an in-depth look at a fundamental but often neglected aspect of the evaluation procedure, i.e. the datasets themselves. The paper critically examines (1) how different datasets have been utilised thus far, (2) the characteristics of these datasets to understand their similarities and differences, and (3) whether the choice of datasets used for evaluation can influence the observations and/or conclusions obtained. The findings in the paper suggest that greater attention needs to be paid to the datasets used for evaluating recommender systems in order to improve the robustness of the obtained results.

The main author, Dr Chin Jin Yao, received his PhD degree from SCSE in January 2022 under the supervision of Prof Cong Gao, and the co-author, Chen Yile, is currently a Year 4 PhD candidate of SCSE supervised by Prof Cong Gao.