{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T13:50:10Z","timestamp":1760709010582,"version":"3.41.0"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2016,2,11]],"date-time":"2016-02-11T00:00:00Z","timestamp":1455148800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Simon Dooms of the Agency for Innovation by Science and Technology (IWT Vlaanderen) and the Spanish Ministry of Science and Innovation","award":["TIN2013-47090-C3-2"],"award-info":[{"award-number":["TIN2013-47090-C3-2"]}]},{"name":"ERCIM \u201cAlain Bensoussan\u201d Fellowship Programme"},{"DOI":"10.13039\/100012220","name":"the Hercules Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100012220","id-type":"DOI","asserted-by":"crossref"}]},{"name":"European Comission FP7","award":["246016"],"award-info":[{"award-number":["246016"]}]},{"name":"Flemish Government - department EWI"},{"name":"Stevin Supercomputer Infrastructure at Ghent University"},{"name":"Ghent University"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2016,4]]},"abstract":"<jats:p>Rating datasets are of paramount importance in recommender systems research. They serve as input for recommendation algorithms, as simulation data, or for evaluation purposes. In the past, public accessible rating datasets were not abundantly available, leaving researchers no choice but to work with old and static datasets like MovieLens and Netflix. More recently, however, emerging trends as social media and smartphones are found to provide rich data sources which can be turned into valuable research datasets. While dataset availability is growing, a structured way for introducing and comparing new datasets is currently still lacking. In this work, we propose a five-step framework to introduce and benchmark new datasets in the recommender systems domain. We illustrate our framework on a new movie rating dataset\u2014called MovieTweetings\u2014collected from Twitter. Following our framework, we detail the origin of the dataset, provide basic descriptive statistics, investigate external validity, report the results of a number of reproducible benchmarks, and conclude by discussing some interesting advantages and appropriate research use cases.<\/jats:p>","DOI":"10.1145\/2751565","type":"journal-article","created":{"date-parts":[[2016,2,22]],"date-time":"2016-02-22T13:07:16Z","timestamp":1456146436000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["A Framework for Dataset Benchmarking and Its Application to a New Movie Rating Dataset"],"prefix":"10.1145","volume":"7","author":[{"given":"Simon","family":"Dooms","sequence":"first","affiliation":[{"name":"Ghent University, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alejandro","family":"Bellog\u00edn","sequence":"additional","affiliation":[{"name":"Centrum Wiskunde &amp; Informatica, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Toon De","family":"Pessemier","sequence":"additional","affiliation":[{"name":"iMinds-Ghent University, Ghent, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luc","family":"Martens","sequence":"additional","affiliation":[{"name":"iMinds-Ghent University, Ghent, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2016,2,11]]},"reference":[{"volume-title":"Ribeiro-Neto","year":"2011","author":"Baeza-Yates Ricardo A.","key":"e_1_2_1_1_1"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2043996"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2507157.2508006"},{"volume-title":"Int. Conf. on Weblogs and Social Media (ICWSM)","year":"2013","author":"Bellog\u00edn Alejandro","key":"e_1_2_1_5_1"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1345448.1345459"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2010.03.009"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-012-9136-x"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2043990"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1979742.1979896"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1864708.1864721"},{"volume-title":"Intelligent Systems Design and Applications, 2007. ISDA 2007. Seventh International Conference on. IEEE","author":"de Castro Pablo A. D.","key":"e_1_2_1_12_1"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1864708.1864767"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2567948.2579232"},{"volume-title":"User Modeling, Adaption and Personalization","author":"Gena Cristina","key":"e_1_2_1_15_1"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/1577069.1755883"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/312624.312682"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-02958-0_2"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2645710.2645742"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-011-9118-4"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401944"},{"volume-title":"The bellkor solution to the netflix grand prize. Netflix Prize Documentation 81","year":"2009","author":"Koren Yehuda","key":"e_1_2_1_22_1"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2009.263"},{"key":"e_1_2_1_24_1","doi-asserted-by":"crossref","unstructured":"Pasquale Lops Marco de Gemmis and Giovanni Semeraro. 2011. Content-based recommender systems: State of the art and trends. In Recommender Systems Handbook. 73--105.  Pasquale Lops Marco de Gemmis and Giovanni Semeraro. 2011. Content-based recommender systems: State of the art and trends. In Recommender Systems Handbook. 73--105.","DOI":"10.1007\/978-0-387-85820-3_3"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1639714.1639717"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1125451.1125659"},{"volume-title":"Carley","year":"2013","author":"Morstatter Fred","key":"e_1_2_1_27_1"},{"volume-title":"The pragmatic theory solution to the netflix grand prize. Netflix Prize Documentation","year":"2009","author":"Piotte Martin","key":"e_1_2_1_29_1"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2645710.2645746"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441933"},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the RecSys\u201909 Workshop on Recommender Systems & The Social Web. CEUR-WS","volume":"532","author":"Said Alan","year":"2009"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Guy Shani and Asela Gunawardana. 2011. Evaluating recommendation systems. In Recommender Systems Handbook. 257--297.  Guy Shani and Asela Gunawardana. 2011. Evaluating recommendation systems. In Recommender Systems Handbook. 257--297.","DOI":"10.1007\/978-0-387-85820-3_8"},{"volume-title":"Bell","year":"2009","author":"T\u00f6scher Andreas","key":"e_1_2_1_34_1"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/951949.952133"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2751565","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2751565","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T05:43:07Z","timestamp":1750225387000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2751565"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,2,11]]},"references-count":33,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,4]]}},"alternative-id":["10.1145\/2751565"],"URL":"https:\/\/doi.org\/10.1145\/2751565","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"type":"print","value":"2157-6904"},{"type":"electronic","value":"2157-6912"}],"subject":[],"published":{"date-parts":[[2016,2,11]]},"assertion":[{"value":"2014-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-02-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}