Overview of newsreel’16: Multi-dimensional evaluation of real-time stream-recommendation algorithms

Benjamin Kille; Andreas Lommatzsch; Gebrekirstos G Gebremeskel; Frank Hopfgartner; Martha Larson; Jonas Seiler; Davide Malagoli; András Serény; Torben Brodt; Arjen P De Vries

doi:10.1007/978-3-319-44564-9_27

Overview of newsreel’16: Multi-dimensional evaluation of real-time stream-recommendation algorithms

Benjamin Kille, Andreas Lommatzsch, Gebrekirstos G Gebremeskel, Frank Hopfgartner, Martha Larson, Jonas Seiler, Davide Malagoli, András Serény, Torben Brodt, Arjen P De Vries

Multimedia Computing

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

16 Citations (Scopus)

Abstract

Successful news recommendation requires facing the challenges of dynamic item sets, contextual item relevance, and of fulfilling non-functional requirements, such as response time. The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to tackle news recommendation and to optimize and evaluate their recommender algorithms both online and offline. In this paper, we summarize the objectives and challenges of NewsREEL 2016. We cover two contrasting perspectives on the challenge: that of the operator (the business providing recommendations) and that of the challenge participant (the researchers developing recommender algorithms). In the intersection of these perspectives, new insights can be gained on how to effectively evaluate real-time stream recommendation algorithms.

Original language	English
Title of host publication	Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016
Subtitle of host publication	International Conference of the Cross-Language Evaluation Forum for European Languages
Place of Publication	Cham
Publisher	Springer
Pages	311-331
Number of pages	21
ISBN (Electronic)	978-3-319-44564-9
ISBN (Print)	978-3-319-44563-2
DOIs	https://doi.org/10.1007/978-3-319-44564-9_27
Publication status	Published - 23 Aug 2016
Event	CLEF 2016 - Conference and Labs of the Evaluation Forum: Information Access Evaluation meets Multilinguality, and Interaction - University of Évora, Évora, Portugal Duration: 5 Sept 2016 → 8 Sept 2016 Conference number: 7th http://clef2016.clef-initiative.eu/

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	9822
ISSN (Electronic)	0302-9743

Conference

Conference	CLEF 2016 - Conference and Labs of the Evaluation Forum
Country/Territory	Portugal
City	Évora
Period	5/09/16 → 8/09/16
Internet address	http://clef2016.clef-initiative.eu/

Keywords

Recommender Systems
News
Multi-dimensional Evaluation
Living Lab
Stream-based Recommender

Access to Document

10.1007/978-3-319-44564-9_27

Cite this

Kille, B., Lommatzsch, A., Gebremeskel, G. G., Hopfgartner, F., Larson, M., Seiler, J., Malagoli, D., Serény, A., Brodt, T., & De Vries, A. P. (2016). Overview of newsreel’16: Multi-dimensional evaluation of real-time stream-recommendation algorithms. In Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016: International Conference of the Cross-Language Evaluation Forum for European Languages (pp. 311-331). (Lecture Notes in Computer Science; Vol. 9822). Springer. https://doi.org/10.1007/978-3-319-44564-9_27

Kille, Benjamin ; Lommatzsch, Andreas ; Gebremeskel, Gebrekirstos G et al. / Overview of newsreel’16 : Multi-dimensional evaluation of real-time stream-recommendation algorithms. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016: International Conference of the Cross-Language Evaluation Forum for European Languages. Cham : Springer, 2016. pp. 311-331 (Lecture Notes in Computer Science).

@inproceedings{b963a59d604d4807be0deca104a9e424,

title = "Overview of newsreel{\textquoteright}16: Multi-dimensional evaluation of real-time stream-recommendation algorithms",

abstract = "Successful news recommendation requires facing the challenges of dynamic item sets, contextual item relevance, and of fulfilling non-functional requirements, such as response time. The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to tackle news recommendation and to optimize and evaluate their recommender algorithms both online and offline. In this paper, we summarize the objectives and challenges of NewsREEL 2016. We cover two contrasting perspectives on the challenge: that of the operator (the business providing recommendations) and that of the challenge participant (the researchers developing recommender algorithms). In the intersection of these perspectives, new insights can be gained on how to effectively evaluate real-time stream recommendation algorithms.",

keywords = "Recommender Systems, News, Multi-dimensional Evaluation , Living Lab , Stream-based Recommender ",

author = "Benjamin Kille and Andreas Lommatzsch and Gebremeskel, {Gebrekirstos G} and Frank Hopfgartner and Martha Larson and Jonas Seiler and Davide Malagoli and Andr{\'a}s Ser{\'e}ny and Torben Brodt and {De Vries}, {Arjen P}",

year = "2016",

month = aug,

day = "23",

doi = "10.1007/978-3-319-44564-9_27",

language = "English",

isbn = "978-3-319-44563-2",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "311--331",

booktitle = "Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016",

note = "CLEF 2016 - Conference and Labs of the Evaluation Forum : Information Access Evaluation meets Multilinguality, and Interaction ; Conference date: 05-09-2016 Through 08-09-2016",

url = "http://clef2016.clef-initiative.eu/",

}

Kille, B, Lommatzsch, A, Gebremeskel, GG, Hopfgartner, F, Larson, M, Seiler, J, Malagoli, D, Serény, A, Brodt, T & De Vries, AP 2016, Overview of newsreel’16: Multi-dimensional evaluation of real-time stream-recommendation algorithms. in Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016: International Conference of the Cross-Language Evaluation Forum for European Languages. Lecture Notes in Computer Science, vol. 9822, Springer, Cham, pp. 311-331, CLEF 2016 - Conference and Labs of the Evaluation Forum, Évora, Portugal, 5/09/16. https://doi.org/10.1007/978-3-319-44564-9_27

Overview of newsreel’16: Multi-dimensional evaluation of real-time stream-recommendation algorithms. / Kille, Benjamin; Lommatzsch, Andreas; Gebremeskel, Gebrekirstos G et al.
Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016: International Conference of the Cross-Language Evaluation Forum for European Languages. Cham: Springer, 2016. p. 311-331 (Lecture Notes in Computer Science; Vol. 9822).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Overview of newsreel’16

T2 - CLEF 2016 - Conference and Labs of the Evaluation Forum

AU - Kille, Benjamin

AU - Lommatzsch, Andreas

AU - Gebremeskel, Gebrekirstos G

AU - Hopfgartner, Frank

AU - Larson, Martha

AU - Seiler, Jonas

AU - Malagoli, Davide

AU - Serény, András

AU - Brodt, Torben

AU - De Vries, Arjen P

N1 - Conference code: 7th

PY - 2016/8/23

Y1 - 2016/8/23

N2 - Successful news recommendation requires facing the challenges of dynamic item sets, contextual item relevance, and of fulfilling non-functional requirements, such as response time. The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to tackle news recommendation and to optimize and evaluate their recommender algorithms both online and offline. In this paper, we summarize the objectives and challenges of NewsREEL 2016. We cover two contrasting perspectives on the challenge: that of the operator (the business providing recommendations) and that of the challenge participant (the researchers developing recommender algorithms). In the intersection of these perspectives, new insights can be gained on how to effectively evaluate real-time stream recommendation algorithms.

AB - Successful news recommendation requires facing the challenges of dynamic item sets, contextual item relevance, and of fulfilling non-functional requirements, such as response time. The CLEF NewsREEL challenge is a campaign-style evaluation lab allowing participants to tackle news recommendation and to optimize and evaluate their recommender algorithms both online and offline. In this paper, we summarize the objectives and challenges of NewsREEL 2016. We cover two contrasting perspectives on the challenge: that of the operator (the business providing recommendations) and that of the challenge participant (the researchers developing recommender algorithms). In the intersection of these perspectives, new insights can be gained on how to effectively evaluate real-time stream recommendation algorithms.

KW - Recommender Systems

KW - News

KW - Multi-dimensional Evaluation

KW - Living Lab

KW - Stream-based Recommender

U2 - 10.1007/978-3-319-44564-9_27

DO - 10.1007/978-3-319-44564-9_27

M3 - Conference contribution

SN - 978-3-319-44563-2

T3 - Lecture Notes in Computer Science

SP - 311

EP - 331

BT - Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016

PB - Springer

CY - Cham

Y2 - 5 September 2016 through 8 September 2016

ER -

Kille B, Lommatzsch A, Gebremeskel GG, Hopfgartner F, Larson M, Seiler J et al. Overview of newsreel’16: Multi-dimensional evaluation of real-time stream-recommendation algorithms. In Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016: International Conference of the Cross-Language Evaluation Forum for European Languages. Cham: Springer. 2016. p. 311-331. (Lecture Notes in Computer Science). doi: 10.1007/978-3-319-44564-9_27

Overview of newsreel’16: Multi-dimensional evaluation of real-time stream-recommendation algorithms

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this