From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)

Nicola Ferro; Norbert Fuhr; Gregory Grefenstette; Tsvi Kuflik; Krister Lindén; Bernardo Magnini; Jian-Yun Nie; Raffaele Perego; Nava Tintarev; null More Authors

doi:10.4230/DagMan.7.1.96

From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)

Nicola Ferro, Norbert Fuhr, Gregory Grefenstette, Tsvi Kuflik, Krister Lindén, Bernardo Magnini, Jian-Yun Nie, Raffaele Perego, Nava Tintarev, More Authors

Web Information Systems

Research output: Contribution to journal › Article › Scientific

186 Downloads (Pure)

Abstract

We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of prediction models describing the relationship between assumptions, features and resulting performance.

Original language	English
Pages (from-to)	96-139
Number of pages	44
Journal	Dagstuhl Manifestos
Volume	7
Issue number	1
DOIs	https://doi.org/10.4230/DagMan.7.1.96
Publication status	Published - 2019
Event	Perspectives Workshop - Schloss Dagatuhl, Wadern, Germany Duration: 30 Oct 2019 → 3 Nov 2019

Access to Document

10.4230/DagMan.7.1.96

dagman-v007-i001-p096-17442Final published version, 2.2 MBLicence: CC BY

http://drops.dagstuhl.de/opus/volltexte/2018/9898

Cite this

Ferro, N., Fuhr, N., Grefenstette, G., Kuflik, T., Lindén, K., Magnini, B., Nie, J.-Y., Perego, R., Tintarev, N., & More Authors (2019). From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442). Dagstuhl Manifestos, 7(1), 96-139. https://doi.org/10.4230/DagMan.7.1.96

@article{f07a881f056f46aa868630597f174442,

title = "From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)",

abstract = "We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of prediction models describing the relationship between assumptions, features and resulting performance. ",

author = "Nicola Ferro and Norbert Fuhr and Gregory Grefenstette and Tsvi Kuflik and Krister Lind{\'e}n and Bernardo Magnini and Jian-Yun Nie and Raffaele Perego and Nava Tintarev and {More Authors}",

year = "2019",

doi = "10.4230/DagMan.7.1.96",

language = "English",

volume = "7",

pages = "96--139",

journal = "Dagstuhl Manifestos",

issn = "2193-2433",

publisher = "Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany",

number = "1",

note = "Perspectives Workshop ; Conference date: 30-10-2019 Through 03-11-2019",

}

Ferro, N, Fuhr, N, Grefenstette, G, Kuflik, T, Lindén, K, Magnini, B, Nie, J-Y, Perego, R, Tintarev, N & More Authors 2019, 'From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)', Dagstuhl Manifestos, vol. 7, no. 1, pp. 96-139. https://doi.org/10.4230/DagMan.7.1.96

From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442). / Ferro, Nicola; Fuhr, Norbert; Grefenstette, Gregory et al.
In: Dagstuhl Manifestos, Vol. 7, No. 1, 2019, p. 96-139.

Research output: Contribution to journal › Article › Scientific

TY - JOUR

T1 - From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)

AU - Ferro, Nicola

AU - Fuhr, Norbert

AU - Grefenstette, Gregory

AU - Kuflik, Tsvi

AU - Lindén, Krister

AU - Magnini, Bernardo

AU - Nie, Jian-Yun

AU - Perego, Raffaele

AU - Tintarev, Nava

AU - More Authors, null

PY - 2019

Y1 - 2019

N2 - We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of prediction models describing the relationship between assumptions, features and resulting performance.

AB - We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of prediction models describing the relationship between assumptions, features and resulting performance.

U2 - 10.4230/DagMan.7.1.96

DO - 10.4230/DagMan.7.1.96

M3 - Article

SN - 2193-2433

VL - 7

SP - 96

EP - 139

JO - Dagstuhl Manifestos

JF - Dagstuhl Manifestos

IS - 1

T2 - Perspectives Workshop

Y2 - 30 October 2019 through 3 November 2019

ER -

From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)

Abstract

Access to Document

Fingerprint

Cite this