Standard

Performance analysis of a video quality ruler methodology for subjective quality assessment. / Freitas, Pedro Garcia; Silva, Alexandre Fieno; Redi, Judith A.; Farias, Mylène C.Q.

In: Journal of Electronic Imaging, Vol. 27, No. 5, 053020, 2018, p. 1-10.

Research output: Contribution to journalArticleScientificpeer-review

Harvard

Freitas, PG, Silva, AF, Redi, JA & Farias, MCQ 2018, 'Performance analysis of a video quality ruler methodology for subjective quality assessment', Journal of Electronic Imaging, vol. 27, no. 5, 053020, pp. 1-10. https://doi.org/10.1117/1.JEI.27.5.053020

APA

Freitas, P. G., Silva, A. F., Redi, J. A., & Farias, M. C. Q. (2018). Performance analysis of a video quality ruler methodology for subjective quality assessment. Journal of Electronic Imaging, 27(5), 1-10. [053020]. https://doi.org/10.1117/1.JEI.27.5.053020

Vancouver

Freitas PG, Silva AF, Redi JA, Farias MCQ. Performance analysis of a video quality ruler methodology for subjective quality assessment. Journal of Electronic Imaging. 2018;27(5):1-10. 053020. https://doi.org/10.1117/1.JEI.27.5.053020

Author

Freitas, Pedro Garcia ; Silva, Alexandre Fieno ; Redi, Judith A. ; Farias, Mylène C.Q. / Performance analysis of a video quality ruler methodology for subjective quality assessment. In: Journal of Electronic Imaging. 2018 ; Vol. 27, No. 5. pp. 1-10.

BibTeX

@article{606c193ac1734b0f871103f3ec2c9c2f,
title = "Performance analysis of a video quality ruler methodology for subjective quality assessment",
abstract = "Objective video quality metrics are designed to be as reliable as the subjective quality assessments on which they are calibrated and validated. However, existing standard methodologies for subjective video quality assessment provide low reliable results for some conditions. We investigate whether an extension of the quality ruler experimental methodology, originally defined for images and shown to be more reliable than, e.g., standard single stimulus (SS) methods, can be adapted to reliably assess the quality of videos. The video quality ruler methodology allows subjects to assess video quality using a set of reference anchor images (the ruler), spanning a wide range of quality altogether, but closely spaced in function of quality one from the other. Subjects are asked to compare the quality of the displayed test video with the quality of these anchor images, displayed on a tablet, and indicate which of the reference images matches in quality the test video. As a result, the video quality assessment task is reduced to a set of visual comparisons between video and reference image quality. We describe how to adapt the original quality ruler methodology to video quality assessment, and we compare the proposed methodology with two other, widely used experimental methodologies: the single stimulus (SS) and the double stimulus (DS) method. Our results show that video quality ruler is a reliable method to assess video quality according to a multitude of criteria.",
keywords = "quality ruler, subjective experiment, subjective quality assessment, testing conditions and methods",
author = "Freitas, {Pedro Garcia} and Silva, {Alexandre Fieno} and Redi, {Judith A.} and Farias, {Myl{\`e}ne C.Q.}",
year = "2018",
doi = "10.1117/1.JEI.27.5.053020",
language = "English",
volume = "27",
pages = "1--10",
journal = "Journal of Electronic Imaging",
issn = "1017-9909",
publisher = "SPIE",
number = "5",

}

RIS

TY - JOUR

T1 - Performance analysis of a video quality ruler methodology for subjective quality assessment

AU - Freitas, Pedro Garcia

AU - Silva, Alexandre Fieno

AU - Redi, Judith A.

AU - Farias, Mylène C.Q.

PY - 2018

Y1 - 2018

N2 - Objective video quality metrics are designed to be as reliable as the subjective quality assessments on which they are calibrated and validated. However, existing standard methodologies for subjective video quality assessment provide low reliable results for some conditions. We investigate whether an extension of the quality ruler experimental methodology, originally defined for images and shown to be more reliable than, e.g., standard single stimulus (SS) methods, can be adapted to reliably assess the quality of videos. The video quality ruler methodology allows subjects to assess video quality using a set of reference anchor images (the ruler), spanning a wide range of quality altogether, but closely spaced in function of quality one from the other. Subjects are asked to compare the quality of the displayed test video with the quality of these anchor images, displayed on a tablet, and indicate which of the reference images matches in quality the test video. As a result, the video quality assessment task is reduced to a set of visual comparisons between video and reference image quality. We describe how to adapt the original quality ruler methodology to video quality assessment, and we compare the proposed methodology with two other, widely used experimental methodologies: the single stimulus (SS) and the double stimulus (DS) method. Our results show that video quality ruler is a reliable method to assess video quality according to a multitude of criteria.

AB - Objective video quality metrics are designed to be as reliable as the subjective quality assessments on which they are calibrated and validated. However, existing standard methodologies for subjective video quality assessment provide low reliable results for some conditions. We investigate whether an extension of the quality ruler experimental methodology, originally defined for images and shown to be more reliable than, e.g., standard single stimulus (SS) methods, can be adapted to reliably assess the quality of videos. The video quality ruler methodology allows subjects to assess video quality using a set of reference anchor images (the ruler), spanning a wide range of quality altogether, but closely spaced in function of quality one from the other. Subjects are asked to compare the quality of the displayed test video with the quality of these anchor images, displayed on a tablet, and indicate which of the reference images matches in quality the test video. As a result, the video quality assessment task is reduced to a set of visual comparisons between video and reference image quality. We describe how to adapt the original quality ruler methodology to video quality assessment, and we compare the proposed methodology with two other, widely used experimental methodologies: the single stimulus (SS) and the double stimulus (DS) method. Our results show that video quality ruler is a reliable method to assess video quality according to a multitude of criteria.

KW - quality ruler

KW - subjective experiment

KW - subjective quality assessment

KW - testing conditions and methods

UR - http://www.scopus.com/inward/record.url?scp=85054299865&partnerID=8YFLogxK

U2 - 10.1117/1.JEI.27.5.053020

DO - 10.1117/1.JEI.27.5.053020

M3 - Article

AN - SCOPUS:85054299865

VL - 27

SP - 1

EP - 10

JO - Journal of Electronic Imaging

JF - Journal of Electronic Imaging

SN - 1017-9909

IS - 5

M1 - 053020

ER -

ID: 47815601