Standard

Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods. / Li, Meng; Albayrak, Armagan; Zhang, Yu; van Eijk, Daan; Yang, Zengyao.

Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018) : Ergonomics in Design, Design for All, Activity Theories for Work Analysis and Design, Affective Design. ed. / Sebastiano Bagnara; Riccardo Tartaglia; Sara Albolino; Thomas Alexander; Yushi Fujita. Vol. VII Cham : Springer Verlag, 2019. p. 1081-1098 (Advances in Intelligent Systems and Computing; Vol. 824).

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Harvard

Li, M, Albayrak, A, Zhang, Y, van Eijk, D & Yang, Z 2019, Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods. in S Bagnara, R Tartaglia, S Albolino, T Alexander & Y Fujita (eds), Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018) : Ergonomics in Design, Design for All, Activity Theories for Work Analysis and Design, Affective Design. vol. VII, Advances in Intelligent Systems and Computing, vol. 824, Springer Verlag, Cham, pp. 1081-1098, IEA 2018: 20th Congress of the International Ergonomics Association, Florence, Italy, 26/08/18. https://doi.org/10.1007/978-3-319-96071-5_110

APA

Li, M., Albayrak, A., Zhang, Y., van Eijk, D., & Yang, Z. (2019). Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods. In S. Bagnara, R. Tartaglia, S. Albolino, T. Alexander, & Y. Fujita (Eds.), Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018) : Ergonomics in Design, Design for All, Activity Theories for Work Analysis and Design, Affective Design (Vol. VII, pp. 1081-1098). (Advances in Intelligent Systems and Computing; Vol. 824). Cham: Springer Verlag. https://doi.org/10.1007/978-3-319-96071-5_110

Vancouver

Li M, Albayrak A, Zhang Y, van Eijk D, Yang Z. Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods. In Bagnara S, Tartaglia R, Albolino S, Alexander T, Fujita Y, editors, Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018) : Ergonomics in Design, Design for All, Activity Theories for Work Analysis and Design, Affective Design. Vol. VII. Cham: Springer Verlag. 2019. p. 1081-1098. (Advances in Intelligent Systems and Computing). https://doi.org/10.1007/978-3-319-96071-5_110

Author

Li, Meng ; Albayrak, Armagan ; Zhang, Yu ; van Eijk, Daan ; Yang, Zengyao. / Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods. Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018) : Ergonomics in Design, Design for All, Activity Theories for Work Analysis and Design, Affective Design. editor / Sebastiano Bagnara ; Riccardo Tartaglia ; Sara Albolino ; Thomas Alexander ; Yushi Fujita. Vol. VII Cham : Springer Verlag, 2019. pp. 1081-1098 (Advances in Intelligent Systems and Computing).

BibTeX

@inproceedings{01057dec59bb4a8f9adb8e594674afca,
title = "Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods",
abstract = "The usability now serves as a fundamental quality of a computational device, e.g. smartphone. Moreover, the smartphone has firmly embedded into our daily life as an indispensable part, so the context and style that user may interact with them are largely different from a decade ago. Nowadays, testing usability with end user has become a common sense. Thus, how valid a usability evaluation method could assess the ‘extent to which a product can be used by specified users’ (ISO 9241-11) to facilitate software design becomes an interesting question to explore. In this research, three usability evaluation methods are compared. Among these methods, IsoMetrics is a standard questionnaire aiming at offer usability data for summative and formative evaluation; SUMI aims to assess quality of software product from end users perspective; User Model Checklist is a method based on user’s cognition-motor chain in specific tasks. The coverage and amount of usability issues, user’s effort of evaluation and software developer’s feedback on evaluation result are compared under a simulated usability test on SMS function with a smartphone. The result indicate that User Model Checklist could cover 90.4{\%} of the usability issues found by IsoMetrics and SUMI, while 26.3{\%} usability issues found by User Model Checklist could not be covered by IsoMetrics and SUMI. Users put highest effort on accomplish IsoMetrics and lowest effort on User Model Checklist. Moreover, the feedbacks from the developers show that the User Model Checklist requires lower usability knowledge, offers clearer improvement points and supports detailed design better.",
keywords = "IsoMetrics, SUMI, Usability evaluation comparison, User Model Checklist",
author = "Meng Li and Armagan Albayrak and Yu Zhang and {van Eijk}, Daan and Zengyao Yang",
note = "Accepted author manuscript",
year = "2019",
doi = "10.1007/978-3-319-96071-5_110",
language = "English",
isbn = "978-3-319-96070-8",
volume = "VII",
series = "Advances in Intelligent Systems and Computing",
publisher = "Springer Verlag",
pages = "1081--1098",
editor = "Sebastiano Bagnara and Riccardo Tartaglia and Sara Albolino and Thomas Alexander and Yushi Fujita",
booktitle = "Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018)",

}

RIS

TY - GEN

T1 - Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods

AU - Li, Meng

AU - Albayrak, Armagan

AU - Zhang, Yu

AU - van Eijk, Daan

AU - Yang, Zengyao

N1 - Accepted author manuscript

PY - 2019

Y1 - 2019

N2 - The usability now serves as a fundamental quality of a computational device, e.g. smartphone. Moreover, the smartphone has firmly embedded into our daily life as an indispensable part, so the context and style that user may interact with them are largely different from a decade ago. Nowadays, testing usability with end user has become a common sense. Thus, how valid a usability evaluation method could assess the ‘extent to which a product can be used by specified users’ (ISO 9241-11) to facilitate software design becomes an interesting question to explore. In this research, three usability evaluation methods are compared. Among these methods, IsoMetrics is a standard questionnaire aiming at offer usability data for summative and formative evaluation; SUMI aims to assess quality of software product from end users perspective; User Model Checklist is a method based on user’s cognition-motor chain in specific tasks. The coverage and amount of usability issues, user’s effort of evaluation and software developer’s feedback on evaluation result are compared under a simulated usability test on SMS function with a smartphone. The result indicate that User Model Checklist could cover 90.4% of the usability issues found by IsoMetrics and SUMI, while 26.3% usability issues found by User Model Checklist could not be covered by IsoMetrics and SUMI. Users put highest effort on accomplish IsoMetrics and lowest effort on User Model Checklist. Moreover, the feedbacks from the developers show that the User Model Checklist requires lower usability knowledge, offers clearer improvement points and supports detailed design better.

AB - The usability now serves as a fundamental quality of a computational device, e.g. smartphone. Moreover, the smartphone has firmly embedded into our daily life as an indispensable part, so the context and style that user may interact with them are largely different from a decade ago. Nowadays, testing usability with end user has become a common sense. Thus, how valid a usability evaluation method could assess the ‘extent to which a product can be used by specified users’ (ISO 9241-11) to facilitate software design becomes an interesting question to explore. In this research, three usability evaluation methods are compared. Among these methods, IsoMetrics is a standard questionnaire aiming at offer usability data for summative and formative evaluation; SUMI aims to assess quality of software product from end users perspective; User Model Checklist is a method based on user’s cognition-motor chain in specific tasks. The coverage and amount of usability issues, user’s effort of evaluation and software developer’s feedback on evaluation result are compared under a simulated usability test on SMS function with a smartphone. The result indicate that User Model Checklist could cover 90.4% of the usability issues found by IsoMetrics and SUMI, while 26.3% usability issues found by User Model Checklist could not be covered by IsoMetrics and SUMI. Users put highest effort on accomplish IsoMetrics and lowest effort on User Model Checklist. Moreover, the feedbacks from the developers show that the User Model Checklist requires lower usability knowledge, offers clearer improvement points and supports detailed design better.

KW - IsoMetrics

KW - SUMI

KW - Usability evaluation comparison

KW - User Model Checklist

UR - http://www.scopus.com/inward/record.url?scp=85052313627&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-96071-5_110

DO - 10.1007/978-3-319-96071-5_110

M3 - Conference contribution

SN - 978-3-319-96070-8

VL - VII

T3 - Advances in Intelligent Systems and Computing

SP - 1081

EP - 1098

BT - Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018)

A2 - Bagnara, Sebastiano

A2 - Tartaglia, Riccardo

A2 - Albolino, Sara

A2 - Alexander, Thomas

A2 - Fujita, Yushi

PB - Springer Verlag

CY - Cham

ER -

ID: 47159350