Modeling, Recognizing, and Explaining Apparent Personality from Videos

Hugo Jair Escalante; Heysem Kaya; Albert Ali Salah; Sergio Escalera; Yağmur  Güç;lütürk; Umut  Güçlü; Xavier Baro; Achmadnoer  Sukma Wicaksana; Cynthia C.S. Liem; null More Authors

doi:10.1109/TAFFC.2020.2973984

Modeling, Recognizing, and Explaining Apparent Personality from Videos

Hugo Jair Escalante, Heysem Kaya, Albert Ali Salah, Sergio Escalera, Yağmur Güç;lütürk, Umut Güçlü, Xavier Baro, Achmadnoer Sukma Wicaksana, Cynthia C.S. Liem, More Authors

Multimedia Computing

Research output: Contribution to journal › Article › Scientific › peer-review

22 Citations (Scopus)

112 Downloads (Pure)

Abstract

Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.

Original language	English
Pages (from-to)	894-911
Number of pages	18
Journal	IEEE Transactions on Affective Computing
Volume	13
Issue number	2
DOIs	https://doi.org/10.1109/TAFFC.2020.2973984
Publication status	Published - 2020

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Explainable computer vision
First impressions
Personality analysis
Multimodal information
Algorithmic accountability

Access to Document

10.1109/TAFFC.2020.2973984

Modeling_Recognizing_and_Explaining_Apparent_Personality_from_VideosFinal published version, 5.39 MB

Cite this

@article{a6d297a8efb340fea4f529e405800cb0,

title = "Modeling, Recognizing, and Explaining Apparent Personality from Videos",

abstract = "Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.",

keywords = "Explainable computer vision, First impressions, Personality analysis, Multimodal information, Algorithmic accountability",

author = "Escalante, {Hugo Jair} and Heysem Kaya and {Ali Salah}, Albert and Sergio Escalera and Yağmur G{\"u}{\c c};l{\"u}t{\"u}rk and Umut G{\"u}{\c c}l{\"u} and Xavier Baro and {Sukma Wicaksana}, Achmadnoer and Liem, {Cynthia C.S.} and {More Authors}",

note = "Green Open Access added to TU Delft Institutional Repository {\textquoteleft}You share, we take care!{\textquoteright} – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. ",

year = "2020",

doi = "10.1109/TAFFC.2020.2973984",

language = "English",

volume = "13",

pages = "894--911",

journal = "IEEE Transactions on Affective Computing",

issn = "1949-3045",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

number = "2",

}

TY - JOUR

T1 - Modeling, Recognizing, and Explaining Apparent Personality from Videos

AU - Escalante, Hugo Jair

AU - Kaya, Heysem

AU - Ali Salah, Albert

AU - Escalera, Sergio

AU - Güç;lütürk, Yağmur

AU - Güçlü, Umut

AU - Baro, Xavier

AU - Sukma Wicaksana, Achmadnoer

AU - Liem, Cynthia C.S.

AU - More Authors, null

N1 - Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2020

Y1 - 2020

N2 - Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.

AB - Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.

KW - Explainable computer vision

KW - First impressions

KW - Personality analysis

KW - Multimodal information

KW - Algorithmic accountability

UR - http://www.scopus.com/inward/record.url?scp=85085992577&partnerID=8YFLogxK

U2 - 10.1109/TAFFC.2020.2973984

DO - 10.1109/TAFFC.2020.2973984

M3 - Article

SN - 1949-3045

VL - 13

SP - 894

EP - 911

JO - IEEE Transactions on Affective Computing

JF - IEEE Transactions on Affective Computing

IS - 2

ER -

Modeling, Recognizing, and Explaining Apparent Personality from Videos

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this