DOI

  • Seungchul Lee
  • Chulhong Min
  • Alessandro Montanari
  • Akhil Mathur
  • Youngjae Chang
  • Junehwa Song
  • Fahim Kawsar

In this paper, we introduce inertial signals obtained from an earable placed in the ear canal as a new compelling sensing modality for recognising two key facial expressions: Smile and frown. Borrowing principles from Facial Action Coding Systems, we first demonstrate that an inertial measurement unit of an earable can capture facial muscle deformation activated by a set of temporal microexpressions. Building on these observations, we then present three different learning schemes - shallow models with statistical features, hidden Markov model, and deep neural networks to automatically recognise smile and frown expressions from inertial signals. The experimental results show that in controlled non-conversational settings, we can identify smile and frown with high accuracy (F1 score: 0.85).

Original languageEnglish
Title of host publicationAH2019
Subtitle of host publicationProceedings of the 10th Augmented Human International Conference 2019
Place of PublicationNew York, NY
PublisherAssociation for Computing Machinery (ACM)
Pages1-4
Number of pages4
ISBN (Print)978-1-4503-6547-5
DOIs
Publication statusPublished - 2019
Event10th Augmented Human International Conference, AH 2019 - Reims, France
Duration: 11 Mar 201912 Mar 2019

Conference

Conference10th Augmented Human International Conference, AH 2019
CountryFrance
CityReims
Period11/03/1912/03/19

    Research areas

  • Earable, Facs, Kinetic modeling, Smile and frown recognition

ID: 52664572