In this paper, we carry out an extensive study of social involvement in free standing conversing groups (the so-called F-formations) from static images. By introducing a novel feature representation, we show that the standard features which have been used to represent full membership in an F-formation cannot be applied to the detection of associates of F-formations due their sparser occurence. We enrich state-of-the-art F-formation modelling by learning a frustum of attention that accounts for the spatial context. That is, F-formation configurations vary with respect to the arrangement of furniture and the non-uniform crowdednessin the space during mingling scenarios. Moroever, the majority of prior works have considered the labelling of conversing groups as an objective task, requiring only a single annotator. However, we show that by embracing the subjectivity of social involvement, we not only generate a richer model of the social interactions in a scene but can use the detected associates to improve initial estimates of the full members of an F-formation. We carry out extensive experimental validation of our proposed approach by collecting a novel set of multi-annotator labels of involvement on two publicly available datasets; The Idiap Poster Data and SALSA data set.

Original languageEnglish
Pages (from-to)1-13
Number of pages13
JournalIEEE Transactions on Affective Computing
Publication statusE-pub ahead of print - 2020

    Research areas

  • F-formations Detection, Feature extraction, Human Behaviour Analysis, Labeling, Psychology, Semantics, Social Group Detection, Surveillance, Task analysis, Visualization

ID: 46669009