Standard

A Hierarchical Approach for Associating Body-Worn Sensors to Video Regions in Crowded Mingling Scenarios. / Cabrera Quiros, Laura; Hung, Hayley.

In: IEEE Transactions on Multimedia, Vol. 21, No. 7, 2019, p. 1867-1879.

Research output: Contribution to journalArticleScientificpeer-review

Harvard

APA

Vancouver

Author

BibTeX

@article{2e1236e4453542dc8878d11f6cb88330,
title = "A Hierarchical Approach for Associating Body-Worn Sensors to Video Regions in Crowded Mingling Scenarios",
abstract = "We address the complex problem of associating several wearable devices with the spatio-temporal region of their wearers in video during crowded mingling events using only acceleration and proximity. This is a particularly important first step for multi-sensor behavior analysis using video and wearable technologies, where the privacy of the participants must be maintained. Most state-of-the-art works using these two modalities perform their association manually, which becomes practically unfeasible as the number of people in the scene increases. We proposed an automatic association method based on a hierarchical linear assignment optimization, which exploits the spatial context of the scene. Moreover, we present extensive experiments on matching from 2 to more than 69 acceleration and video streams, showing significant improvements over a random baseline in a real world crowded mingling scenario. We also show the effectiveness of our method for incomplete or missing streams (up to a certain limit) and analyze the trade-off between length of the streams and number of participants. Finally, we provide an analysis of failure cases, showing that deep understanding of the social actions within the context of the event is necessary to further improve performance on this intriguing task.",
keywords = "acceleration, association, computer vision, Mingling, wearable sensor",
author = "{Cabrera Quiros}, Laura and Hayley Hung",
note = "Accepted author manuscript",
year = "2019",
doi = "10.1109/TMM.2018.2888798",
language = "English",
volume = "21",
pages = "1867--1879",
journal = "IEEE Transactions on Multimedia",
issn = "1520-9210",
publisher = "IEEE",
number = "7",

}

RIS

TY - JOUR

T1 - A Hierarchical Approach for Associating Body-Worn Sensors to Video Regions in Crowded Mingling Scenarios

AU - Cabrera Quiros, Laura

AU - Hung, Hayley

N1 - Accepted author manuscript

PY - 2019

Y1 - 2019

N2 - We address the complex problem of associating several wearable devices with the spatio-temporal region of their wearers in video during crowded mingling events using only acceleration and proximity. This is a particularly important first step for multi-sensor behavior analysis using video and wearable technologies, where the privacy of the participants must be maintained. Most state-of-the-art works using these two modalities perform their association manually, which becomes practically unfeasible as the number of people in the scene increases. We proposed an automatic association method based on a hierarchical linear assignment optimization, which exploits the spatial context of the scene. Moreover, we present extensive experiments on matching from 2 to more than 69 acceleration and video streams, showing significant improvements over a random baseline in a real world crowded mingling scenario. We also show the effectiveness of our method for incomplete or missing streams (up to a certain limit) and analyze the trade-off between length of the streams and number of participants. Finally, we provide an analysis of failure cases, showing that deep understanding of the social actions within the context of the event is necessary to further improve performance on this intriguing task.

AB - We address the complex problem of associating several wearable devices with the spatio-temporal region of their wearers in video during crowded mingling events using only acceleration and proximity. This is a particularly important first step for multi-sensor behavior analysis using video and wearable technologies, where the privacy of the participants must be maintained. Most state-of-the-art works using these two modalities perform their association manually, which becomes practically unfeasible as the number of people in the scene increases. We proposed an automatic association method based on a hierarchical linear assignment optimization, which exploits the spatial context of the scene. Moreover, we present extensive experiments on matching from 2 to more than 69 acceleration and video streams, showing significant improvements over a random baseline in a real world crowded mingling scenario. We also show the effectiveness of our method for incomplete or missing streams (up to a certain limit) and analyze the trade-off between length of the streams and number of participants. Finally, we provide an analysis of failure cases, showing that deep understanding of the social actions within the context of the event is necessary to further improve performance on this intriguing task.

KW - acceleration

KW - association

KW - computer vision

KW - Mingling

KW - wearable sensor

UR - http://www.scopus.com/inward/record.url?scp=85058887309&partnerID=8YFLogxK

U2 - 10.1109/TMM.2018.2888798

DO - 10.1109/TMM.2018.2888798

M3 - Article

VL - 21

SP - 1867

EP - 1879

JO - IEEE Transactions on Multimedia

T2 - IEEE Transactions on Multimedia

JF - IEEE Transactions on Multimedia

SN - 1520-9210

IS - 7

ER -

ID: 49468884