Interactive exploration of journalistic video footage through multimodal semantic matching

Sarah Ibrahimi; Shuo Chen; Devanshu Arya; Arthur Câmara; Yunlu Chen; Tanja Crijns; Maurits Van Der Goes; Thomas Mensink; Emiel Van Miltenburg; null More Authors

doi:10.1145/3343031.3350597

Interactive exploration of journalistic video footage through multimodal semantic matching

Sarah Ibrahimi, Shuo Chen, Devanshu Arya, Arthur Câmara, Yunlu Chen, Tanja Crijns, Maurits Van Der Goes, Thomas Mensink, Emiel Van Miltenburg, More Authors

Web Information Systems

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

143 Downloads (Pure)

Abstract

This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.

Original language	English
Title of host publication	MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia
Publisher	Association for Computing Machinery (ACM)
Pages	2196-2198
Number of pages	3
ISBN (Electronic)	9781450368896
DOIs	https://doi.org/10.1145/3343031.3350597
Publication status	Published - 15 Oct 2019
Event	27th ACM International Conference on Multimedia, MM 2019 - Nice, France Duration: 21 Oct 2019 → 25 Oct 2019

Conference

Conference	27th ACM International Conference on Multimedia, MM 2019
Country/Territory	France
City	Nice
Period	21/10/19 → 25/10/19

Keywords

Exploration
Matching
Multimodal
Semantics
Video

Access to Document

10.1145/3343031.3350597

p2196-ibrahimiFinal published version, 4.33 MB

Cite this

Ibrahimi, S., Chen, S., Arya, D., Câmara, A., Chen, Y., Crijns, T., Van Der Goes, M., Mensink, T., Van Miltenburg, E., & More Authors (2019). Interactive exploration of journalistic video footage through multimodal semantic matching. In MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia (pp. 2196-2198). Association for Computing Machinery (ACM). https://doi.org/10.1145/3343031.3350597

@inproceedings{63fe23c211cb47659ea54e921daff42e,

title = "Interactive exploration of journalistic video footage through multimodal semantic matching",

abstract = "This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.",

keywords = "Exploration, Matching, Multimodal, Semantics, Video",

author = "Sarah Ibrahimi and Shuo Chen and Devanshu Arya and Arthur C{\^a}mara and Yunlu Chen and Tanja Crijns and {Van Der Goes}, Maurits and Thomas Mensink and {Van Miltenburg}, Emiel and {More Authors}",

year = "2019",

month = oct,

day = "15",

doi = "10.1145/3343031.3350597",

language = "English",

pages = "2196--2198",

booktitle = "MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

note = "27th ACM International Conference on Multimedia, MM 2019 ; Conference date: 21-10-2019 Through 25-10-2019",

}

Ibrahimi, S, Chen, S, Arya, D, Câmara, A, Chen, Y, Crijns, T, Van Der Goes, M, Mensink, T, Van Miltenburg, E & More Authors 2019, Interactive exploration of journalistic video footage through multimodal semantic matching. in MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia. Association for Computing Machinery (ACM), pp. 2196-2198, 27th ACM International Conference on Multimedia, MM 2019, Nice, France, 21/10/19. https://doi.org/10.1145/3343031.3350597

Interactive exploration of journalistic video footage through multimodal semantic matching. / Ibrahimi, Sarah; Chen, Shuo; Arya, Devanshu et al.
MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia. Association for Computing Machinery (ACM), 2019. p. 2196-2198.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Interactive exploration of journalistic video footage through multimodal semantic matching

AU - Ibrahimi, Sarah

AU - Chen, Shuo

AU - Arya, Devanshu

AU - Câmara, Arthur

AU - Chen, Yunlu

AU - Crijns, Tanja

AU - Van Der Goes, Maurits

AU - Mensink, Thomas

AU - Van Miltenburg, Emiel

AU - More Authors, null

PY - 2019/10/15

Y1 - 2019/10/15

N2 - This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.

AB - This demo presents a system for journalists to explore video footage for broadcasts. Daily news broadcasts contain multiple news items that consist of many video shots and searching for relevant footage is a labor intensive task. Without the need for annotated video shots, our system extracts semantics from footage and automatically matches these semantics to query terms from the journalist. The journalist can then indicate which aspects of the query term need to be emphasized, e.g. the title or its thematic meaning. The goal of this system is to support the journalists in their search process by encouraging interaction and exploration with the system.

KW - Exploration

KW - Matching

KW - Multimodal

KW - Semantics

KW - Video

UR - http://www.scopus.com/inward/record.url?scp=85074826713&partnerID=8YFLogxK

U2 - 10.1145/3343031.3350597

DO - 10.1145/3343031.3350597

M3 - Conference contribution

SP - 2196

EP - 2198

BT - MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia

PB - Association for Computing Machinery (ACM)

T2 - 27th ACM International Conference on Multimedia, MM 2019

Y2 - 21 October 2019 through 25 October 2019

ER -

Interactive exploration of journalistic video footage through multimodal semantic matching

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this