Paper

SimTube: Generating Simulated Video Comments through Multimodal AI and User Personas

Yun-Chien, Huang, Shao-Hua, Sun, Yu-Kai, Hung, Ting-Yu, Su, Lung-Pan, Cheng, Yen-Ting, Lin, Wang, Bryan

2024

Overview

Audience feedback is crucial for refining video content, yet it typically comes after publication, limiting creators' ability to make timely adjustments. To bridge this gap, we introduce SimTube, a generative AI system designed to simulate audience feedback in the form of video comments before a video's release. SimTube features a computational pipeline that integrates multimodal data from the video-such as visuals, audio, and metadata-with user personas derived from a broad and diverse corpus of audience demographics, generating varied and contextually relevant feedback. Furthermore, the system's UI allows creators to explore and customize the simulated comments. Through a comprehensive evaluation-comprising quantitative analysis, crowd-sourced assessments, and qualitative user studies-we show that SimTube's generated comments are not only relevant, believable, and diverse but often more detailed and informative than actual audience comments, highlighting its potential to help creators refine their content before release.

• Publisher: Cornell University Library, arXiv.org
• Subject: Audio data / Feedback / Generative artificial intelligence / Qualitative analysis