Dissertation

Automating Multimodal Data Storytelling for Embodied Team Learning

Fernández Nieto, Gloria Milena

2022

Overview

There is a growing interest in creating Learning Analytics (LA) interfaces that support students and teachers directly. Thus far, many of these solutions have been materialised as dashboards and visualisations. However, although a growing number of prototypes and commercial products aimed at supporting students/teachers exist, their limitations are coming under scrutiny. For instance, many visual LA tools are failing to provide meaningful and relevant insights that can support students reflections on their embodied teamwork activity. Moreover, there are additional challenges in visualising and communicating the wide variety of multimodal sensor data captured from physical spaces, in a way that supports educational stakeholders (e.g., teachers or students), who as casual users, have limited training in data analysis and interpretation. Thus, this thesis engages research in Information Visualisation (InfoVis) and specifically the Guidance visualisation paradigm that aims to support casual users, or those users with low analysis expertise, to narrow the gap of data visualisation interpretations. Data Storytelling is one way to provide guidance, as a compression technique to help an audience effectively understand what is important in a visualisation, communicating key messages combining data, visualisations, and narratives. 'Telling stories' with data in these ways should enable the elicitation of deeper reflections in an effective manner. This thesis tackles the above challenges specifically for professional sectors, whose educational and training scenarios can be challenging because they need to develop theory, procedural knowledge and also learn from bodily experiences. This research progresses in by investigating: \"How can salient aspects of embodied team activity be automatically identified, and derived insights be communicated to support timely, productive reflection?\" Four research questions were derived: (1) What modelling techniques can enable identification of salient aspects of multimodal embodied team activity according to the learning design (i.e., teachers' pedagogical intentions)? (2) How can insights be extracted from multimodal sensors and communicated to students and teachers to support teaching and reflection on embodied team activity? (3) To what extent can students and teachers reflect on embodied team activity using MMLA interfaces? and (4) To what extent can MMLA interfaces for students and teachers be automatically generated? This research makes three types of contribution: modelling, prototypes (MMLA interfaces), and implementation. Results from this research point to the potential of creating alternative ways to communicate multimodal data insights to teachers and students, by combining visualisation, narrative and storytelling, driven by teachers' pedagogical intentions and the learning design.

• Publisher: ProQuest Dissertations & Theses
• Subject: Artificial intelligence / Classrooms / Educational technology / Instructional Design / Learning analytics / Pedagogy / Teachers / Teamwork
• ISBN: 9798379784607