Dissertation

Optimizing Data Collection Processes for Operational Military Education: A Performance Improvement Approach

Leitheiser, Jessica Spring

2025

Overview

This study explores and addresses a performance issue at an operational military education institution. Currently, the school faces significant challenges employing data-informed approaches to curriculum development to validate instructional outcomes. Data-driven decision making is a command priority, yet limited data hinders the school’s ability to clearly articulate program outcomes to senior leadership. Without actionable insights, the school risks missed opportunities for evidence-based curriculum improvements and to demonstrate its impact on mission readiness. Addressing this issue is essential for ensuring informed decision-making, enhancing instructional quality, and maintaining the school's reputation as a leader in agile, high impact education for the force. The study leverages action research as a strategic approach to identify challenges, test interventions, and refine practices through iterative cycles of data collection and reflection. This problem-oriented approach is particularly relevant in a dynamic military education setting where complex problems require context-specific solutions. The Human Performance Technology (HPT) model served as the conceptual framework for analyzing the root causes of limited data, designing interventions, and implementing data-driven, actionable, and measurable solutions. Both action research and the HPT model emphasize stakeholder participation to ensure that those most affected by the issue contribute to understanding the problem and shaping solutions.Integrating qualitative analysis with the Human Performance Technology (HPT) model provided an in-depth, contextual understanding of the problem. Qualitative methods are particularly effective for exploring underlying meanings and patterns in relationships (Babbie, 2021). This method enabled the collection and interpretation of narrative and visual data, uncovering organizational, cultural, and environmental factors that influence human performance (Bloomberg, 2022). Rather than seeking to generalize findings, this study aimed to generate rich, descriptive insights into the school’s processes, identifying areas for improvement specific to its context. The school is an agile military institution that adapts its educational products to meet contemporary operational challenges. Unlike more traditional professional military education institutions, which follow standardized curricula, the school maintains flexible learning processes. This adaptability necessitates a tailored approach to performance improvement, reinforcing the need for qualitative research to capture nuanced perspectives on existing processes, inefficiencies, and opportunities for enhancement (Bloomberg, 2022).Data was triangulated from multiple perspectives to identify organizational strengths and weaknesses. Additionally, comprehensive questionnaires captured diverse viewpoints during the gap and cause analysis phases of the HPT model. This approach facilitated an in-depth examination of processes and workflows, shedding light on how work is actually performed and where improvements can be made. Performance analysis findings revealed critical gaps in courseware management, assessment, and instructional design. The absence of structured, codified processes for tracking course modifications impairs the school’s ability to make data driven decisions. Without defined roles, responsibilities, workflows, and centralized storage, any new assessment tools will likely be ineffective in capturing accurate data or supporting informed decision-making. Further, without reliable data, instructional improvements rely on assumptions rather than empirical evidence, limiting the school’s capacity for continuous improvement. The root causes of the problem point to four underlying issues that contribute to this problem. The first challenge is cultural resistance to standardized instructional systems design practice. Secondly, the unclear division of responsibilities between Instructional Systems Designers (ISD) and cadre leads to inefficiencies in courseware management and evaluation. The third challenge, disorganized storage and inconsistent data collection, hinders instructional continuity and evaluation. Finally, a lack of performance metrics and associated governance structures leads to inconsistent adherence to instructional design standards. Intervention selection followed a structured, collaborative process involving key stakeholders. A targeted solution set was developed, a Courseware Management Program (CMP) comprised of performance support and learning interventions. The CMP establishes a centralized courseware management system (CMS), supported by structured documentation, formalized workflows, and role definitions. It provides a standardized and accessible repository for instructional content, ensuring consistent practices across departments and enhancing transparency, efficiency, and curriculum continuity. The accompanying Data Management & Storage Standards ensure reliable data collection and use, improving the accuracy of instructional evaluations and enabling evidence-based decision-making. The CMP aims to standardize courseware development, enhance data governance, and embed systematic instructional design practices based on the ADDIE model.Addressing all these challenges simultaneously would be difficult for a military school with limited resources that concurrently supports broader missions. Implementation of the Courseware Management Program (CMP) followed a phased approach designed to prioritize foundational changes and generate early successes to build momentum. Initial formative evaluation efforts informed meaningful refinements to the CMP and yielded measurable indicators of its success. The phased implementation continues, with summative and confirmative evaluations scheduled to assess long-term impact and sustainability.

• Publisher: ProQuest Dissertations & Theses
• Subject: Curriculum development / Educational evaluation / Educational technology / Instructional Design
• ISBN: 9798270242886