Explore the vast range of titles available.

This bibliometric analysis attempts to map out the scientific literature on automated writing evaluation (AWE) systems for teaching, learning, and assessment. A total of 170 documents published between 2002 and 2021 in Social Sciences Citation Index journals were reviewed from four dimensions, namely size (productivity and citations), time (publication growth), space (geographical distribution and publication venues), and composition (topic development). Overall findings show an increasing or expanding trend in all four dimensions, which is likely to continue for the next several years. The field has also shown movement toward methodological and theoretical maturity, especially during the past five years. This study has also provided strong evidence of the positive impact of AWE tools on the different dependent variables, such as writing accuracy, writing quality, and plagiarism. In terms of topic development, data shows that the field has significantly expanded and has maintained a sustained interest in popular research topics. However, a great majority of them remain underexplored. Practical, theoretical, and methodological implications and directions for future studies are discussed.

Deep learning-based automated essay scoring (AES) models exhibit a remarkable ability to identify complex patterns within essays and then generate accurate score predictions in an end-to-end training fashion. However, these models face a critical limitation in explaining the specific patterns and features utilized for scoring, which are essential for interpreting the scores and offering constructive feedback to essay authors. Numerous studies have focused on essay scoring, with the aim of modeling prompt-specific, domain-adaptable, or trait-specific AES. While existing surveys on AES cover topics ranging from representation to scoring models, they primarily emphasize scoring models. This study addresses a crucial gap by encompassing research on feedback generation for essay assessment tasks. By delving into essay scoring and feedback generation, we synthesize several existing literature to provide readers with a comprehensive understanding of ongoing research in both deep learning-based essay scoring and automated feedback generation. We categorized the existing essay scoring studies into prompt-specific and cross-prompt AES models, noting that prompt-specific AES is extensively researched category. However, we have only come across a few studies concerning automated feedback generation, likely because of the limited availability of suitable datasets for researching such types of tasks. Moreover, this survey provides insights into approaches for essay representation, prevalent datasets, evaluation metrics, and challenges in automated essay scoring tasks. By shedding light on these aspects, our goal is to delineate the current landscape, identify key research directions, and pave the way for further advancements in automated essay assessment.

Various studies empirically proved the value of highly informative feedback for enhancing learner success. However, digital educational technology has yet to catch up as automated feedback is often provided shallowly. This paper presents a case study on implementing a pipeline that provides German-speaking university students enrolled in an introductory-level educational psychology lecture with content-specific feedback for a lecture assignment. In the assignment, students have to discuss the usefulness and educational grounding (i.e., connection to working memory, metacognition or motivation) of ten learning tips presented in a video within essays. Through our system, students received feedback on the correctness of their solutions and content areas they needed to improve. For this purpose, we implemented a natural language processing pipeline with two steps: (1) segmenting the essays and (2) predicting codes from the resulting segments used to generate feedback texts. As training data for the model in each processing step, we used 689 manually labelled essays submitted by the previous student cohort. We then evaluated approaches based on GBERT, T5, and bag-of-words baselines for scoring them. Both pipeline steps, especially the transformer-based models, demonstrated high performance. In the final step, we evaluated the feedback using a randomised controlled trial. The control group received feedback as usual (essential feedback), while the treatment group received highly informative feedback based on the natural language processing pipeline. We then used a six items long survey to test the perception of feedback. We conducted an ordinary least squares analysis to model these items as dependent variables, which showed that highly informative feedback had positive effects on helpfulness and reflection. (DIPF/Orig.).

Effective learning depends on effective feedback, which in turn requires a set of skills, dispositions and practices on the part of both students and teachers which have been termed feedback literacy. A previously published teacher feedback literacy competency framework has identified what is needed by teachers to implement feedback well. While this framework refers in broad terms to the potential uses of educational technologies, it does not examine in detail the new possibilities of automated feedback (AF) tools, especially those that are open by offering varying degrees of transparency and control to teachers. Using analytics and artificial intelligence, open AF tools permit automated processing and feedback with a speed, precision and scale that exceeds that of humans. This raises important questions about how human and machine feedback can be combined optimally and what is now required of teachers to use such tools skillfully. The paper addresses two research questions: Which teacher feedback competencies are necessary for the skilled use of open AF tools? and What does the skilled use of open AF tools add to our conceptions of teacher feedback competencies? We conduct an analysis of published evidence concerning teachers’ use of open AF tools through the lens of teacher feedback literacy, which produces summary matrices revealing relative strengths and weaknesses in the literature, and the relevance of the feedback literacy framework. We conclude firstly, that when used effectively, open AF tools exercise a range of teacher feedback competencies. The paper thus offers a detailed account of the nature of teachers’ feedback literacy practices within this context. Secondly, this analysis reveals gaps in the literature, signalling opportunities for future work. Thirdly, we propose several examples of automated feedback literacy, that is, distinctive teacher competencies linked to the skilled use of open AF tools.

A design study was conducted to test a machine learning (ML)-enabled automated feedback system developed to support students’ revision of scientific arguments using data from published sources and simulations. This paper focuses on three simulation-based scientific argumentation tasks called Trap, Aquifer, and Supply. These tasks were part of an online science curriculum module addressing groundwater systems for secondary school students. ML was used to develop automated scoring models for students’ argumentation texts as well as to explore emerging patterns between students’ simulation interactions and argumentation scores. The study occurred as we were developing the first version of simulation feedback to augment the existing argument feedback. We studied two cohorts of students who used argument only (AO) feedback (n = 164) versus argument and simulation (AS) feedback (n = 179). We investigated how AO and AS students interacted with simulations and wrote and revised their scientific arguments before and after receiving their respective feedback. Overall, the same percentages of students (49% each) revised their arguments after feedback, and their revised arguments received significantly higher scores for both feedback conditions, p < 0.001. Significantly greater numbers of AS students (36% across three tasks) reran the simulations after feedback as compared with the AO students (5%), p < 0.001. For AS students who reran the simulation, their simulation scores increased for the Trap task, p <. 001, and for the Aquifer task, p < 0.01. AO students who did not receive simulation feedback but reran the simulations increased simulation scores only for the Trap task, p <. 05. For the Trap and Aquifer tasks, students who increased simulation scores were more likely to increase argument scores in their revisions than those who did not increase simulation scores or did not revisit simulations at all after simulation feedback was provided. This pattern was not found for the Supply task. Based on these findings, we discuss strengths and weaknesses of the current automated feedback design, in particular the use of ML.

Technological tools that provide automated feedback on classroom teaching afford a unique opportunity for educators to engage in self-reflection and work towards improvement goals, in particular to ensure that their instructional environment is equitable and productive for students. More information is needed about how teachers experience automated professional learning tools, including what they perceive as relevant and impactful for their everyday teaching. This mixed-methods study explored the perceptions and engagement of 21 math teachers who used an AI-based tool that generates information about their discourse practices from classroom recordings. Findings indicate that teachers perceived the tool to have a high utility value, especially those who elected to use it over two school years. These teachers increased their use of talk moves over time, suggesting that they were making intentional changes due to their review and uptake of the personalized feedback. These results from this study speak to promising directions for developing AI-based professional learning tools that can support teacher learning and instructional improvement, particularly tools with robust perceived utility.

Providing feedback on students’ writing is considered important by both writing teachers and students. However, contextual constraints including excess workloads and large classes pose major and recurrent challenges for teachers. To lighten the feedback burden, teachers can take advantage of a range of automated feedback tools. This paper investigated how automated feedback can be integrated into traditional teacher feedback by analyzing the focus of teacher and Grammarly feedback through a written feedback analysis of language- and content-related issues. This inquiry considered whether and how successfully students exploited feedback from different sources in their revisions and how the feedback provisions helped improve their writing performance. The study sample of texts was made up of 216 argumentative and narrative essays written by 27 low-intermediate level students at a Myanmar university over a 13-week semester. By analyzing data from the feedback analysis, we found that Grammarly provided feedback on surface-level errors, whereas teacher feedback covered both lower- and higher-level writing concerns, suggesting a potential for integration. The results from the revision analysis and pre- and post-tests suggested that students made effective use of the feedback received, and their writing performance improved according to the assessment criteria. The data were triangulated with self-assessment questionnaires regarding students’ emic perspectives on how useful they found the feedback. The pedagogical implications for integrating automated and teacher feedback are presented.

This study examined a naturalistic, districtwide implementation of an automated writing evaluation (AWE) software program called MI Write in elementary schools. We specifically examined the degree to which aspects of MI Write were implemented, teacher and student attitudes towards MI Write, and whether MI Write usage along with other predictors like demographics and writing self-efficacy explained variability in students’ performance on a proximal and distal measure of writing performance. The participants included 1935 students in Grades 3–5 and 135 writing teachers from 14 elementary schools in a mid-Atlantic school district. Findings indicated that though MI Write was somewhat under-utilized, teachers and students held positive attitudes towards the AWE system. Usage of MI Write had a mixed and limited predictive effect on outcomes: The number of essays written had a small predictive effect on state test performance for Grades 3 and 5; gain on revision had a moderate predictive effect on posttest writing quality and a small predictive effect for Grade 5 state test performance. Students’ average AWE scores showed consistently moderate to large predictive effects for all outcomes. Interpreted in light of the underlying architecture of MI Write, findings have implications for other school districts considering implementing AWE as well as the design of AWE systems intended to support the teaching and learning of writing.

We present CPSCoach 2.0, an automated system that provides feedback, instructional scaffolding, and practice to help individuals improve three collaborative problem-solving (CPS) skills drawn from a theoretical CPS framework: construction of shared knowledge, negotiation/coordination, and maintaining team function. CPSCoach 2.0 was developed and tested in the context of computer-mediated collaboration (video conferencing) with an educational game. It automatically analyzes users’ speech during a round of collaborative gameplay to provide personalized feedback and to select a target CPS skill for improvement. After multiple cycles of iterative testing and refinement, we tested CPSCoach 2.0 in a user study where 21 dyads (n = 42) completed four rounds of feedback and scaffolding embedded within five rounds of game-play in a single session. Using a quasi-experimental matching procedure, we found that the use of CPSCoach 2.0 was associated with improvement in CPS skill development compared to matched controls. Further, users found the automated feedback to be moderately accurate and had positive perceptions of the system, and these impressions were stronger for those who received higher scores overall. Results demonstrate the use of automated feedback and instructional scaffolds to support the development of CPS skills.

IntroductionLarge language models (LLMs) are an emerging form of generative artificial intelligence (AI) with promising applications in medical education, and their ability to provide automated feedback may enhance medical imaging education for trainees. This review aims to systematically examine and synthesize the published literature on the use of LLMs in providing automated feedback in medical imaging education.MethodsWe conducted this systematic review in accordance with the PRISMA 2020 guidelines. A comprehensive search of the PubMed, Scopus, and Embase databases was conducted, covering studies published through January 2026. Our search strategy included keywords related to “feedback, generative artificial intelligence, large language models, radiology, and medical imaging.” Studies were eligible if they examined the use of LLMs to generate automated feedback for medical trainees within medical imaging education. Extracted data were synthesized using descriptive synthesis, with quality appraisal assessed using ROBINS-I and GRADE.ResultsOf 1,003 identified records, 7 met the inclusion criteria. All studies examined the applications of automated LLM feedback in the medical education of radiology residents, with one study also including fellows. Reported educational outcomes included enhanced report quality, improved diagnostic accuracy, and increased efficiency in discrepancy detection. LLM feedback was generally well-received among trainees, with learners expressing satisfaction with the LLM feedback and preferring a hybrid human-AI feedback model. Additionally, fine-tuned models generally showed stronger performance than general-purpose LLMs and demonstrated variable agreement with expert-human consensus.ConclusionLLMs show a potentially promising role as supportive tools for providing automated feedback in medical imaging education, alongside human feedback. This includes reported gains in accuracy, efficiency, and learner satisfaction. However, the current published evidence is preliminary and limited. Larger multicenter studies with standardized methods are necessary before widespread adoption can be justified. Our systematic review emphasizes that human expert oversight remains essential, as the current evidence supports preliminary technical feasibility, but not yet definitive educational effectiveness.Systematic review registrationhttps://www.crd.york.ac.uk/PROSPERO/view/CRD420251081394, Identifier CRD420251081394