Explore the vast range of titles available.

The aim of the present study is to reconcile previous findings (a) that testing mode has no effect on test outcomes or cognitive load (Comput Hum Behav 77:1–10, 2017) and (b) that younger learners’ working memory processes are more sensitive to computer-based test formats (J Psychoeduc Assess 37(3):382–394, 2019). We addressed key methodological limitations in past cognitive load research by employing a repeated measures design with 263, year 9 (aged 13–14) science students in Western Australia. Question difficulty (intrinsic cognitive load) and test mode (extraneous cognitive load) were manipulated to measure changes in test performance, cognitive load and scratch paper use on equivalent paper and computer-based versions of an Ohm’s Law revision quiz. Hierarchical linear modelling indicated significantly higher paper-based test performance on difficult questions in addition to greater cognitive load and scratch paper use for all paper questions. Testing mode effects on test score, as well as both measures of cognitive load, were not significant when controlling for working memory capacity, although the testing mode*question difficulty interaction remained significant. Together, these results contradict previous findings that computer-based testing can be implemented without consequence for all learners. With the increased use of computer-based testing in national and international-level assessments, these findings warrant further research into the effect of different testing modes on school-aged students.

The outbreak of the COVID-19 pandemic has transformed the educational landscape in a way unseen before. Educational institutions are navigating between offline and online learning worldwide. Computer-based testing is rapidly taking over paper-and-pencil testing as the dominant mode of assessment. In some settings, computer-based and paper-and-pencil assessments can also be offered side-by-side, in which case test developers should ensure the evidence of equivalence between both versions. This study aims to establish the equivalency evidence of different delivery modes of the English Competency Test, an English language assessment for civil service officers developed and used by the Human Resources Development Education and Training Center, a civil service training institution under the Ministry of Finance of the Republic of Indonesia. Psychometric analyses were carried out with the Rasch model to measure the unidimensionality, reliability, separation, and standard error of measurement estimates. The findings demonstrate that the paper-and-pencil and computer-based versions of the language assessment exhibit comparatively equivalent psychometric properties. The computer-based version of the English Competency Test is proven to offer a reliable and comparable alternative to the paper-and-pencil version.

This review illustrates how, after unilateral brain damage, the presence and severity of spatial awareness deficits for the contralesional hemispace depend greatly on the quantity of attentional resources available for performance. After a brief description of neglect and extinction, different frameworks accounting for spatial and non-spatial attentional processes will be outlined. The central part of the review describes how the performance of brain-damaged patients is negatively affected by increased task demands, which can result in the emergence of severe awareness deficits for contralesional space even in patients who perform normally on paper-and-pencil tests. Throughout the review neglect is described as a spatial syndrome that can be exacerbated in the presence and severity by both spatial and non-spatial tasks. The take-home message is that the presence and degree of contralesional neglect and extinction can be dramatically overlooked based on standard clinical (paper-and-pencil) testing, where patients can easily compensate for their deficits. Only tasks where compensation is made impossible represent an appropriate approach to detect these disabling contralesional deficits of awareness when they become subtle in post-acute stroke phases.

Background Current demand for multiple-choice questions (MCQs) in medical assessment is greater than the supply. Consequently, an urgency for new item development methods arises. Automatic Item Generation (AIG) promises to overcome this burden, generating calibrated items based on the work of computer algorithms. Despite the promising scenario, there is still no evidence to encourage a general application of AIG in medical assessment. It is therefore important to evaluate AIG regarding its feasibility, validity and item quality. Objective Provide a narrative review regarding the feasibility, validity and item quality of AIG in medical assessment. Methods Electronic databases were searched for peer-reviewed, English language articles published between 2000 and 2021 by means of the terms ‘Automatic Item Generation’, ‘Automated Item Generation’, ‘AIG’, ‘medical assessment’ and ‘medical education’. Reviewers screened 119 records and 13 full texts were checked according to the inclusion criteria. A validity framework was implemented in the included studies to draw conclusions regarding the validity of AIG. Results A total of 10 articles were included in the review. Synthesized data suggests that AIG is a valid and feasible method capable of generating high-quality items. Conclusions AIG can solve current problems related to item development. It reveals itself as an auspicious next-generation technique for the future of medical assessment, promising several quality items both quickly and economically.

The use of computer-based tests (CBTs), for both formative and summative purposes, has greatly increased over the past years. One major advantage of CBTs is the easy integration of multimedia. It is unclear, though, how to design such CBT environments with multimedia. The purpose of the current study was to examine whether guidelines for designing multimedia instruction based on the Cognitive Load Theory (CLT) and Cognitive Theory of Multimedia Learning (CTML) will yield similar effects in CBT. In a within-subject design, thirty-three vocational students completed a computer-based arithmetic exam, in which half of the items were presented in an original design format, and the other half was redesigned based on the CTML principles for instructional design (i.e., adapted format). Results showed that applying CTML principles to a CBT decreased the difficulty of the test items, i.e., students scored significantly higher on the adapted test items. Moreover, eyetracking data showed that the adapted items required less visual search and increased attention for the question and answer. Finally, cognitive load, measured as silent pauses during a secondary think-aloud task, decreased. Mean fixation duration (a different indicator of cognitive load), however, did not significantly differ between adapted and original items. These results indicate that applying multimedia principles to CBTs can be beneficial. It seems to prevent cognitive overload and helps students to focus on important parts of the test items (e.g., the question), leading to better test results.

Purpose We sought to provide empirical insight and develop theory for a new organizational phenomenon: remote proctoring for Internet-based tests. We examined whether this technology is effective at decreasing cheating and whether it has unintended effects on test-taker reactions, performance, or selection procedures. Design/methodology/approach Participants (582) were randomly assigned to a webcam proctored or honor code condition and completed two (one searchable, one non-searchable) cognitive ability tests online. Complete data were collected from 295 participants. We indirectly determined levels of cheating by examining the pattern of test-score differences across the two conditions. We directly measured dropout rates, test performance, and participants' perceived tension and invasion of privacy. Findings The use of remote proctoring was associated with more negative test-taker reactions and decreased cheating. Remote proctoring did not directly affect test performance or interact with individual differences to predict test performance or test-taker reactions. Implications Technological advances in selection should be accompanied by empirical evidence. Although remote proctoring may be effective at decreasing cheating, it may also have unintended effects on test-taker reactions. By outlining an initial classification of remote proctoring technology, we contribute to the theoretical understanding of technology-enhanced assessment, while providing timely insight into the practice of Internet-based testing. Originality/value We provide timely insight into the development and evaluation of remotely proctored tests. The current study utilizes a unique randomized experimental design in order to indirectly determine levels of cheating across two conditions. Following the results of the current study, we outline an integrative model for future research on remotely proctored tests.

The German school system employs centrally organized performance assessments (some of which are called \"VERA\") as a way of promoting lesson development. In recent years, several German federal states introduced a computer-based performance testing system which will replace the paper-pencil testing system in the future. Scores from computer-based testing are required to be equivalent to paper-pencil testing scores so that the new testing medium does not lead to disadvantages for students. Therefore, the current study aimed at investigating the size of the mode effect and the moderating impact of students' gender, academic achievement and mainly spoken language in everyday life. In addition, the variance of the mode effect across tasks was investigated. The study was conducted in four German federal states in 2019 using a field experimental design. The test scores of 5140 eighth-graders from 165 schools in the subject German were analysed. The results of multi-level modelling revealed that students' test scores in the computerized version of the VERA test were significantly lower than in the paper-pencil version. Students with a lower academic achievement were more disadvantaged by the VERA computerized test. The results were inconsistent regarding the interactions between testing mode and students' gender and mainly spoken language in everyday life. The variance of the mode effect across tasks was high. Research into different subjects and in other federal states and countries under different testing conditions might yield further evidence about the generalizability of these results. (ZPID).

伴隨數位時代來臨，國際大型評量開始轉換紙筆測驗為電腦化測驗。在此過程中，臺灣學生的學科表現是否會受電腦化測驗影響？以及是否會因其背景特性差異呈現數位落差現象？本研究經由分析國際數學與科學教育成就趨勢調查（Trends in International Mathematics and Science Study, TIMSS）2019及其橋接研究資料，檢視臺灣八年級學生數學與科學成就表現是否因紙筆測驗轉換為電腦化測驗而有變化，及其家庭社經背景、學校所在的都市發展程度、性別是否造成此差異程度不同，並探討電腦化問題解決與探究能力測驗之表現型態。研究結果顯示，對臺灣學生而言，相同試題由紙筆測驗轉變為電腦化測驗後，平均答對率顯著降低；電腦化測驗表現並未因學生背景特性而呈現數位落差；高家庭社經背景或都會區學校學生在電腦化問題解決與探究能力表現較一般題型差。本研究並據此結果提出教育政策與未來研究建議。 To meet the demands and advantages of the digital age, most international large-scale assessments have transitioned students’ response modes from paper-and-pencil to computer-based testing (CBT). With the aid of digitalization, CBT can better align with the fundamental principles of authentic assessment (Herrington & Herrington, 1998) by incorporating multimedia presentations and capturing test-takers’ maneuvering behaviors in greater detail. However, when the item characteristics or constructs that an item is designed to assess differ between paper-and-pencil and CBT formats, a mode effect may arise as a result of variations in students’ response modes. If this mode effect correlates with factors related to digital divide, such as disparities in students’ socioeconomic status (SES), the allocation of urban-rural resources, and gender (Cooper, 2006; Hsieh & Ming, 2022; Hu & Chang, 2023; Kuo, 2022), CBT raises concerns regarding educational equity. Therefore, addressing the digital divide in CBT implementations is crucial to ensuring educational equity and accessibility before full adoption. As the first assessment cycle to implement CBT, in the Trends in International Mathematics and Science Study (TIMSS) 2019, half of the participating countries, including Taiwan, implemented CBT as the primary testing format (eTIMSS) for the main survey while concurrently administering a bridge study using the paper-and-pencil format (paperTIMSS). The purpose of this bridge study was to identify and equate potential mode effects in TIMSS 2019, ensuring item equivalence across modes and supporting robust trend analyses across assessment cycles. Drawing on data of Taiwanese eighth-grade students from TIMSS 2019 and its accompanying bridge study, this research examines concerns regarding educational equity by examining the relationships between mode effects and three students background factors: SES, school location, and gender. Specifically, this study seeks to determine whether Taiwanese eighth-grade students’ performance in mathematics and science differed between the two testing formats and whether a digital divide was evident based on SES, school location, or gender. Additionally, TIMSS 2019 incorporated computerized innovative items to assess problem-solving and inquiry (PSI) tasks (Mullis et al., 2021). Therefore, this study also explores how Taiwanese eighth-grade students from different backgrounds performed on these computerized PSI tasks (eTIMSS-PSI). Method Analyzing the performance of Taiwanese eighth-grade students on trend items in mathematics and science from eTIMSS 2019 and its bridge study, paperTIMSS, mode effects were identified by comparing the item average percent correct between the paper-and-pencil and the CBT formats. Subsequently, the digital divide related to demographic factors was examined by comparing scale score performance on trend items between eTIMSS and paperTIMSS among students with varying SES, school locations, and genders. Finally, students’ performance on eTIMSS-PSI items was analyzed using scale scores to assess how Taiwanese eighth-grade students from different backgrounds performed. Results Significant reductions in the item average percent correct on trend items were observed in eTIMSS compared to paperTIMSS among Taiwanese eighth-grade students in both mathematics and science, indicating substantial mode effects across both subjects. Based on these identified mode effects, scaling procedures were implemented using the international average mode effect and performance data from previous assessment cycles (which included only paperTIMSS) for each subject to generate scale scores. Consequently, the average scale scores for mathematics and science were adjusted to ensure comparability between eTIMSS and paperTIMSS, as well as between TIMSS 2019 and prior assessment cycles. By comparing the average scale score performance between eTIMSS and paperTIMSS among Taiwanese eighth-grade students with varying SES, school locations, and genders, no significant difference was found in the relationship between mode effects and students’ background factors in either mathematics or science. Therefore, the digital divide due to mode effects was not evident in the scale scores across both subjects based on SES, school location, or gender. Furthermore, the analysis of eTIMSS-PSI scale scores for Taiwanese eighth-grade students with varying backgrounds revealed a declining trend in performance among students from higher SES compared to those from lower SES backgrounds, as well as among students attending schools in urban areas compared to those in rural areas. Conclusions The performance differences between paper-and-pencil and CBT formats in TIMSS 2019 mathematics and science among Taiwanese eighth-grade students from different backgrounds reveal the following key findings: 1. Overall mode effects: Taiwanese eighth-grade students, similar to their counterparts in most other countries, demonstrated lower average percent correct on items presented in the CBT format compared to the paper-and-pencil format in both mathematics and science. Substantial mode effects were observed, particularly in mathematics. 2. No widening digital divide: After applying scaling procedures to ensure comparability of scale score between the two testing formats, no evidence of a digital divide was found based on SES, school location, or gender, in either mathematics or science. Students from different backgrounds performed similarly across both formats. 3. Performance disadvantage on PSI tasks: As computerized PSI tasks captured test-takers’ maneuvering behaviors in greater detail, students from higher SES backgrounds or urban areas demonstrated a performance disadvantage on PSI tasks. It remains unclear whether this disadvantage results from weaker PSI skills among Taiwanese students in general or potential impacts of interactive nature of CBT items on specific student groups. Future research is needed to explore these factors. In the digital age, the transition to CBT is inevitable as technology continues to transform education and assessment practices. The integration of digital tools in learning environments makes the adoption of CBT essential for aligning with modern educational needs while offering more interactive and efficient assessment methods. The urgency of digitalization in education become even more apparent during the COVID-19 pandemic, highlighting the need for accessible and equitable digital learning solutions. While CBT offers benefits, concerns related to educational equity, including mode effects and the digital divide, remain. By analyzing the impact of CBT in TIMSS 2019, educators and policymakers can accordingly develop evidence-based strategies to ensure fair, reliable, and valid assessments for all students. With precise scaling procedures to eliminate originally evident mode effects, it is assuring that no widening digital divide was identified. In conclusion, the transition to digital assessments in TIMSS 2019 did not compromise educational equity for Taiwanese eighth-grade students.

Objective. To determine whether elimination of backward navigation during an examination resulted in changes in examination score or time to complete the examination. Methods. Student performance on six examinations in which backward navigation was eliminated was compared to performance on examinations administered to pharmacy students the previous year when backwards navigation was allowed. The primary comparison of interest was change in student performance on a subset of identical questions included on both examinations. Secondary outcomes included change in total examination score and completion time. Results. No significant reduction in examination scores was observed as a result of eliminating backward navigation. The average time that students spent on a question was significantly reduced on two of the six examinations. Conclusion. Restricting pharmacy students’ ability to revisit questions previously answered (elimination of backward navigation) on an examination had no adverse effect on scores or testing time when assessed across three years of the didactic pharmacy curriculum.

Computer-based testing (CBT), which refers to delivering assessments with computers, has been widely used in large English proficiency tests worldwide. Despite an increasing CBT in China, limited research is available concerning whether CBT can be used for the Test for English Majors-Band 4 (TEM 4). The current study investigated whether testing mode impacted TEM 4 score and factors (i.e., computer familiarity level and attitude towards CBT) that might correlate with performance on CBT of TEM 4. Overall 92 Chinese undergraduate students were randomly assigned to one of the groups, i.e., CBT or paper-based testing (PBT), and took the test. A mixed method was employed, including (1) quantitative and qualitative analysis of test performance in two modes, as well as CBT group participants’ computer familiarity and attitudes towards the mode; and (2) thematic analysis of semi-structured interviews. The results revealed that (1) test scores in CBT and PBT were comparable; (2) two items in the computer familiarity questionnaire, i.e., comfort level of reading articles on the computer and forgetting time when using computers, positively correlated with CBT scores; and (3) participants’ attitude towards CBT did not impact test performance.