Explore the vast range of titles available.

Background Assessing competence of advanced undergraduate medical students based on performance in the clinical context is the ultimate, yet challenging goal for medical educators to provide constructive alignment between undergraduate medical training and professional work of physicians. Therefore, we designed and validated a performance-based 360-degree assessment for competences of advanced undergraduate medical students. Methods This study was conducted in three steps: 1) Ten facets of competence considered to be most important for beginning residents were determined by a ranking study with 102 internists and 100 surgeons. 2) Based on these facets of competence we developed a 360-degree assessment simulating a first day of residency. Advanced undergraduate medical students (year 5 and 6) participated in the physician’s role. Additionally knowledge was assessed by a multiple-choice test. The assessment was performed twice (t 1 and t 2 ) and included three phases: a consultation hour, a patient management phase, and a patient handover. Sixty-seven (t 1 ) and eighty-nine (t 2 ) undergraduate medical students participated. 3) The participants completed the Group Assessment of Performance (GAP)-test for flight school applicants to assess medical students‘ facets of competence in a non-medical context for validation purposes. We aimed to provide a validity argument for our newly designed assessment based on Messick’s six aspects of validation: (1) content validity, (2) substantive/cognitive validity, (3) structural validity, (4) generalizability, (5) external validity, and (6) consequential validity. Results Our assessment proved to be well operationalised to enable undergraduate medical students to show their competences in performance on the higher levels of Bloom’s taxonomy. Its generalisability was underscored by its authenticity in respect of workplace reality and its underlying facets of competence relevant for beginning residents. The moderate concordance with facets of competence of the validated GAP-test provides arguments of convergent validity for our assessment. Since five aspects of Messick’s validation approach could be defended, our competence-based 360-degree assessment format shows good arguments for its validity. Conclusion According to these validation arguments, our assessment instrument seems to be a good option to assess competence in advanced undergraduate medical students in a summative or formative way. Developments towards assessment of postgraduate medical trainees should be explored.

Objective. To evaluate levels of entrustability and practice readiness in advanced pharmacy practice experience (APPE) students using a pilot instrument designed to assess their competency in performing the entrustable professional activities (EPAs) expected of new pharmacy graduates. Methods. A pilot instrument was developed directly from EPAs to measure entrustability levels on a scale of one to five. Five APPE preceptors from several different practice areas participated. Fourth-year students used the instrument to self-evaluate their knowledge and skills at the beginning, midpoint, and end of the APPE. The preceptors evaluated students using the same instrument at APPE midpoint and end. The instrument had assigned weights for each EPA and entrustability level for a score of 100 if all items were marked five. If a rating of non-applicable was chosen, score adjustments were made. All students in the graduating class of 2018 were invited to perform a self-evaluation at the end of the fourth (APPE) year using the same instrument that study participants used. Results. Twenty-eight students and five preceptors completed evaluations during the APPE year. Overall scores from both preceptor evaluations of students and student self-evaluations increased significantly from pre-APPE to midpoint to final. Student self-evaluations were only slightly higher than preceptor evaluations. The mean (SD) preceptor scores for students and student self-assessment scores at the end of each APPE were 85.4% (7.1) and 87.2% (10.3), respectfully. One practice manager EPA and three population health EPAs were considered to not be applicable by preceptors on ≥50% of evaluations. Approximately 94% of all graduating students completed the year-end self-evaluations, with a mean (SD) score of 89% (8.6) and no EPAs marked as not applicable. Conclusion. Pharmacy students’ proficiency in EPA improved during individual APPEs. According to preceptors, students’ greatest improvement in entrustability was in educating patients and colleagues regarding appropriate use of medications and collecting information to identify medication-related problems.

PurposeResearchers piloted a problem-based learning (PBL) activity in a master’s degree-granting strategic studies program to explore how students apply knowledge and skills learned from the curriculum to their formulation of a strategy addressing a real-world global security scenario.Design/methodology/approachThis mixed-methods pilot study used ethnographic observation, participant feedback, document analysis and surveys to assess the learning and engagement of multinational postgraduate students in the context of a PBL environment.FindingsFindings revealed gaps in students’ causal logic and literacy, as well as student discomfort with ambiguity and reliance upon heuristic frameworks over willingness to conduct substantive, current and relevant research. Additionally, observed group dynamics represented a lack of inclusive collaboration in mixed gender and multinational teams. These findings suggest foundational issues with the curriculum, teaching methodologies and evaluation practices of the studied institution.Originality/valueThis study highlights the need to include explicit instruction in problem-solving and causal literacy (i.e. logical reasoning) in postgraduate programs for national and global security professionals, as well as authentic opportunities for those students to practice interpersonal communication.

This study investigated the Performance-Based Assessment (PBA) impact on academic resilience (AR), motivation, teacher support (TS), and personal best goals (PBGs) in different learning environments, specifically online classes and traditional physical classrooms. The research involved 84 participants divided into experimental (online classes, N = 41), and control (physical classes, N = 43) groups. Questionnaires were administered before and after the treatment to assess the participants’ AR, motivation, TS, and PBGs. The data were analyzed using Chi-square tests, revealing significant differences in AR, motivation, and PBGs between the two groups after the treatment. Online classes were found to enhance AR, motivation, PBGs, and acknowledgment of TS compared to the physical environment. These results suggest that PBA can have a positive impact on students’ psychosocial variables and shed light on the potential benefits of online learning environments. The implications of the study are discussed, and suggestions for further research are provided.

The development of any assessment should be an iterative and careful process. Ideally, this process is guided by a well-defined framework (see for example Downing in: Downing and Haladyna (eds) Handbook of test development, Lawrence Erlbaum Associates, Mahwah, 2006; Mislevy et al. in On the roles of task model variables in assessment design (CSE Technical Report 500), Educational Testing Service, Princeton, 1999; AERA et al. in Standards for educational and psychological testing, AERA, Washington, DC, 2004), but such a framework is not always available when the instrument to be developed is new or innovative. Frameworks for the development of traditional computer-based tests have been published and experimented with since the late 1990s, by which time CBT had already existed for more than a decade. In an earlier empirical pilot study, we described a new type of assessment for Dutch vocational education, called multimedia-based performance assessment (MBPA) (self-revealing reference 2014). This CBT uses multiple media formats and interactive tasks to measure skills that are currently measured by performancebased assessment. In conducting that pilot study, deficits in the existing literature made it difficult to ground all developmental steps in sound scientific theory. To remedy those deficits, this article presents and validates a framework for the design and development of MBPA, combining a search of the relevant literature from several subfields of educational assessment and consultation with assessment experts. The framework unites assessment development and multimedia development theory, focus solely on vocational education, and answers the call for a framework from the scientific community. The first step in validating the prototype framework involved five semi-structured interviews with Dutch assessment and multimedia experts to produce a final version of the framework. Second, the pilot MBPA was reconstructed in accordance with this finalized framework, resulting in an improved MBPA and demonstrating that the proposed framework is a useful and applicable tool for the design and development of MBPA in vocational education.

This study examined the reliability and validity of the Executive Function Performance Test (EFPT). The EFPT assesses executive function deficits in the performance of real-world tasks. It uses a structured cueing and scoring system to assess higher-level cognitive functions, specifically initiation, organization, sequencing safety and judgment, and task completion. Seventy-three participants with mild to moderate stroke and 22 age- and education-matched controls completed the 4 EFPT tasks (cooking, using the telephone, managing medications, and paying bills). Significant differences were found between participants with mild and moderate stroke and healthy control participants. The EFPT can help occupational therapists determine the level of support needed by people with cognitive impairments to perform complex instrumental tasks. Objective information derived from this assessment is an essential part of the process of determining whether the person can live independently and helping families understand how to support the performance of their family members at home.

The types of assessment tasks affect the learners’ psychological well-being and the process of learning. For years, educationalists were in search of finding and implementing accurate and convenient approaches to assess learners efficiently. Despite the significant role of performance-based assessment (PBA) in affecting second/foreign language (L2) learning processes, few empirical studies have tried to explore how PBA affects reading comprehension achievement (RCA), academic motivation (AM), foreign language anxiety (FLA), and students’ self-efficacy (SS-E). To fill this lacuna of research, the current study intended to gauge the impact of PBA on the improvement of RCA, AM, FLA, and SS-E in English as a foreign language (EFL) context. In so doing, a sample of 88 intermediate EFL learners were randomly divided into experimental group (EG) and CG (control group). During this research (16 sessions), the learners in the CG (N = 43) received the tradition assessment. In contrast, the learners in the EG (N = 45) were exposed to some modification based on the underpinning theories of PBA. Data inspection applying the one-way multivariate analysis of variance (i.e., the one-way MANOVA) indicated that the learners in the EG outperformed their counterparts in the CG. The results highlighted the significant contributions of PBA in fostering RCA, AM, FLA, and S-E beliefs. The implications of this study may redound to the benefits of language learners, teachers, curriculum designers, and policy makers in providing opportunities for further practice of PBA.

Background Physicians need a set of specific competences to perform well in interprofessional teams in their first year of residency. These competences should be achieved with graduation from medical school. Assessments during undergraduate medical studies are mostly rated by supervisors only. The aim of our study was to compare the rating of core facets of competence of medical students late in their undergraduate training as well as the rating confidence between three different groups of assessors (supervisors, residents, and nurses) in an assessment simulating the first day of residency. Methods Sixty-seven advanced medical students from three different medical schools (Hamburg, Oldenburg and Munich) participated in a 360-degree assessment simulating the first working day of a resident. Each participant was rated by three assessors – a supervisor, a resident and a nurse – in seven facets of competence relevant for the first year of residency: (1) responsibility, (2) teamwork and collegiality, (3) knowing and maintaining own personal bounds and possibilities, (4) structure, work planning and priorities, (5) coping with mistakes, (6) scientifically and empirically grounded method of working, and (7) verbal communication with colleagues and supervisors. Means of all assessed competences and confidences of judgement of the three rating groups were compared. Additionally, correlations between assessed competences and confidence of judgement within each group of raters were computed. Results All rating groups showed consistent assessment decisions (Cronbach’s α : supervisors = .90, residents = .80, nurses = .78). Nurses assessed the participants significantly higher in all competences compared to supervisors and residents (all p  ≤ .05) with moderate and high effect sizes ( d  = .667–1.068). While supervisors’ and residents’ ratings were highest for “teamwork and collegiality”, participants received the highest rating by nurses for “responsibility”. Competences assessed by nurses were strongly positively correlated with their confidence of judgment while supervisors’ assessments correlated only moderately with their confidence of judgment in two competences. Conclusions Different professional perspectives provide differentiated competence ratings for medical students in the role of a beginning resident. Rating confidence should be enhanced by empirically derived behavior checklists with anchors, which need to be included in rater training to decrease raters’ subjectivity.

A quasi-experimental research design was conducted to investigate the impact of performance-based assessment (PBA) on pre-service teachers’ (PSTs) self-efficacy and academic achievement in general physics at Atebubu College of Education. The study was conducted on 144 primary education students in the second year who were assigned into experimental and control groups after conducting a pre-test. 72 students each were non-randomly assigned into the experimental and control group. Information on PSTs’ self-efficacy was derived by administering pre-service teachers’ self-efficacy scale whiles the academic achievement was determined by general physics achievement test. Statistical package for social sciences was used to analyze data to generate descriptive and inferential statistics. The self-efficacy of the experimental group and control group differed significantly after treatment. The experimental group had significantly higher post-test scores than the control group. This study identified PBA as a useful teaching strategy that improves PSTs’ problem-solving abilities and boosts their confidence in studying general physics.

Background The 5-item non-technical skills scale for trauma (T-NOTECHS) with five response categories is developed to assess non-technical skills in trauma team resuscitations. This validated instrument assesses behavioral aspects in teamwork. Outcome instruments should undergo a robust adaptation process followed by psychometric validation to maintain their measurement properties when translated into different languages. The translatability of the T-NOTECHS into a non-Anglo-Saxon language has not been thus far unraveled. The authors aimed to assess whether the T-NOTECHS would be translatable into a non-Anglo-Saxon language and to investigate its psychometric properties for simulated multi-professional trauma team resuscitations. Methods The T-NOTECHS (scores: 1 = poor; 5 = excellent) was translated and cross-culturally adapted into Finnish. Data was derived from 61 real hospital trauma team resuscitation simulations with 193 multi-professional participants. Floor-ceiling effects, internal consistency, and inter-rater reliability were analyzed. An exploratory factor analysis was conducted to test construct validity. Results After pre-testing, minor changes were made to the Finnish translation of the T-NOTECHS. Mean scores of two raters were 3.76 and 4.01, respectively. The T-NOTECHS instrument showed no floor-effect either in single items or in the total score. The total score of the T-NOTECHS instrument showed a percentage of maximum scores of 1.6 and 4.9% by the Raters 1 and 2, respectively. Internal consistency (Cronbach’s alpha) was 0.70 with inter-item correlation of 0.54. The intraclass correlation coefficient was 0.54 and coefficient of repeatability 1.53. The T-NOTECHS loaded on one factor. Conclusions The T-NOTECHS translated well into a difficult non-Anglo-Saxon language. The rigorous adaptation process used here can be recommended in the translation of observational performance assessment instruments. The translated version demonstrated fair reliability and good construct validity for assessing team performance in simulated multi-professional trauma team resuscitations. The translated T-NOTECHS instrument can be used to assess the efficacy of simulated in-situ trauma team resuscitations allowing benchmarking and international collaboration.