Explore the vast range of titles available.

Background Interviewers’ judgements play a critical role in competency-based assessments for selection such as the multiple-mini-interview (MMI). Much of the published research focuses on the psychometrics of selection and the impact of rater subjectivity. Within the context of selecting for entry into specialty postgraduate training, we used an interpretivist and socio-constructivist approach to explore how and why interviewers make judgments in high stakes selection settings whilst taking part in an MMI. Methods We explored MMI interviewers’ work processes through an institutional observational approach, based on the notion that interviewers’ judgements are socially constructed and mediated by multiple factors. We gathered data through document analysis, and observations of interviewer training, candidate interactions with interviewers, and interviewer meetings. Interviews included informal encounters in a large selection centre. Data analysis balanced description and explicit interpretation of the meanings and functions of the interviewers’ actions and behaviours. Results Three themes were developed from the data showing how interviewers make professional judgements, specifically by; ‘Balancing the interplay of rules and agency,’ ‘Participating in moderation and shared meaning making; and ‘A culture of reflexivity and professional growth.’ Interviewers balanced the following of institutional rules with making judgment choices based on personal expertise and knowledge. They engaged in dialogue, moderation, and shared meaning with fellow interviewers which enabled their consideration of multiple perspectives of the candidate’s performance. Interviewers engaged in self-evaluation and reflection throughout, with professional learning and growth as primary care physicians and supervisors being an emergent outcome. Conclusion This study offers insights into the judgment-making processes of interviewers in high-stakes MMI contexts, highlighting the balance between structured protocols and personal expertise within a socially constructed framework. By linking MMI practices to the broader work-based assessment literature, we contribute to advancing the design and implementation of more valid and fair selection tools for postgraduate training. Additionally, the study underscores the dual benefit of MMIs—not only as a selection tool but also as a platform for interviewers’ professional growth. These insights offer practical implications for refining future MMI practices and improving the fairness of high-stakes selection processes.

Background Multiple Mini Interviews (MMIs) are being used by a growing number of postgraduate training programs and medical schools as their interview process for selection entry. The Australian General Practice and Training (AGPT) used a National Assessment Centre (NAC) approach to selection into General Practice (GP) Training, which include MMIs. Interviewing is a resource intensive process, and implementation of the MMI requires a large number of interviewers, with a number of candidates being interviewed simultaneously. In 2015, 308 interviewers participated in the MMI process – a decrease from 340 interviewers in 2014, and 310 in 2013. At the same time, the number of applicants has steadily increased, with 1930 applications received in 2013; 2254 in 2014; and 2360 in 2015. This has raised concerns regarding the increasing recruitment needs, and the need to retain interviewers for subsequent years of MMIs. In order to investigate interviewers’ reasons for participating in MMIs, we utilised self-determination theory (SDT) to consider interviewers’ motivation to take part in MMIs at national selection centres. Methods In 2015, 308 interviewers were recruited from 17 Regional Training Providers (RTPs) to participate in the MMI process at one of 15 NACs. For this study, a convenience sample of NAC sites was used. Forty interviewers were interviewed ( n  = 40; 40/308 = 13%) from five NACs. Framework analysis was used to code and categorise data into themes. Results Interviewers’ motivation to take part as interviewers were largely related to their sense of duty, their desire to contribute their expertise to the process, and their desire to have input into selection of GP Registrars; a sense of duty to their profession; and an opportunity to meet with colleagues and future trainees. Interviewers also highlighted factors hindering motivation, which sometimes included the large number of candidates seen in one day. Conclusion Interviewers’ motivation for contributing to the MMIs was largely related to their desire to contribute to their profession, and ultimately improve future patient care. Interviewers recognised the importance of interviewing, and felt their individual roles made a crucial contribution to the profession of general practice. Good administration and leadership at each NAC is needed. By gaining an understanding of interviewers’ motivation, and enhancing this, engagement and retention of interviewers may be increased.

Background Entry into specialty training was determined by a National Assessment Centre (NAC) approach using a combination of a behavioural Multiple-Mini-Interview (MMI) and a written Situational Judgement Test (SJT). We wanted to know if interviewers could make reliable and valid decisions about the non-cognitive characteristics of candidates with the purpose of selecting them into general practice specialty training using the MMI. Second, we explored the concurrent validity of the MMI with the SJT. Methods A variance components analysis estimated the reliability and sources of measurement error. Further modelling estimated the optimal configurations for future MMI iterations. We calculated the relationship of the MMI with the SJT. Results Data were available from 1382 candidates, 254 interviewers, six MMI questions, five alternate forms of a 50-item SJT, and 11 assessment centres. For a single MMI question and one assessor, 28% of the variance between scores was due to candidate-to-candidate variation. Interviewer subjectivity, in particular the varying views that interviewer had for particular candidates accounted for 40% of the variance in scores. The generalisability co-efficient for a six question MMI was 0.7; to achieve 0.8 would require ten questions. A disattenuated correlation with the SJT (r = 0.35), and in particular a raw score correlation with the subdomain related to clinical knowledge (r = 0.25) demonstrated evidence for construct and concurrent validity. Less than two per cent of candidates would have failed the MMI. Conclusion The MMI is a moderately reliable method of assessment in the context of a National Assessment Centre approach. The largest source of error relates to aspects of interviewer subjectivity, suggesting enhanced interviewer training would be beneficial. MMIs need to be sufficiently long for precise comparison for ranking purposes. In order to justify long term sustainable use of the MMI in a postgraduate assessment centre approach, more theoretical work is required to understand how written and performance based test of non-cognitive attributes can be combined, in a way that achieves acceptable generalizability, and has validity.

Background Internationally, recruiting the best candidates is central to the success of postgraduate training programs and the quality of the medical workforce. So far there has been little theoretically informed research considering selection systems from the perspective of the candidates. We explored candidates’ perception of the fairness of a National Assessment Centre (NAC) approach for selection into Australian general practice training, where candidates were assessed by a Multiple Mini Interview (MMI) and a written Situational Judgment Test (SJT), for suitability to undertake general practice (GP) training. Methods In 2013, 1,930 medical practitioners, who were eligible to work in Australia attended one of 14 NACs in each of 5 states and 2 territories. A survey was distributed to each candidate at the conclusion of their assessment, which included open-ended questions aimed at eliciting candidates’ perceived benefits and challenges of the selection process. A framework analysis was informed by the theoretical lens of Social Validity Theory. Results Qualitative data was available from 46% (n = 886/1,930) of candidates, who found the NAC experience fair and informative for their training and career goals, but wanted to be provided with more information in preparation. Candidates valued being able to communicate their skills during the MMI, but found some difficulty in interpreting the questions. A significant minority had concerns that a lack of relevant GP experience may inhibit their performance. Candidates also expressed concerns about the time limits within the written paper, particularly if English was not their first language. They also expressed a desire for formative feedback during the interview process. Conclusion During any job selection process, not only is the organisation assessing the candidates, but the candidates are also assessing the organisation. However, a focus on the candidate experience throughout an organisation’s selection process may provide benefits to both candidates and the organisation, regardless of whether or not candidates secured the job. Social Validity Theory is a useful addition to the methods for demonstrating the reasonableness of any selection system.