Explore the vast range of titles available.

The justification for making a measurement can be sought in asking what decisions are based on measurement, such as in assessing the compliance of a quality characteristic of an entity in relation to a specification limit, SL. The relative performance of testing devices and classification algorithms used in assessing compliance is often evaluated using the venerable and ever popular receiver operating characteristic (ROC). However, the ROC tool has potentially all the limitations of classic test theory (CTT) such as the non-linearity, effects of ordinality and confounding task difficulty and instrument ability. These limitations, inherent and often unacknowledged when using the ROC tool, are tackled here for the first time with a modernised approach combining measurement system analysis (MSA) and item response theory (IRT), using data from pregnancy testing as an example. The new method of assessing device ability from separate Rasch IRT regressions for each axis of ROC curves is found to perform significantly better, with correlation coefficients with traditional area-under-curve metrics of at least 0.92 which exceeds that of linearised ROC plots, such as Linacre’s, and is recommended to replace other approaches for device assessment. The resulting improved measurement quality of each ROC curve achieved with this original approach should enable more reliable decision-making in conformity assessment in many scenarios, including machine learning, where its use as a metric for assessing classification algorithms has become almost indispensable.

This paper aims to provide a structured analysis of the performance metrics used in computer-aided detection (CAD) systems, specifically focusing on lung nodule detection in computed tomography (CT) images. By examining key metrics along with their respective strengths and limitations, this study offers guidelines to assist in selecting appropriate metrics. Evaluation methods for CAD systems for lung nodule detection are primarily categorized into per-scan and per-nodule approaches. For per-scan analysis, a key metric is the area under the receiver operating characteristic (ROC) curve (AUROC), which evaluates the ability of the system to distinguish between scans with and without nodules. For per-nodule analysis, the nodule-level sensitivity at fixed false positives per scan is often used, supplemented by the free-response receiver operating characteristic (FROC) curve and the competition performance metric (CPM). However, the CPM does not provide normalized scores because it theoretically ranges from zero to infinity and largely varies depending on the characteristics of the data. To address the advantages and limitations of ROC and FROC curves, an alternative FROC (AFROC) was introduced to combine the strengths of both per-scan and per-nodule analyses. This paper discusses the principles of each metric and their relative strengths, providing insights into their clinical implications and practical utility.

Background Timely, accurate diagnosis of pancreatic adenocarcinoma (PA) is hampered by the lack of effective circulating biomarkers. No single test has emerged that improves upon the commonly used biomarker cancer antigen 19–9 (CA 19–9) to discriminate PA from benign conditions effectively. The goals of this study were to validate two acute-phase proteins, haptoglobin and serum amyloid A (SAA), as biomarkers for PA and determine if the combination of haptoglobin, SAA, and CA 19–9 would improve PA diagnosis over CA 19–9 alone. Methods Levels of haptoglobin, SAA, and CA 19–9 were measured in pretreatment sera from 75 PA patients, 32 patients with chronic pancreatitis, 42 patients with other benign pancreatic disease or biliary stricture, and 150 healthy control subjects by enzyme-linked immunosorbent assay or colorimetric binding assay. Relative levels of haptoglobin or SAA were compared between groups using analysis of variance. The diagnostic accuracy of serum haptoglobin and SAA levels were investigated using receiver operating characteristics (ROC) analysis. Using classification tree analysis, an algorithm was developed that used haptoglobin, SAA, and CA 19–9 in a diagnostic screening panel. Results Both haptoglobin and SAA were significantly elevated in sera from PA patients compared to healthy control subjects ( p  < 0.0001) and patients with chronic pancreatitis ( p  = 0.01). Haptoglobin was significantly elevated in sera from PA patients relative to patients with other benign diseases ( p  = 0.0015), whereas SAA fell short of significance in the same comparison ( p  = 0.0508). ROC analysis indicated that haptoglobin [area under the curve (AUC) = 0.792] was a better diagnostic marker than SAA (AUC = 0.691) over multiple threshold cutoffs. Using specific cutoffs that minimized overall misclassification, haptoglobin yielded a sensitivity of 82.7% and a specificity of 71.1%, and SAA yielded a sensitivity of 34.7% and a specificity of 90.2% when discriminating PA cases from all non-PA controls. In the same sample set, CA 19–9 yielded a sensitivity of 77.3% and a specificity of 91.1%. Combining data from haptoglobin, SAA, and CA 19–9 in a diagnostic screening panel improved the overall accuracy when compared to CA 19–9 alone, yielding a sensitivity of 81.3% and a specificity of 95.5%. Conclusions These data demonstrate that haptoglobin and SAA are useful for discriminating PA from benign conditions as well as healthy controls when used in a diagnostic screening panel. This study supports the use of combined biomarkers for improved accuracy in the diagnosis of PA.

Background: An increasing number of diagnostic tests and biomarkers have been validated during the last decades, and this will still be a prominent field of research in the future because of the need for personalized medicine. Strict evaluation is needed whenever we aim at validating any potential diagnostic tool, and the first requirement a new testing procedure must fulfill is diagnostic accuracy. Summary: Diagnostic accuracy measures tell us about the ability of a test to discriminate between and/or predict disease and health. This discriminative and predictive potential can be quantified by measures of diagnostic accuracy such as sensitivity and specificity, predictive values, likelihood ratios, area under the receiver operating characteristic curve, overall accuracy and diagnostic odds ratio. Some measures are useful for discriminative purposes, while others serve as a predictive tool. Measures of diagnostic accuracy vary in the way they depend on the prevalence, spectrum and definition of the disease. In general, measures of diagnostic accuracy are extremely sensitive to the design of the study. Studies not meeting strict methodological standards usually over- or underestimate the indicators of test performance and limit the applicability of the results of the study. Key Messages: The testing procedure should be verified on a reasonable population, including people with mild and severe disease, thus providing a comparable spectrum. Sensitivities and specificities are not predictive measures. Predictive values depend on disease prevalence, and their conclusions can be transposed to other settings only for studies which are based on a suitable population (e.g. screening studies). Likelihood ratios should be an optimal choice for reporting diagnostic accuracy. Diagnostic accuracy measures must be reported with their confidence intervals. We always have to report paired measures (sensitivity and specificity, predictive values or likelihood ratios) for clinically meaningful thresholds. How much discriminative or predictive power we need depends on the clinical diagnostic pathway and on misclassification (false positives/negatives) costs.

In recent years, hemoglobin A1c (HbA1c) is accepted among the algorithms used for making diagnosis for diabetes and prediabetes since it does not require subjects to be prepared for giving a blood sample. The aim of this study is to assess the performance of HbA1c against fasting plasma glucose (FPG) and oral glucose tolerance test (OGTT) in detecting prediabetes and diabetes. A total of 315 subjects were included in this study. The success of HbA1c in distinguishing the three diagnostic classes was examined by three-way receiver operating characteristic (ROC) analysis. The best cut-off points for HbA1c were found for discriminating the three disease status. The performance of HbA1c, measured by the volume under the ROC surface (VUS), is found to be statistically significant (VUS = 0.535, P < 0.001). The best cut-off points for discriminating between normal and prediabetes groups and between prediabetes and diabetes groups are c1 = 5.2% and c2 = 6.4% respectively. The performance of HbA1c in distinguishing between the prediabetes and diabetes groups was higher than its ability in distinguishing between healthy and prediabetes groups. This study provides enough information to understand what proportion of diabetes patients were skipped with the HbA1c especially when the test result is healthy or prediabetes. If a subject was diagnosed as healthy or prediabetes by HbA1c, it would be beneficial to verify the status of that subject by the gold standard test (OGTT and FPG).

The aim of our study is to evaluate the diagnostic accuracy of double inversion recovery (DIR) in detection of multiple sclerosis (MS) lesions as well as the correlation between the expanded disability status scale (EDSS) and lesion load measurement detected by DIR, fluid attenuated inversion recovery (FLAIR) and T2 weighted imaging (T2WI) in order to reveal the essential role of DIR sequence in assessing clinical inability as a practicable experiment. A total of 97 patients were assessed on a 3T Siemens Skyra MRI scanner using DIR, FLAIR, and T2W_TSE sequences. EDSS was used to assess the physical disability in patients with MS. The diagnostic accuracy of DIR, FLAIR and T2WI sequences was also determined in different anatomical regions. Sensitivity and specificity were assessed by relative operating characteristics/ receiver operating characteristics (ROC) curve at different cut off points. Spearman correlation was applied to identify the significant relationships between the number of lesions displayed by DIR, FLAIR and T2WI at different regions and EDSS score. Our results pointed out the highest sensitivity (92.9%) and specificity (73.5%) for the number of lesions in infratentorial region at the cut-off point of 4.5 and the highest correlation between the number of lesions and EDSS was observed in infratentorial region (r= 0.584, p<0.001) for DIR sequence. According to the findings of ROC analysis, the number of lesions detected by DIR technique in the infratentorial region is the best predictor of EDSS as a gold standard. DIR can be used as a complementary technique comparing to conventional T2 and FLAIR sequences and describe physical and cognitive dysfunction as well. Due to the higher potential of the DIR sequence to reveal a greater number of MS lesions and to overcome the technical defect of conventional MRI sequences in the diagnosis of cortical lesions, it is recommended that DIR sequences be routinely added to MRI imaging protocols for patients with MS.

We evaluated the diagnostic value of long non-coding RNA growth arrest-specific transcript 5 (GAS5) and its relationship with hippocampal volume in Alzheimer's disease (AD). One hundred and eight patients with AD and 83 healthy controls were included, and demographic data, biochemical parameters, GAS5 levels, and hippocampal volume were recorded. Chi-squared tests or independent sample t-tests were used to compare the baseline characteristics, relative expression of GAS5, and hippocampal volume. Correlations between variables were determined using Spearman's rank correlation test. Receiver operating characteristic (ROC) curves were generated to compare the diagnostic value of GAS5 and total hippocampal volume in AD. The levels of GAS5 were significantly upregulated in patients with AD compared with those in controls and were negatively correlated with MMSE score. There were differences in left hippocampal volume, right hippocampal volume, and total hippocampal volume between the two groups. Total hippocampal volume was positively correlated with MMSE score and negatively correlated with GAS5 expression in patients with AD. The area under the curve (AUC) of for GAS5 expression was 0.831, the sensitivity was 61.1%, and the specificity was 95.2%. The AUC of the combined total hippocampal volume was 0.891, the sensitivity was 74.1%, and the specificity was 92.8%. The results suggested that GAS5 may be an excellent indicator of AD progression alone or in combination with hippocampal volume.

Ecological niche modeling (and the related species distribution modeling) has been used as a tool with which to assess potential impacts of climate change processes on geographic distributions of species. However, the factors introducing variation into niche modeling outcomes are not well understood: To this end, we used seven algorithms to develop models (Maxent, GARP, BIOCLIM, artificial neural networks, support‐vector machines, climate envelope, and environmental distance) to estimate the potential geographic distribution of olives (Olea europaea sensu lato, including Olea ferruginea) under two climatic data sets (current 2000 and future 2050). Five general circulation models and two representative concentration pathway scenarios were used as predictor variables in future projections of the geographic potential of this species; models were fit at global extents (10′ spatial resolution) but transferred and interpreted for a region of particular interest in Central Asia, which largely avoids problems with truncation of niche estimates. We found marked differences among approaches in predicted distributions and model performance, as well as in the future distributional pattern reconstructed, from one algorithm to another. These general approaches, when model‐to‐model variation is managed appropriately, appear promising in predicting the potential geographic distribution of O. europaea sensu lato and thus can be an effective tool in restoration and conservation planning for wild populations, as well as possible commercial plantations of this species.

A statistical model to predict oral frailty based on information obtained from questionnaires might help to estimate its prevalence and clarify its determinants. In this study, we aimed to develop and validate a predictive model to assess oral frailty thorough a secondary data analysis of a previous cross-sectional study on oral frailty conducted on 843 patients aged ≥ 65 years. The data were split into training and testing sets (a 70/30 split) using random sampling. The training set was used to develop a multivariate stepwise logistic regression model. The model was evaluated on the testing set and its performance was assessed using a receiver operating characteristic (ROC) curve. The final model in the training set consisted of age, number of teeth present, difficulty eating tough foods compared with six months ago, and recent history of choking on tea or soup. The model showed good accuracy in the testing set, with an area of 0.860 (95% confidence interval: 0.806–0.915) under the ROC curve. These results suggested that the prediction model was useful in estimating the prevalence of oral frailty and identifying the associated factors.

Classification criteria for Sjögren's syndrome (SS) were developed and validated between 1989 and 1996 by the European Study Group on Classification Criteria for SS, and broadly accepted. These have been re-examined by consensus group members, who have introduced some modifications, more clearly defined the rules for classifying patients with primary or secondary SS, and provided more precise exclusion criteria.