Explore the vast range of titles available.

Background: This investigation included both a study of potential non-invasive diagnostic approaches for the bladder cancer biomarker UBC® Rapid Test and a study including comparative methods about sensitivity–specificity characteristic (SS-ROC) and predictive receiver operating characteristic (PV-ROC) curves that used bladder cancer as a useful example. Methods: The study included 289 urine samples from patients with tumors of the urinary bladder, patients with non-evidence of disease (NED) and healthy controls. The UBC® Rapid Test is a qualitative point of care assay. Using a photometric reader, quantitative data can also be obtained. Data for pairs of sensitivity/specificity as well as positive/negative predictive values were created by variation of threshold values for the whole patient cohort, as well as for the tumor-free control group. Based on these data, sensitivity–specificity and predictive value threshold distribution curves were constructed and transformed into SS-ROC and PV-ROC curves, which were included in a single SS/PV-ROC plot. Results: The curves revealed TPP-asymmetric improper curves which cross the diagonal from above. Evaluation of the PV-ROC curve showed that two or more distinct positive predictive values (PPV) can correspond to the same value of a negative predictive value (NPV) and vice versa, indicating a complexity in PV-ROC curves which did not exist in SS-ROC curves. In contrast to the SS-ROC curve, the PV-ROC curve had neither an area under the curve (AUC) nor a range from 0% to 100%. Sensitivity of the qualitative assay was 58.5% and specificity 88.2%, PPV was 75.6% and NPV 77.3%, at a threshold value of approximately 12.5 µg/L. Conclusions: The SS/PV-ROC plot is a new diagnostic approach which can be used for direct judgement of gain and loss of predictive values, sensitivity and specificity according to varied threshold value changes, enabling characterization, comparison and evaluation of qualitative and quantitative bioassays.

Binary classification is a common task for which machine learning and computational statistics are used, and the area under the receiver operating characteristic curve (ROC AUC) has become the common standard metric to evaluate binary classifications in most scientific fields. The ROC curve has true positive rate  (also called sensitivity or recall ) on the y axis and false positive rate on the x axis, and the ROC AUC can range from 0 (worst result) to 1 (perfect result). The ROC AUC, however, has several flaws and drawbacks. This score is generated including predictions that obtained insufficient sensitivity and specificity, and moreover it does not say anything about positive predictive value  (also known as precision ) nor negative predictive value (NPV) obtained by the classifier, therefore potentially generating inflated overoptimistic results. Since it is common to include ROC AUC alone without precision and negative predictive value, a researcher might erroneously conclude that their classification was successful. Furthermore, a given point in the ROC space does not identify a single confusion matrix nor a group of matrices sharing the same MCC value. Indeed, a given (sensitivity, specificity) pair can cover a broad MCC range, which casts doubts on the reliability of ROC AUC as a performance measure. In contrast, the Matthews correlation coefficient (MCC) generates a high score in its [ - 1 ; + 1 ] interval only if the classifier scored a high value for all the four basic rates of the confusion matrix: sensitivity, specificity, precision, and negative predictive value. A high MCC (for example, MCC = 0.9), moreover, always corresponds to a high ROC AUC, and not vice versa. In this short study, we explain why the Matthews correlation coefficient should replace the ROC AUC as standard statistic in all the scientific studies involving a binary classification, in all scientific fields.

Background/Objectives: This work introduces accuracy- and precision-ROC curves in addition to SS– and PV–ROC curves and shows a novel means of profiling biomarker characteristics for validation of optimal cutoffs in clinical diagnostics and decision making. Methods: This investigation included 91 patients with a confirmed bladder cancer diagnosis and 1152 patients without evidence of cancer. The study performed a quantitative investigation of used-up test cassettes from the visual UBC® Rapid qualitative point-of-care assay, which had already been applied in routine diagnostics. Using a photometric reader, quantitative data could also be obtained from the test line of the used cassettes. The ROC curves were constructed using different thresholds or cutoff levels to determine the TP/TN and FP/FN values for each threshold at concentrations of 5, 10, 30, 50, 90, 110, 250 and 300 µg/L. The resulting TP/TN and FP/FN values were used to calculate the sensitivity/specificity, accuracy, precision and predictive values in order to plot the ROC curves with integrated cutoff value distributions and their index cutoff diagrams. Results: A common, optimal cutoff value for all the diagnostic parameters was derived with the aid of an ROC index cutoff diagram. It includes higher specificity and an acceptable number of NPVs. As a result, use of a sensitivity–specificity ROC curve and the Youden index only permits the selection of a maximal threshold value or cutoff point for the biomarker of interest but disregards the clinical status of the patient, whereas the precision, accuracy and predictive values give information related to the disease. Conclusions: This work’s novelty compared to the existing methodology includes the first international publication of accuracy- and precision-ROC curves. It enables the investigation of the relationship among the sensitivity, specificity, precision, accuracy and predictive values at varied cutoff levels within a bioassay, presenting these in a single graph consisting of selected receiver operating characteristic (ROC) curves for each parameter, including cutoff distribution curves. This is a transparent method to identify appropriate cutoffs for multiple diagnostic parameters. According to the results, the single-sided use of a sensitivity–specificity ROC curve including the maximal Youden index value as an optimal cutoff or single-point determinations for predictive values cannot provide diagnostic information of the same quality as that given by a multi-parameter diagnostic profile and a multi-parameter cutoff-index-diagram-derived optimal value as presented within this work. The proposed multi-parameter cutoff-index diagram includes novel index cutoff AOX. It is a new different method that allows a quantitative comparison of the results from multi-parameter ROC curves, which cannot be performed with the AUC. However, the methods are different and do not exclude each other.

Gold standard tests are usually used for diagnosis of a disease, but the gold standard tests may not be always available or cannot be administrated due to many reasons such as cost, availability, ethical issues etc. In such cases, some instruments or screening tools can be used to diagnose the disease. However, before the screening tools can be applied, it is crucial to evaluate the accuracy of these screening tools compared to the gold standard tests. In this assay, we will discuss how to assess the accuracy of a diagnostic test through an example using R program.

Introduction An important application of ROC analysis is the determination of the optimal cut-point for biomarkers in diagnostic studies. This comprehensive review provides a framework of cut-point election for biomarkers in diagnostic medicine. Methods Several methods were proposed for the selection of optional cut-points. The validity and precision of the proposed methods were discussed and the clinical application of the methods was illustrated with a practical example of clinical diagnostic data of C-reactive protein (CRP), erythrocyte sedimentation rate (ESR) and malondialdehyde (MDA) for prediction of inflammatory bowel disease (IBD) patients using the NCSS software. Results Our results in the clinical data suggested that for CRP and MDA, the calculated cut-points of the Youden index, Euclidean index, Product and Union index methods were consistent in predicting IBD patients, while for ESR, only the Euclidean and Product methods yielded similar estimates. However, the diagnostic odds ratio (DOR) method provided more extreme values for the optimal cut-point for all biomarkers analyzed. Conclusion Overall, the four methods including the Youden index, Euclidean index, Product, and IU can produce quite similar optimal cut-points for binormal pairs with the same variance. The cut-point determined with the Youden index may not agree with the other three methods in the case of skewed distributions while DOR does not produce valid informative cut-points. Therefore, more extensive Monte Carlo simulation studies are needed to investigate the conditions of test result distributions that may lead to inconsistent findings in clinical diagnostics.

Performance metrics are measures of success or performance that can be used to evaluate how well a model makes accurate predictions or classifications. However, there is no single measure since each performance metric emphasizes a different classification aspect. Model selection procedures based on information criteria offer a quantitative measure that balances model complexity with goodness of fit, providing a better alternative to classical approaches. In this paper, we introduce and develop a novel Information Complexity–Receiver Operating Characteristic, abbreviated as ICOMP-ROC, criterion approach to fit and study the performance of ROC curve models. We construct and derive the Universal ROC (UROC) for a combination of sixteen Bi-distributional ROC models to choose the best Bi-distributional ROC by minimizing the ICOMP-ROC criterion. We conduct large-scale Monte Carlo simulations using the sixteen Bi-distributional ROC models with the Normal–Normal and Weibull–Gamma pairs as the pseudo-true ROC models. We report the frequency of hits of the ICOMP-ROC criterion, showing its remarkable recovery rate. In addition to Bi-distributional fitting, we consider a high-dimensional real Magnetic Resonance Imaging (MRI) of the Brain dataset and Wisconsin Breast Cancer (WBC) dataset to study the performance of the common performance metrics and the ICOMP-ROC criterion using several machine learning (ML) classification algorithms. We use the genetic algorithm (GA) to reduce the dimensions of these two datasets to choose the best subset of the features to study and compare the performance of the newly proposed ICOMP-ROC criterion along with the traditional performance metrics. The choice of a suitable metric is not just contingent upon the ML model used, but it also depends upon the complexity and high dimensionality of the input datasets, since the traditional performance metrics give different results and have inherent limitations. Our numerical results show the consistency and reliability of the ICOMP-ROC criterion over the traditional performance metrics as a clever model selection criterion to choose the best fitting Bi-distributional ROC model and the best classification algorithm among the ones considered. This shows the utility and the versatility of our newly proposed approach in ROC curve modeling that integrates and robustifies currently used procedures.

Background: Transient elastography (FibroScan) is a new, non-invasive, rapid, and reproducible method allowing evaluation of liver fibrosis by measurement of liver stiffness. In cirrhotic patients, liver stiffness measurements range from 12.5 to 75.5 kPa. However, the clinical relevance of these values is unknown. The aim of this prospective study was to evaluate the accuracy of liver stiffness measurement for the detection of cirrhosis in patients with chronic liver disease. Methods: A total of 711 patients with chronic liver disease were studied. Aetiologies of chronic liver diseases were hepatitis C virus or hepatitis B virus infection, alcohol, non-alcoholic steatohepatitis, other, or a combination of the above aetiologies. Liver fibrosis was evaluated according to the METAVIR score. Results: Stiffness was significantly correlated with fibrosis stage (r = 0.73, p<0.0001). Areas under the receiver operating characteristic curve (95% confidence interval) were 0.80 (0.75–0.84) for patients with significant fibrosis (F>2), 0.90 (0.86–0.93) for patients with severe fibrosis (F3), and 0.96 (0.94–0.98) for patients with cirrhosis. Using a cut off value of 17.6 kPa, patients with cirrhosis were detected with a positive predictive value and a negative predictive value (NPV) of 90%. Liver stiffness was significantly correlated with clinical, biological, and morphological parameters of liver disease. With an NPV >90%, the cut off values for the presence of oesophageal varices stage 2/3, cirrhosis Child-Pugh B or C, past history of ascites, hepatocellular carcinoma, and oesophageal bleeding were 27.5, 37.5, 49.1, 53.7, and 62.7 kPa, respectively. Conclusion: Transient elastography is a promising non-invasive method for detection of cirrhosis in patients with chronic liver disease. Its use for the follow up and management of these patients could be of great interest and should be evaluated further.

BackgroundControlling Nutritional Status (CONUT), as calculated from serum albumin, total cholesterol concentration, and total lymphocyte count, was previously shown to be useful for nutritional assessment. The current study investigated the potential use of CONUT as a prognostic marker in gastric cancer patients after curative resection.MethodsPreoperative CONUT was retrospectively calculated in 416 gastric cancer patients who underwent curative resection at Kumamoto University Hospital from 2005 to 2014. The patients were divided into two groups: CONUT-high (≥4) and CONUT-low (≤3), according to time-dependent receiver operating characteristic (ROC) analysis. The associations of CONUT with clinicopathological factors and survival were evaluated.ResultsCONUT-high patients were significantly older (p < 0.001) and had a lower body mass index (p = 0.019), deeper invasion (p < 0.001), higher serum carcinoembryonic antigen (p = 0.037), and higher serum carbohydrate antigen 19-9 (p = 0.007) compared with CONUT-low patients. CONUT-high patients had significantly poorer overall survival (OS) compared with CONUT-low patients according to univariate and multivariate analyses (hazard ratio: 5.09, 95% confidence interval 3.12–8.30, p < 0.001). In time-dependent ROC analysis, CONUT had a higher area under the ROC curve (AUC) for the prediction of 5-year OS than the neutrophil lymphocyte ratio, the Modified Glasgow Prognostic Score, or pStage. When the time-dependent AUC curve was used to predict OS, CONUT tended to maintain its predictive accuracy for long-term survival at a significantly higher level for an extended period after surgery when compared with the other markers tested.ConclusionsCONUT is useful for not only estimating nutritional status but also for predicting long-term OS in gastric cancer patients after curative resection.

Background: This investigation is both a study of potential non-invasive diagnostic approaches for the bladder cancer biomarker UBC® Rapid test and a study including novel comparative methods for bioassay evaluation and comparison that uses bladder cancer as a useful example. The objective of the paper is not to investigate specific data. It is used only for demonstration, partially to compare ROC methodologies and also to show how both sensitivity/specificity and predictive values can be used in clinical diagnostics and decision making. This study includes ROC curves with integrated cut-off distribution curves for a comparison of sensitivity/specificity (SS) and positive/negative predictive values (PPV/NPV or PV), as well as SS-J index/PV-PSI index–ROC curves and SS-J/PV-PSI index cut-off diagrams (J = Youden, PSI = Predictive Summary Index) for the unified direct comparison of SS-J/PV results achieved via quantitative and/or qualitative bioassays and an identification of optimal separate or unified index cut-off points. Patients and Methods: According to the routine diagnostics, there were 91 patients with confirmed bladder cancer and 1152 patients with no evidence of bladder cancer, leading to a prevalence value of 0.073. This study performed a quantitative investigation of used-up test cassettes from the visual UBC® Rapid qualitative point-of-care assay, which had already been applied in routine diagnostics. Using a photometric reader, quantitative data could also be obtained from the test line of the used cassettes. Interrelations between SS and PV values were evaluated using cumulative distribution analysis (CAD), SS/PV–ROC curves, SS-J/PV-PSI index–ROC curves, and the SS-J/PV-PSI index cut-off diagram. The maximum unified SS-J/PV-PSI index value and its corresponding cut-off value were determined and calculated with the SS-J/PV-PSI index cut-off diagram. Results: The use of SS/PV–ROC curves with integrated cut-off concentration distribution curves provides improved diagnostic information compared to “traditional” ROC curves. The threshold distributions integrated as curves into SS/PV–ROC curves and SS-J/PV-PSI index–ROC curves run in opposite directions. In contrast to the SS–ROC curves, the PV–ROC and the novel PV-PSI index–ROC curves had neither an area under the curve (AUC) nor a range from 0% to 100%. The cut-off level of the qualitative assay was 7.5 µg/L, with a sensitivity of 65.9% and a specificity of 63.3%, and the PPV was 12.4% and the NPV was 95.9%, at a threshold value of 12.5 µg/L. Based on these set concentrations, the reader-based evaluation revealed a graphically estimated 5% increase in sensitivity and a 13% increase in specificity, as compared to the visual qualitative POC test. In the case of predictive values, there was a gain of 8% for PPV and 10% for NPV. The index values and cut-offs were as follows: visual SS-J index, 0.328 and 35 µg/L; visual PV-PSI index, 0.083 and 5.4 µg/L; maximal reader Youden index, 0.0558 and 250 µg/L; and maximal PV-PSI index, 0.459 and 250 µg/L, respectively. The maximum unified SS-J/PV-PSI index value was 0.32, and the cut-off was 43 µg/L. The reciprocal SS-J index correctly detected one out of three patients, while the reciprocal PV-PSI index gave one out of twelve patients a correct diagnosis. Conclusions: ROC curves including cut-off distribution curves supplement the information lost in “traditionally plotted” ROC curves. The novel sets of ROC and index–ROC curves and the new SS/PV index cut-off diagrams enable the simultaneous comparison of sensitivity/specificity and predictive value profiles of diagnostic tools and the identification of optimal cut-off values at maximal index values, even in a unifying SS/PV approach. Because the curves within an SS-J/PV-PSI index cut-off diagram are distributed over the complete cut-off range of a quantitative assay, this field is open for special clinical considerations, with the need to vary the mentioned clinical diagnostic parameters. Complete or partial areas over the x-axis (AOX) can be calculated for summarized quantitative or qualitative effectivity evaluations with respect to single and/or unified SS-J and PV-PSI indices and with respect to single, several, or several unified assays. The SS-J/PV-PSI index-AOX approach is a new tool providing additional joint clinical information, and the reciprocal SS-J indices can predict the number of patients with a correct diagnosis and the number of persons who need to be examined in order to correctly predict a diagnosis of the disease. These methods could be used in applications like medical or plant epidemiology, machine learning algorithms, and neural networks.

With the development of data mining, machine learning offers opportunities to improve discrimination by analyzing complex interactions among massive variables. To test the ability of machine learning algorithms for predicting risk of type 2 diabetes mellitus (T2DM) in a rural Chinese population, we focus on a total of 36,652 eligible participants from the Henan Rural Cohort Study. Risk assessment models for T2DM were developed using six machine learning algorithms, including logistic regression (LR), classification and regression tree (CART), artificial neural networks (ANN), support vector machine (SVM), random forest (RF) and gradient boosting machine (GBM). The model performance was measured in an area under the receiver operating characteristic curve, sensitivity, specificity, positive predictive value, negative predictive value and area under precision recall curve. The importance of variables was identified based on each classifier and the shapley additive explanations approach. Using all available variables, all models for predicting risk of T2DM demonstrated strong predictive performance, with AUCs ranging between 0.811 and 0.872 using laboratory data and from 0.767 to 0.817 without laboratory data. Among them, the GBM model performed best (AUC: 0.872 with laboratory data and 0.817 without laboratory data). Performance of models plateaued when introduced 30 variables to each model except CART model. Among the top-10 variables across all methods were sweet flavor, urine glucose, age, heart rate, creatinine, waist circumference, uric acid, pulse pressure, insulin, and hypertension. New important risk factors (urinary indicators, sweet flavor) were not found in previous risk prediction methods, but determined by machine learning in our study. Through the results, machine learning methods showed competence in predicting risk of T2DM, leading to greater insights on disease risk factors with no priori assumption of causality.