Catalogue Search | MBRL
Search Results Heading
Explore the vast range of titles available.
MBRLSearchResults
-
DisciplineDiscipline
-
Is Peer ReviewedIs Peer Reviewed
-
Item TypeItem Type
-
SubjectSubject
-
YearFrom:-To:
-
More FiltersMore FiltersSourceLanguage
Done
Filters
Reset
26,298
result(s) for
"ROC Curve"
Sort by:
Establishment and Characterization of an Empirical Biomarker SS/PV-ROC Plot Using Results of the UBC® Rapid Test in Bladder Cancer
2020
Background: This investigation included both a study of potential non-invasive diagnostic approaches for the bladder cancer biomarker UBC® Rapid Test and a study including comparative methods about sensitivity–specificity characteristic (SS-ROC) and predictive receiver operating characteristic (PV-ROC) curves that used bladder cancer as a useful example. Methods: The study included 289 urine samples from patients with tumors of the urinary bladder, patients with non-evidence of disease (NED) and healthy controls. The UBC® Rapid Test is a qualitative point of care assay. Using a photometric reader, quantitative data can also be obtained. Data for pairs of sensitivity/specificity as well as positive/negative predictive values were created by variation of threshold values for the whole patient cohort, as well as for the tumor-free control group. Based on these data, sensitivity–specificity and predictive value threshold distribution curves were constructed and transformed into SS-ROC and PV-ROC curves, which were included in a single SS/PV-ROC plot. Results: The curves revealed TPP-asymmetric improper curves which cross the diagonal from above. Evaluation of the PV-ROC curve showed that two or more distinct positive predictive values (PPV) can correspond to the same value of a negative predictive value (NPV) and vice versa, indicating a complexity in PV-ROC curves which did not exist in SS-ROC curves. In contrast to the SS-ROC curve, the PV-ROC curve had neither an area under the curve (AUC) nor a range from 0% to 100%. Sensitivity of the qualitative assay was 58.5% and specificity 88.2%, PPV was 75.6% and NPV 77.3%, at a threshold value of approximately 12.5 µg/L. Conclusions: The SS/PV-ROC plot is a new diagnostic approach which can be used for direct judgement of gain and loss of predictive values, sensitivity and specificity according to varied threshold value changes, enabling characterization, comparison and evaluation of qualitative and quantitative bioassays.
Journal Article
Interrelationships Among Sensitivity, Precision, Accuracy, Specificity and Predictive Values in Bioassays, Represented as Combined ROC Curves with Integrated Cutoff Distribution Curves and Novel Index Values
2025
Background/Objectives: This work introduces accuracy- and precision-ROC curves in addition to SS– and PV–ROC curves and shows a novel means of profiling biomarker characteristics for validation of optimal cutoffs in clinical diagnostics and decision making. Methods: This investigation included 91 patients with a confirmed bladder cancer diagnosis and 1152 patients without evidence of cancer. The study performed a quantitative investigation of used-up test cassettes from the visual UBC® Rapid qualitative point-of-care assay, which had already been applied in routine diagnostics. Using a photometric reader, quantitative data could also be obtained from the test line of the used cassettes. The ROC curves were constructed using different thresholds or cutoff levels to determine the TP/TN and FP/FN values for each threshold at concentrations of 5, 10, 30, 50, 90, 110, 250 and 300 µg/L. The resulting TP/TN and FP/FN values were used to calculate the sensitivity/specificity, accuracy, precision and predictive values in order to plot the ROC curves with integrated cutoff value distributions and their index cutoff diagrams. Results: A common, optimal cutoff value for all the diagnostic parameters was derived with the aid of an ROC index cutoff diagram. It includes higher specificity and an acceptable number of NPVs. As a result, use of a sensitivity–specificity ROC curve and the Youden index only permits the selection of a maximal threshold value or cutoff point for the biomarker of interest but disregards the clinical status of the patient, whereas the precision, accuracy and predictive values give information related to the disease. Conclusions: This work’s novelty compared to the existing methodology includes the first international publication of accuracy- and precision-ROC curves. It enables the investigation of the relationship among the sensitivity, specificity, precision, accuracy and predictive values at varied cutoff levels within a bioassay, presenting these in a single graph consisting of selected receiver operating characteristic (ROC) curves for each parameter, including cutoff distribution curves. This is a transparent method to identify appropriate cutoffs for multiple diagnostic parameters. According to the results, the single-sided use of a sensitivity–specificity ROC curve including the maximal Youden index value as an optimal cutoff or single-point determinations for predictive values cannot provide diagnostic information of the same quality as that given by a multi-parameter diagnostic profile and a multi-parameter cutoff-index-diagram-derived optimal value as presented within this work. The proposed multi-parameter cutoff-index diagram includes novel index cutoff AOX. It is a new different method that allows a quantitative comparison of the results from multi-parameter ROC curves, which cannot be performed with the AUC. However, the methods are different and do not exclude each other.
Journal Article
A new concordant partial AUC and partial c statistic for imbalanced data in the evaluation of machine learning algorithms
by
Holzinger, Andreas
,
Carrington, André M.
,
Fieguth, Paul W.
in
Algorithms
,
Area Under Curve
,
Area under the ROC curve
2020
Background
In classification and diagnostic testing, the receiver-operator characteristic (ROC) plot and the area under the ROC curve (AUC) describe how an adjustable threshold causes changes in two types of error: false positives and false negatives. Only part of the ROC curve and AUC are informative however when they are used with imbalanced data. Hence, alternatives to the AUC have been proposed, such as the partial AUC and the area under the precision-recall curve. However, these alternatives cannot be as fully interpreted as the AUC, in part because they ignore some information about actual negatives.
Methods
We derive and propose a new concordant partial AUC and a new partial
c
statistic for ROC data—as foundational measures and methods to help understand and explain parts of the ROC plot and AUC. Our partial measures are continuous and discrete versions of the same measure, are derived from the AUC and c statistic respectively, are validated as equal to each other, and validated as equal in summation to whole measures where expected. Our partial measures are tested for validity on a classic ROC example from Fawcett, a variation thereof, and two real-life benchmark data sets in breast cancer: the Wisconsin and Ljubljana data sets. Interpretation of an example is then provided.
Results
Results show the expected equalities between our new partial measures and the existing whole measures. The example interpretation illustrates the need for our newly derived partial measures.
Conclusions
The concordant partial area under the ROC curve was proposed and unlike previous partial measure alternatives, it maintains the characteristics of the AUC. The first partial c statistic for ROC plots was also proposed as an unbiased interpretation for part of an ROC curve. The expected equalities among and between our newly derived partial measures and their existing full measure counterparts are confirmed. These measures may be used with any data set but this paper focuses on imbalanced data with low prevalence.
Future work
Future work with our proposed measures may: demonstrate their value for imbalanced data with high prevalence, compare them to other measures not based on areas; and combine them with other ROC measures and techniques.
Journal Article
Clinical evaluation of fever-screening thermography: impact of consensus guidelines and facial measurement location
2020
Significance: Infrared thermographs (IRTs) have been used for fever screening during infectious disease epidemics, including severe acute respiratory syndrome, Ebola virus disease, and coronavirus disease 2019 (COVID-19). Although IRTs have significant potential for human body temperature measurement, the literature indicates inconsistent diagnostic performance, possibly due to wide variations in implemented methodology. A standardized method for IRT fever screening was recently published, but there is a lack of clinical data demonstrating its impact on IRT performance.
Aim: Perform a clinical study to assess the diagnostic effectiveness of standardized IRT-based fever screening and evaluate the effect of facial measurement location.
Approach: We performed a clinical study of 596 subjects. Temperatures from 17 facial locations were extracted from thermal images and compared with oral thermometry. Statistical analyses included calculation of receiver operating characteristic (ROC) curves and area under the curve (AUC) values for detection of febrile subjects.
Results: Pearson correlation coefficients for IRT-based and reference (oral) temperatures were found to vary strongly with measurement location. Approaches based on maximum temperatures in either inner canthi or full-face regions indicated stronger discrimination ability than maximum forehead temperature (AUC values of 0.95 to 0.97 versus 0.86 to 0.87, respectively) and other specific facial locations. These values are markedly better than the vast majority of results found in prior human studies of IRT-based fever screening.
Conclusion: Our findings provide clinical confirmation of the utility of consensus approaches for fever screening, including the use of inner canthi temperatures, while also indicating that full-face maximum temperatures may provide an effective alternate approach.
Journal Article
Novel Tools for Single Comparative and Unified Evaluation of Qualitative and Quantitative Bioassays: SS/PV–ROC and SS-J/PV-PSI Index–ROC Curves with Integrated Concentration Distributions, and SS-J/PV-PSI Index Cut-Off Diagrams
2024
Background: This investigation is both a study of potential non-invasive diagnostic approaches for the bladder cancer biomarker UBC® Rapid test and a study including novel comparative methods for bioassay evaluation and comparison that uses bladder cancer as a useful example. The objective of the paper is not to investigate specific data. It is used only for demonstration, partially to compare ROC methodologies and also to show how both sensitivity/specificity and predictive values can be used in clinical diagnostics and decision making. This study includes ROC curves with integrated cut-off distribution curves for a comparison of sensitivity/specificity (SS) and positive/negative predictive values (PPV/NPV or PV), as well as SS-J index/PV-PSI index–ROC curves and SS-J/PV-PSI index cut-off diagrams (J = Youden, PSI = Predictive Summary Index) for the unified direct comparison of SS-J/PV results achieved via quantitative and/or qualitative bioassays and an identification of optimal separate or unified index cut-off points. Patients and Methods: According to the routine diagnostics, there were 91 patients with confirmed bladder cancer and 1152 patients with no evidence of bladder cancer, leading to a prevalence value of 0.073. This study performed a quantitative investigation of used-up test cassettes from the visual UBC® Rapid qualitative point-of-care assay, which had already been applied in routine diagnostics. Using a photometric reader, quantitative data could also be obtained from the test line of the used cassettes. Interrelations between SS and PV values were evaluated using cumulative distribution analysis (CAD), SS/PV–ROC curves, SS-J/PV-PSI index–ROC curves, and the SS-J/PV-PSI index cut-off diagram. The maximum unified SS-J/PV-PSI index value and its corresponding cut-off value were determined and calculated with the SS-J/PV-PSI index cut-off diagram. Results: The use of SS/PV–ROC curves with integrated cut-off concentration distribution curves provides improved diagnostic information compared to “traditional” ROC curves. The threshold distributions integrated as curves into SS/PV–ROC curves and SS-J/PV-PSI index–ROC curves run in opposite directions. In contrast to the SS–ROC curves, the PV–ROC and the novel PV-PSI index–ROC curves had neither an area under the curve (AUC) nor a range from 0% to 100%. The cut-off level of the qualitative assay was 7.5 µg/L, with a sensitivity of 65.9% and a specificity of 63.3%, and the PPV was 12.4% and the NPV was 95.9%, at a threshold value of 12.5 µg/L. Based on these set concentrations, the reader-based evaluation revealed a graphically estimated 5% increase in sensitivity and a 13% increase in specificity, as compared to the visual qualitative POC test. In the case of predictive values, there was a gain of 8% for PPV and 10% for NPV. The index values and cut-offs were as follows: visual SS-J index, 0.328 and 35 µg/L; visual PV-PSI index, 0.083 and 5.4 µg/L; maximal reader Youden index, 0.0558 and 250 µg/L; and maximal PV-PSI index, 0.459 and 250 µg/L, respectively. The maximum unified SS-J/PV-PSI index value was 0.32, and the cut-off was 43 µg/L. The reciprocal SS-J index correctly detected one out of three patients, while the reciprocal PV-PSI index gave one out of twelve patients a correct diagnosis. Conclusions: ROC curves including cut-off distribution curves supplement the information lost in “traditionally plotted” ROC curves. The novel sets of ROC and index–ROC curves and the new SS/PV index cut-off diagrams enable the simultaneous comparison of sensitivity/specificity and predictive value profiles of diagnostic tools and the identification of optimal cut-off values at maximal index values, even in a unifying SS/PV approach. Because the curves within an SS-J/PV-PSI index cut-off diagram are distributed over the complete cut-off range of a quantitative assay, this field is open for special clinical considerations, with the need to vary the mentioned clinical diagnostic parameters. Complete or partial areas over the x-axis (AOX) can be calculated for summarized quantitative or qualitative effectivity evaluations with respect to single and/or unified SS-J and PV-PSI indices and with respect to single, several, or several unified assays. The SS-J/PV-PSI index-AOX approach is a new tool providing additional joint clinical information, and the reciprocal SS-J indices can predict the number of patients with a correct diagnosis and the number of persons who need to be examined in order to correctly predict a diagnosis of the disease. These methods could be used in applications like medical or plant epidemiology, machine learning algorithms, and neural networks.
Journal Article
Automatic detection of atrial fibrillation using stationary wavelet transform and support vector machine
by
Moussavi, Maryam
,
Asgari, Shadnaz
,
Mehrnia, Alireza
in
Accuracy
,
Algorithms
,
Arrhythmias, Cardiac - diagnosis
2015
Atrial fibrillation (AF) is the most common cardiac arrhythmia, and a major public health burden associated with significant morbidity and mortality. Automatic detection of AF could substantially help in early diagnosis, management and consequently prevention of the complications associated with chronic AF. In this paper, we propose a novel method for automatic AF detection.
Stationary wavelet transform and support vector machine have been employed to detect AF episodes. The proposed method eliminates the need for P-peak or R-Peak detection (a pre-processing step required by many existing algorithms), and hence its performance (sensitivity, specificity) does not depend on the performance of beat detection. The proposed method has been compared with those of the existing methods in terms of various measures including performance, transition time (detection delay associated with transitioning from a non-AF to AF episode), and computation time (using MIT-BIH Atrial Fibrillation database).
Results of a stratified 2-fold cross-validation reveals that the area under the Receiver Operative Characteristics (ROC) curve of the proposed method is 99.5%. Moreover, the method maintains its high accuracy regardless of the choice of the parameters׳ values and even for data segments as short as 10s. Using the optimal values of the parameters, the method achieves sensitivity and specificity of 97.0% and 97.1%, respectively.
The proposed AF detection method has high sensitivity and specificity, and holds several interesting properties which make it a suitable choice for practical applications.
•A novel method for automatic detection of Atrial fibrillation (AF) is proposed.•This method eliminates the need for P and/or R peak detection.•Sensitivity and specificity of the method is 97.0% and 97.1%, respectively.•It holds several interesting properties suitable for practical applications.
Journal Article
Real-time use of instantaneous wave–free ratio: Results of the ADVISE in-practice: An international, multicenter evaluation of instantaneous wave–free ratio in clinical practice
by
Mates, Martin
,
Piek, Jan
,
Haine, Steven
in
Aged
,
Area Under Curve
,
Cardiac Catheterization - methods
2014
To evaluate the first experience of real-time instantaneous wave–free ratio (iFR) measurement by clinicians.
The iFR is a new vasodilator-free index of coronary stenosis severity, calculated as a trans-lesion pressure ratio during a specific period of baseline diastole, when distal resistance is lowest and stable. Because all previous studies have calculated iFR offline, the feasibility of real-time iFR measurement has never been assessed.
Three hundred ninety-two stenoses with angiographically intermediate stenoses were included in this multicenter international analysis. Instantaneous wave–free ratio and fractional flow reserve (FFR) were performed in real time on commercially available consoles. The classification agreement of coronary stenoses between iFR and FFR was calculated.
Instantaneous wave–free ratio and FFR maintain a close level of diagnostic agreement when both are measured by clinicians in real time (for a clinical 0.80 FFR cutoff: area under the receiver operating characteristic curve [ROCAUC] 0.87, classification match 80%, and optimal iFR cutoff 0.90; for a ischemic 0.75 FFR cutoff: iFR ROCAUC 0.90, classification match 88%, and optimal iFR cutoff 0.85; if the FFR 0.75-0.80 gray zone is accounted for: ROCAUC 0.93, classification match 92%). When iFR and FFR are evaluated together in a hybrid decision-making strategy, 61% of the population is spared from vasodilator while maintaining a 94% overall agreement with FFR lesion classification.
When measured in real time, iFR maintains the close relationship to FFR reported in offline studies. These findings confirm the feasibility and reliability of real-time iFR calculation by clinicians.
[Display omitted]
Journal Article
Diagnosis of cirrhosis by transient elastography (FibroScan): a prospective study
2006
Background: Transient elastography (FibroScan) is a new, non-invasive, rapid, and reproducible method allowing evaluation of liver fibrosis by measurement of liver stiffness. In cirrhotic patients, liver stiffness measurements range from 12.5 to 75.5 kPa. However, the clinical relevance of these values is unknown. The aim of this prospective study was to evaluate the accuracy of liver stiffness measurement for the detection of cirrhosis in patients with chronic liver disease. Methods: A total of 711 patients with chronic liver disease were studied. Aetiologies of chronic liver diseases were hepatitis C virus or hepatitis B virus infection, alcohol, non-alcoholic steatohepatitis, other, or a combination of the above aetiologies. Liver fibrosis was evaluated according to the METAVIR score. Results: Stiffness was significantly correlated with fibrosis stage (r = 0.73, p<0.0001). Areas under the receiver operating characteristic curve (95% confidence interval) were 0.80 (0.75–0.84) for patients with significant fibrosis (F>2), 0.90 (0.86–0.93) for patients with severe fibrosis (F3), and 0.96 (0.94–0.98) for patients with cirrhosis. Using a cut off value of 17.6 kPa, patients with cirrhosis were detected with a positive predictive value and a negative predictive value (NPV) of 90%. Liver stiffness was significantly correlated with clinical, biological, and morphological parameters of liver disease. With an NPV >90%, the cut off values for the presence of oesophageal varices stage 2/3, cirrhosis Child-Pugh B or C, past history of ascites, hepatocellular carcinoma, and oesophageal bleeding were 27.5, 37.5, 49.1, 53.7, and 62.7 kPa, respectively. Conclusion: Transient elastography is a promising non-invasive method for detection of cirrhosis in patients with chronic liver disease. Its use for the follow up and management of these patients could be of great interest and should be evaluated further.
Journal Article
Distributed non-disclosive validation of predictive models by a modified ROC-GLM
by
Hoffmann, Verena S.
,
Bischl, Bernd
,
Rehms, Raphael
in
Algorithms
,
Area Under Curve
,
Area under the ROC curve
2024
Background
Distributed statistical analyses provide a promising approach for privacy protection when analyzing data distributed over several databases. Instead of directly operating on data, the analyst receives anonymous summary statistics, which are combined into an aggregated result. Further, in discrimination model (prognosis, diagnosis, etc.) development, it is key to evaluate a trained model w.r.t. to its prognostic or predictive performance on new independent data. For binary classification, quantifying discrimination uses the receiver operating characteristics (ROC) and its area under the curve (AUC) as aggregation measure. We are interested to calculate both as well as basic indicators of calibration-in-the-large for a binary classification task using a distributed and privacy-preserving approach.
Methods
We employ DataSHIELD as the technology to carry out distributed analyses, and we use a newly developed algorithm to validate the prediction score by conducting distributed and privacy-preserving ROC analysis. Calibration curves are constructed from mean values over sites. The determination of ROC and its AUC is based on a generalized linear model (GLM) approximation of the true ROC curve, the ROC-GLM, as well as on ideas of differential privacy (DP). DP adds noise (quantified by the
ℓ
2
sensitivity
Δ
2
(
f
^
)
) to the data and enables a global handling of placement numbers. The impact of DP parameters was studied by simulations.
Results
In our simulation scenario, the true and distributed AUC measures differ by
Δ
AUC
<
0.01
depending heavily on the choice of the differential privacy parameters. It is recommended to check the accuracy of the distributed AUC estimator in specific simulation scenarios along with a reasonable choice of DP parameters. Here, the accuracy of the distributed AUC estimator may be impaired by too much artificial noise added from DP.
Conclusions
The applicability of our algorithms depends on the
ℓ
2
sensitivity
Δ
2
(
f
^
)
of the underlying statistical/predictive model. The simulations carried out have shown that the approximation error is acceptable for the majority of simulated cases. For models with high
Δ
2
(
f
^
)
, the privacy parameters must be set accordingly higher to ensure sufficient privacy protection, which affects the approximation error. This work shows that complex measures, as the AUC, are applicable for validation in distributed setups while preserving an individual’s privacy.
Journal Article
Robustness of viral load over CD4 cell count in measuring the quality of life of people with HIV at second line regimen in Amhara region
2025
Access to medication for people living with HIV in need is a global health priority. Monitoring the quality of life of people under treatment helps to optimize the resource allocated for the program. Therefore, the main objective of the current study was to assess the quality of life comparing viral load and CD4 cell count and factors affecting the variable of interest among HIV-positive people under the second-line regimens. A hospital-based retrospective secondary data were used for the current investigation. The study was conducted in governmental hospitals in Amhara region. ROC curve was used for comparing CD4 cell count and viral load in assessing the quality of life and its predictors. Among the participants in second-line regimen, about 18.3% of them had CD4 cell count < 200 cells/ml of blood and 49.4% of them had detectable/unsuppressed viral load. The result of ROC curve in the current study indicates that viral load was in favors of CD4 cell count in this regard. Among the predictors for quality of life, age of patients, level of CD4 cell count while transferring to second-line regimen, sex of patients, social discrimination, level of education of patients, functional status, adherence level, disclosure status of the disease, mental depression, existence of opportunistic infections and residence area had significant effect on the variable of interest (quality of life). Viral load was in favor of CD4 cell count in assessing the quality of people under treatment. Hence, a close follow ups of patients under treatment at second-line regimen using viral load assessment is highly recommended. Due attention should be given to patients with Unsuppressed HIV viral loads. Hence, awareness creation on how the quality of life be improved should be formulated for patients during visiting times. Knowledge of HIV transmission is also important to reduce the violence and discrimination against those HIV-positive adults to improve their health status. Experience sharing between medication adherent and non-adherent patients may encourage those non-adherent patients to get lessons from adherent patients.
Journal Article