An Asymptotic Multivariate Test for Testing the Equality of the Average Areas Under Independent Receiver Operating Characteristic Curves - Simulation Study and an Application
[1]
Marina Roshini Sooriyarachchi, Department of Statistics, University of Colombo, Colombo, Sri Lanka.
[2]
Nuzhi Ahmed Meyen, Department of Statistics, University of Colombo, Colombo, Sri Lanka.
The Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC) Curve has become a popular summary measure of the curve. In a previous paper, the authors proposed an asymptotic bivariate test for comparing AUCs for paired data. In this case, the test statistic derived was found to follow a distribution proportional to the Beta distribution. This test can also be applied to the multivariate case for independent data as shown in this paper. The properties of the developed test are examined by using simulation studies for the scenario of multivariate independent ROC curves. The general method is illustrated for this case by applying it to a published data set in the Rockit manual. The simulation studies found that the developed test has good properties for large samples.
Multivariate Test, Receiving Operating Characteristic (ROC) Curve, Area Under the Curve (AUC), Beta Distribution, Maximum Likelihood Estimates, Simulation
[1]
Metz CE. Basic principles of ROC analysis. Seminars in Nuclear Medicine. 1978; 8 (4), 283-298.
[2]
Schutts, Joshua William, "The Use of Receiver Operating Characteristic Curve Analysis for Academic Progress and Degree Completion" (2016). Dissertations. 357.
[3]
Pepe M S. The statistical evaluation of medical Tests for classification and prediction. New York: Oxford University Press; 2003.
[4]
Krzanowski WJ, Hand DJ. ROC curves for continuous data. CRC/Chapman and Hall; 2009.
[5]
Grégoire Thomas, Louise C. Kenny, Philip N. Baker and Robin Tuytten. A novel method for interrogating receiver operating characteristic curves for assessing prognostic tests. Diagnostic and Prognostic Research (2017) 1: 17.
[6]
Hanley JA, McNeil BJ. The meaning and Use of the Area under a Receiver Operating Characteristic (ROC) curve. Radiology 1982; 143 (1): 29-36.
[7]
DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometric. 1988, 44 (3), 837-45.
[8]
Cleves MA. Comparing Areas Under Receiver Operating Characteristics Curves from Two or More Probit or Logit Models. The STATA Journal, 2002; 2: 301-31.
[9]
Hanley JA, McNeil BJ. A method of comparing the Areas under Receiver Operating Characteristic Curves Derived from the same cases. Radiology 1983; 148 (3): 839-843.
[10]
Dorfman DD, Alf E. Maximum likelihood estimation of parameters of signal detection theory and determination of Confidence intervals-rating method data. Journal of Mathematical Psychology 1969; 6 (3): 487-496.
[11]
. Park SH, Goo JM, Jo C. Receiver Operating Characteristic (ROC) curve: Practical review for radiologists. Korean Journal of Radiology. 2004; 5 (1): 11-18.
[12]
Vergara IA, Normbuena T, Ferrada E, Slater AW, Melo F. StAR: a simple tool for the statistical comparison of ROC curves. BMC Bioinformatics. 2008; 9: 265 Open Access.
[13]
Pablo Martı´nez-Camblor. Fully non-parametric receiver operating characteristic curve estimation for random-effects meta-analysis. Statistical Methods in Medical Research 2017, Vol. 26 (1) 5–20.
[14]
Sonia Perez Fernandez (2017). Non-Standard ROC Curve Analysis. Package ‘nsROC’. Cran package in R.
[15]
Honghu-Liu, Li G, William G. Cumberland and Tongtong Wu. Testing Statistical Significance of the Area under a Receiving Operating Characteristics Curve for Repeated Measures Design with Bootstrapping Journal of Data Science 3 (2005), 257-278.
[16]
Krzysztof Gajowniczek, Tomasz Ząbkowski, Ryszard Szupiluk (2014). ESTIMATING THE ROC CURVE AND ITS SIGNIFICANCE FOR CLASSIFICATION MODELS’ ASSESSMENT. QUANTITATIVE METHODS IN ECONOMICS Vol. XV, No. 2, 2014, pp. 382-391.
[17]
Hanley JA. The robustness of the “binormal” assumptions used in fitting ROC curves. Medical Decision Making. 1988; 8 (3), 197-203.
[18]
Pepe, M., Longton, G., & Janes, H. (2009). Estimation and Comparison of Receiver Operating Characteristic Curves. The Stata Journal, 9 (1), 1.
[19]
. Metz CE, Herman BA, Roe CA. Statistical Comparison of Two ROC-curve estimates obtained from partially-paired datasets. Medical Decision Making 1998a; 18 (1): 110-121.
[20]
. Hall P, Zhou X. Nonparametric Estimation of Component Distributions in a Multivariate Mixture: The Annals of Statistics. 2003: 31 (1) 201-224.
[21]
Bamber D. The Area above the Ordinal Dominance Graph and the Area below the Receiver Operating Characteristic Graph. J. Math Psychol, 1975; 12 (4): 387-415.
[22]
. Grey, D. M. & Morgan, B. T. “ Some aspects of ROC curve fitting: Normal and Logistic models.”Journal of Mathematical Psychology, Volume 9, pp. 128-139, 1972.
[23]
Dorfman, D. D. & Berbaum, K. S. “Degeneracy and discrete reciever operating characteristic rating data.” Academic Radiology, 2 (10), pp. 907-915, 1995.
[24]
Liu C, Dobson J, Cawley P. 2017 Efficient generation of receiver operating characteristics for the evaluation of damage detection in practical structural health monitoring applications. Proc. R. Soc. A 473: 20160736.
[25]
Casella, G & Berger, R. L. “Statistical Inference.” Duxbury Press. Second Edition, 2002.
[26]
Hotelling H. Multivariate Quality Control. In C. Eisenhart, M. W. Hastay and W. A. Wallis, eds. Techniques of Statistical Analysis. New York: McGraw-Hill, 1947.
[27]
Williams JD, Woodall WH, Birch JB, Sullivan JH. Distribution of Hotelling’s T-squared Statistic Based on the Successive Differences Estimator. Journal of Quality Technology.
[28]
Wilks SS. Multivariate Statistical outliers. Sankhya (Indian Statistical Journal) 1963.
[29]
Gnanadesikan R, Kettering JR. Robust Estimates, Residuals and Outlier detection with Multi-response data. Biometrics 1972; 28: 81-124.
[30]
Mardia KV, Kent JT, Bibby JM. Multivariate Analysis, Academic Press, 1979.
[31]
Seneratna, D., Sooriyarachchi, M. R. and Meyan, N. Bivariate Test for Testing the EQUALITY of the Average Areas under Correlated Receiver Operating Characteristic Curves (Test for Comparing of AUC’s of Correlated ROC Curves). American Journal of Applied Mathematics and Statistics, 2015, Vol. 3, No. 5, 190-198.
[32]
Hotelling H, Frankel LR. The transformation of Statistics to simplify their Distribution. Annals of Mathematical Statistics, 1938; 9 (2): 87-96.
[33]
Wallace DL. Asymptotic Approximations to Distributions, Annals of Mathematical Statistics, 1958; 29 (3): 635-654.
[34]
Cleeves, A. M. (2002). Comparative assessment of three common algorithms for estimating the variance of the area under the nonparametric receiver operating characteristic curve. The Stata Journal, 2 (3), 280-289.
[35]
Hosmer DW, Lemeshow S. Applied logistic regression: Wiley Series in probability and statistics. 2000.
[36]
Agresti A. An introduction to categorical data analysis: Wiley-Interscience. 2007.
