1 Introduction

The ROCpAI package allows to calculate the area under the ROC curve (AUC) and the partial area under the ROC curve (pAUC) as well as partial area indexes related with the pAUC over a restricted false positive interval of interest even if the ROC curve is improper.

To do this, the package calculates the specificty and sensibility from a classification model with the function points.curve, after that, it calculates the partia area and two standardized: the McClish index (McClish 1989) with the mcpAUC if the ROC curve is proper over the restricted interval, and the tpAUC index according for any shape of the ROC curve over such interval, even if it is improper. Moreover,the functions tpAUCboot and mcpAUCboot apply bootstrap resampling (Davison and Hinkley 1997) to calculate the empiral standard deviation and the confidence interval of these standardized indexes.

1.1 Installation

The installation of ROCpAI package is performed via Github:

devtools::install_github("juanpegarcia/ROCpAI")

1.2 Prerequisites

The package ROCpAI depends on the following Bioconductor package: SummarizedExperiment (Morgan et al. 2019) and boot (Angelo and Ripley 2019),

2 Using ROCpAI

The fission dataset (Leong et al. 2014) contains the data of 36 samples of yeast and 7039 genes in a RangedSummarizedExperiment object type. These 36 samples are divided in 18 native samples and 18 mutated samples.

Data.frame example

genes 
##            Strain Gene1 Gene2 Gene3 Gene4 Gene5
## GSM1368273      1   105     0    12    95    37
## GSM1368274      1    35     0     3    29     5
## GSM1368275      1    35     2     2    67    21
## GSM1368276      1    21     0     0    40    27
## GSM1368277      1    36     0     0    44     6
## GSM1368278      1    14     0     6    27     4
## GSM1368279      1    41     0     7    60    33
## GSM1368280      1    24     0     5    51    32
## GSM1368281      1    34    40    11    39    54
## GSM1368282      1    15     0     2    25    10
## GSM1368283      1    86     1    15   103    32
## GSM1368284      1    38    15     7    57    23
## GSM1368285      1    50     0    10    65    25
## GSM1368286      1    57     1     1    32    10
## GSM1368287      1    16     4     3    25    18
## GSM1368288      1    62     0     2    32    25
## GSM1368289      1    57     0     8    43    29
## GSM1368290      1    36     0     4    28    17
## GSM1368291      2    41     0    13   106    19
## GSM1368292      2    84     1     6   190    43
## GSM1368293      2    95     6    16   215    92
## GSM1368294      2    48     6    21    74    38
## GSM1368295      2    54     5    15   155    25
## GSM1368296      2    47     4     8    56    33
## GSM1368297      2    59    30    10   162   139
## GSM1368298      2    66    12    24    92    94
## GSM1368299      2    42    25    18   107   115
## GSM1368300      2    64     4    14   133    82
## GSM1368301      2    72    18    10   112    81
## GSM1368302      2    62     9     9    97    59
## GSM1368303      2    44     4    14    87    44
## GSM1368304      2    60     7    13    69    68
## GSM1368305      2    46     9    17    78    62
## GSM1368306      2    46     7    18    70    51
## GSM1368307      2    48     4     7    58    33
## GSM1368308      2    17     1     8    48    29

3 Functions:

3.1 PointCurves

This function generates the points of the ROC curve given by the pairs of the true positive rate (sensitibity) and false positive rate (1-specificity) for each possible cut-off point. pointsCurve needs 2 parameters: x and y. The first parameter correspond to a vector with the cases’ tags, while the second one is a vector with the value of each case. This function returns a matrix with the sensibility and specifity.

pointsCurve(genes[,1], genes[,2])
##        fpr.point  sen.point
##  [1,] 0.00000000 0.00000000
##  [2,] 0.05555556 0.00000000
##  [3,] 0.05555556 0.05555556
##  [4,] 0.11111111 0.05555556
##  [5,] 0.11111111 0.11111111
##  [6,] 0.11111111 0.16666667
##  [7,] 0.11111111 0.22222222
##  [8,] 0.11111111 0.27777778
##  [9,] 0.11111111 0.27777778
## [10,] 0.16666667 0.33333333
## [11,] 0.16666667 0.38888889
## [12,] 0.16666667 0.44444444
## [13,] 0.16666667 0.44444444
## [14,] 0.27777778 0.44444444
## [15,] 0.27777778 0.50000000
## [16,] 0.33333333 0.50000000
## [17,] 0.33333333 0.50000000
## [18,] 0.33333333 0.61111111
## [19,] 0.33333333 0.66666667
## [20,] 0.33333333 0.66666667
## [21,] 0.33333333 0.77777778
## [22,] 0.33333333 0.83333333
## [23,] 0.33333333 0.88888889
## [24,] 0.33333333 0.88888889
## [25,] 0.38888889 0.94444444
## [26,] 0.44444444 0.94444444
## [27,] 0.44444444 0.94444444
## [28,] 0.55555556 0.94444444
## [29,] 0.55555556 0.94444444
## [30,] 0.66666667 0.94444444
## [31,] 0.72222222 0.94444444
## [32,] 0.77777778 0.94444444
## [33,] 0.83333333 0.94444444
## [34,] 0.83333333 1.00000000
## [35,] 0.88888889 1.00000000
## [36,] 0.94444444 1.00000000
## [37,] 1.00000000 1.00000000

3.2 mcpAUC

This function calculates the pAUC its standardised partial area index given by (McClish 1989). This function has five parameters. The first one is a data.frame or a RangedSummarizedExperiment object, if it is a data.frame object, the “Gold Standard” must be in the first column. The other four parameters are optionals; low.value and up.value are the false positive rate values that the function will use to calculate the pAUC. If low.value is NULL the function will take 0 as the lower limit. If up.value is NULL the function will use 1 as the upper value. The parameter plot generates a graph with the ROC curves generated. The last parameter is called “selection” and is only used if the parameter “dataset” is a RangedSummarizedExperiment object. This parameter is used to select the variables that will be analysed. If this parameter is NULL the function will analyse all the variables in the dataset. The function returns as RangedSummarizedExperiment object with the pAUC and the mcpAUC scores,and the TPR and FPR values for each ROC curve generated.

resultMc <- mcpAUC(genes, low.value = 0, up.value = 0.25, plot=TRUE)
resultMc
## class: SummarizedExperiment 
## dim: 4 1 
## metadata(0):
## assays(1): ''
## rownames(4): St_pAUC pAUC Sensitivity FPR
## rowData names(1): metrics
## colnames: NULL
## colData names(0):

test.Mc<- assay(resultMc)
test.Mc$St_pAUC
## [[1]]
## [1] NA
## 
## [[2]]
## [1] NA
## 
## [[3]]
## [1] 0.7654321
## 
## [[4]]
## [1] 0.8130511
## 
## [[5]]
## [1] 0.8412698
test.Mc$pAUC
## [[1]]
## [1] 0.05709877
## 
## [[2]]
## [1] 0.119213
## 
## [[3]]
## [1] 0.1473765
## 
## [[4]]
## [1] 0.1682099
## 
## [[5]]
## [1] 0.1805556

3.3 tpAUC

This function calculates the pAUC and its standardised pAUC by the tighter partial area index (tpAUC) (Vivo, Franco, and Vicari 2018). This function has five parameters. The first one is a data.frame or a RangedSummarizedExperiment object, if it is a data.frame object the “Gold Standard” must be in the first column. The other four parameters are optionals; low.value and up.value are the false positive rate values that the function will use to calculate the pAUC. If low. value is NULL the function will take 0 as the lower limit. If up.value is NULL the function will use 1 as the upper value. The parameter plot generates a graph for each ROC curve generated. The last parameter is called “selection” and is only used if the parameter “dataset” is a RangedSummarizedExperiment object. This parameter is used to select the variables that will be analysed. If this parameter is NULL the function will do a ROC curve with all the variables in the object. The function returns as RangedSummarizedExperiment object with the pAUC and the tpAUC scores,and the TPR and FPR values for each ROC curve generated

resultsT <- tpAUC(genes, low.value = 0, up.value = 0.25, plot=TRUE)
resultsT
## class: SummarizedExperiment 
## dim: 4 1 
## metadata(0):
## assays(1): ''
## rownames(4): St_pAUC pAUC Sensitivity FPR
## rowData names(1): metrics
## colnames: NULL
## colData names(0):