In this document we illustrate some approaches to
working with UK Biobank summary statistics.
Be sure that that the python module ukbb_pan_ancestry
has been installed where reticulate can find it.
(We don’t use basilisk as of 12/24/2022 because of
issues in the terra spark cluster.)
If the above command indicates that BiocHail is not available, see the Installation section near the end of this document.
We have produced a representation of summary statistics for a sample of 9888 loci. This 5GB resource can be retrieved and cached with the following code:
hl = hail_init()
ss = get_ukbb_sumstat_10kloci_mt(hl) # can take about a minute to unzip 5GB
ss$count() # but if a persistent MatrixTable is at the location given
# by env var HAIL_UKBB_SUMSTAT_10K_PATH it goes quickly
To get a description of available content, we need a python chunk:
Here’s a basic description of the summary stats table, with code that works in terra.bio:
hl = bare_hail()
hl$init(idempotent=TRUE, spark_conf=list(
'spark.hadoop.fs.gs.requester.pays.mode'= 'CUSTOM',
'spark.hadoop.fs.gs.requester.pays.buckets'= 'ukb-diverse-pops-public',
'spark.hadoop.fs.gs.requester.pays.project.id'= Sys.getenv("GOOGLE_PROJECT")))
We need to use a python chunk to get the output, using gs:// storage references.
We’ll collect the column metadata to start to understand details of annotation of phenotypes.
sscol = ss$cols()$collect() # OK because we are just working over column metadata
ss1 = sscol[[1]]
names(ss1)
ss1$get("phenocode")
ss1$get("description")
We’ve introduced a function that extracts selected fields for a given phenotype, that accommodates availability of results for specific populations.
This can be iterated over all the elements of sscol
to produce
a comprehensive searchable table. Here’s a small illustration:
We’ll trim the material to be worked with by sampling both rows and columns. (2023.01.08: In future revisions we will be able to control the seed for random sampling.)
Row metadata are simple to collect:
The summary statistics themselves reside in entries of the MatrixTable. This can be expensive to collect and so filtering methods beyond random sampling must be mastered. But here is a basic view.
The summary_stats
component has the association p-values – log10 transformed?
BiocHail
BiocHail
should be installed as follows:
## R version 4.4.0 RC (2024-04-16 r86468)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 22.04.4 LTS
##
## Matrix products: default
## BLAS: /home/biocbuild/bbs-3.20-bioc/R/lib/libRblas.so
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_GB LC_COLLATE=C
## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## time zone: America/New_York
## tzcode source: system (glibc)
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] ggplot2_3.5.1 BiocHail_1.5.0 BiocStyle_2.33.0
##
## loaded via a namespace (and not attached):
## [1] sass_0.4.9 utf8_1.2.4 generics_0.1.3
## [4] RSQLite_2.3.6 lattice_0.22-6 digest_0.6.35
## [7] magrittr_2.0.3 evaluate_0.23 grid_4.4.0
## [10] bookdown_0.39 fastmap_1.1.1 blob_1.2.4
## [13] jsonlite_1.8.8 Matrix_1.7-0 DBI_1.2.2
## [16] tinytex_0.50 BiocManager_1.30.22 httr_1.4.7
## [19] fansi_1.0.6 scales_1.3.0 jquerylib_0.1.4
## [22] cli_3.6.2 rlang_1.1.3 dbplyr_2.5.0
## [25] basilisk.utils_1.17.0 munsell_0.5.1 bit64_4.0.5
## [28] withr_3.0.0 cachem_1.0.8 yaml_2.3.8
## [31] tools_4.4.0 dir.expiry_1.13.0 parallel_4.4.0
## [34] memoise_2.0.1 dplyr_1.1.4 colorspace_2.1-0
## [37] filelock_1.0.3 basilisk_1.17.0 BiocGenerics_0.51.0
## [40] curl_5.2.1 reticulate_1.36.1 png_0.1-8
## [43] vctrs_0.6.5 R6_2.5.1 magick_2.8.3
## [46] BiocFileCache_2.13.0 lifecycle_1.0.4 bit_4.0.5
## [49] pkgconfig_2.0.3 gtable_0.3.5 pillar_1.9.0
## [52] bslib_0.7.0 Rcpp_1.0.12 glue_1.7.0
## [55] highr_0.10 xfun_0.43 tibble_3.2.1
## [58] tidyselect_1.2.1 knitr_1.46 htmltools_0.5.8.1
## [61] rmarkdown_2.26 compiler_4.4.0