The Bioconductor package repositories may be mirrored with rsync
.
If you would like to become a mirror for package and data package
repositories, please use the commands below.
We have increased security requirements for the public Bioconductor
mirror sites. A mirror is considered “public” if it’s an option in R’s
chooseBioCmirror()
function and listed on our mirror
page. Public site maintainers must support https on their
site and use secure rsync when retrieving packages from the master. If you are
interested in hosting a publicly available mirror site, please send a suitable
public key to
webmaster@bioconductor.org.
Once your key is added to the bioc-rsync
account you can rsync
from
master.bioconductor.org as this user. You can use a config file to manage your
ssh keys or include the full path to your key in the -e
statement:
rsync -e "ssh -i path/to/your/key" bioc-rsync@master.bioconductor.org ...
To host a non-secure mirror (maybe for internal purposes), invoke
the rsync commands without the bioc-rsync
user, e.g.,
rsync -avn master.bioconductor.org::release ## check size
rsync -zrtlv --delete master.bioconductor.org::release /dest/packages/release
rsync -zrtlv --delete master.bioconductor.org::devel/bioc /dest/packages/devel etc ...
If you want to mirror the current Bioconductor release version (currently 3.12), use the following commands:
Pick a destination directory where files will be mirrored. Let’s say
this will be in /dest
. This directory should be served by your web
server. Under that you’ll need a directory called packages
. This
directory must be present as it is part of the structure of a
Bioconductor repository. Underneath packages
should be a directory
corresponding to the versions of Bioconductor that you will host. The
current release version is 3.12 and the
current devel version is 3.13. We recommend
you use symlinks called release
and devel
that always point to the
current release and devel versions; this way you will never have to
change your rsync commands. But you should change the symlink
targets with every Bioconductor release (see the
release schedule for exact dates).
The following
commands will create the directory structure you’ll need (remember
that /dest
is just an example of the destination directory
you could use; you can put this directory anywhere on your system
where there is enough free space).
mkdir -p /dest/packages
mkdir /dest/packages/3.12 # current release
mkdir /dest/packages/3.13 # current devel
ln -s /dest/packages/3.12 /dest/packages/release # change these links
ln -s /dest/packages/3.12 /dest/packages/devel # every 6 months (with Bioc release)
rsync -e "ssh" -zrtlv --delete bioc-rsync@master.bioconductor.org:release /dest/packages/release
rsync -e "ssh" -zrtlv --delete bioc-rsync@master.bioconductor.org:release/bioc /dest/packages/release/bioc
rsync -e "ssh" -zrtlv --delete bioc-rsync@master.bioconductor.org:release/data /dest/packages/release/data
If you want to mirror the Bioconductor devel repos (currently 3.13), please use the following commands:
rsync -e "ssh" -zrtlv --delete bioc-rsync@master.bioconductor.org:devel /dest/packages/devel
rsync -e "ssh" -zrtlv --delete bioc-rsync@master.bioconductor.org:devel/bioc /dest/packages/devel/bioc
rsync -e "ssh" -zrtlv --delete bioc-rsync@master.bioconductor.org:devel/data /dest/packages/devel/data
Make sure the directory above packages
is served by a web server.
Bioconductor is big (> 188GB for BioC 3.13).
Please check the size of what will be transferred with e.g. rsync -e "ssh"
-avn bioc-rsync@master.bioconductor.org:release
and make sure you have enough
room on your local disk before you start.
It is recommended that package repositories be synced once per day, scheduled with cron.
Begin using your new local repository by making it accessible on your webserver. See the “contriburl” option to install.packages() (utils) for more information.
The Bioconductor master package repositories reside at https://master.bioconductor.org in the Amazon cloud.