ESG mirror for input data


Posted by Lila Warszawski on Aug. 2, 2012

A group at Argonne National Lab in Chicago have kindly set up a mirror of the ISI-MIP input-data archive on their Earth Systems Grid node. This will help to further improve bandwidth and data access, and we encourage you to consider this mirror site as a second option for input data downloads, besides the DKRZ server in Hamburg. Below are instructions on how to access the mirror:

"For now, access to the mirror is only through GridFTP using a tool called Globus Online (GO). Its easy to set up and provides for much faster downloads then what you'll typically get from html. In future, we will also publish the mirror to the ESG portal so that there will also be the standard ESG options for download like html and wget etc.

1. Pre-requisites

a. The ISIMIP Argonne archive password has been provided to the user

username: isimipuser
password: ###

b. User has a Globus Online account, https://www.globusonline.org/SignUp

2. To download using website:

a. Log into Globus Online, Start Transfer https://www.globusonline.org/xfer/StartTransfer
b. In the column on the left, choose isimipgo#us-archive as source (ISIMIP Argonne archive)
c. Hit go, and a box will popup to provide the username and password (see above).
d. On the right side, choose your destination, if you're downloading to a local machine, choose to install Globus Connect by clicking "Get Globus Connect" and following the instructions. If you're working on a network machine, you will probably need to ask your admin person to install the endpoint for you.
e. Browse through the structure of ISI-MIP files and select the files you want, then browse through the folders on your computer to select the destination folder. Click the big arrow to  request transfer.
f. Once the transfer is requested, go to the 'view transfer activity' tab to check progress. Once the transfer completes (or if there are any issues) you will receive email notification.

3. To download using CLI:

a. Configure Globus Online account for CLI use, see https://support.globusonline.org/entries/20979751-how-do-i-add-an-ssh-key-to-my-globus-online-account
b. Activate isimip-archive endpoint

echo ### | ssh -t GO-USERNAME@cli.globusonline.org endpoint-activate -U isimipuser isimipgo#us-archive

c. Start desired download transfer, see https://www.globusonline.org/usingcli/ for guide on how to use CLI.

The most frequent issue that you might face on this is a firewall blocking GO from making the transfer. Its an easy issue to fix and here's the FAQ that explains which ports need to be open: https://support.globusonline.org/entries/20999718-what-ports-does-globus-connect-need-open

If you are seeing issues despite the requirements for firewall being met, contact Joshua Elliott and I'll have support look at it. Or you can use https://support.globusonline.org/anonymous_requests/new for your specific issue, and we'll track it."