Accessing ISIMIP data on the DKRZ server


Modellers actively participating in ISIMIP need a user account at DKRZ to access the ISIMIP input data and upload and access output data.


Creating a DKRZ account to access ISIMIP data

An individual username and password can be obtained using these steps:

  • Fill out this form here.
  • Click on the link in the email you receive and click on 'log in' to join the ISIMIP project.
  • Log in using your email address and the password you created.
  • Select 'Projects' > 'join existing project'
  • Enter project number 820, shortly describe your institutional background, your context for using the ISIMIP data and send.
  • The project managers will approve your request as soon as possible (normally within 2-3 working days).
  • You will receive a confirmation email including your username (something like b383300 - not your email address).
  • You now have an account for the DKRZ server (and other services) where the ISIMIP input data are stored.

Storage resources at DKRZ

By creating a DKRZ account, you will be granted access to the Mistral file system, which offers three different storage spaces:

  • HOME – 24 GB of personal backed up storage space
  • WORK – the ISIMIP project storage space – use only the UploadArea within this directory; any other use should be agreed with the ISIMIP Data Management Team beforehand
  • SCRATCH – 15 TB of personal temporary storage space, automatically deleted after 14 days, which you can use to process large data sets

Further info about the file systems is available here.

Logging in to the DKRZ server

ISIMIP data at the DKRZ is preferably accessible via this server:

  • mistralpp.dkrz.de - a physical cluster running Linux for file transfers and post-processing.

Before using the system please visit the DKRZ user portal for more details.

Connecting and transferring data to and from these servers is only possible using rsync, ssh, sftp and scp - not http. For example:

via a terminal:

ssh [YOUR-USERNAME]@mistralpp.dkrz.de

using WinSCP:

File Protocol: SFTP
Host name: mistralpp.dkrz.de 
Port number: 22
User name: [YOUR-USERNAME]

Directory structure

The ISIMIP data are located in the folder to /work/bb0820/ISIMIP/.

Data are then organised according to the simulation round. For each simulation round there are folders for:

  • InputData. All data necessary to run your ISIMIP simulations organised into folders according to the type of data.
  • UploadArea. This is where you should place your simulation data organised into sectors, models and time frames covered. Take care that only data belonging to a certain period is uploaded in the periods folders accordingly. Data will be moved from this folder on a daily basis for the purposes of the quality check. You cannot and should not download simulation data from other models from this location. Please be sure to read these important instructions about preparing your simulation files.
  • OutputData. This is where you will find simulation data that has passed the quality check. If, for whatever reason, data in OutputData is replaced or removed, these changes will be documented in the Output Data Change Log. It is your responsibility to check this log before making official use of the output data (e.g. in a paper or conference presentation), to ensure you have the latest data.
  • DerivedOutputData. Here you will find data products derived from ISIMIP simulation data, available for you to use.

Examples

You want to download the land-only, bias-corrected, atmospheric data for ISIMIP2b for the GCM IPSL-CM5A-LR for the RCP2.6 scenario?

scp -r [YOUR-USERNAME]@mistralpp.dkrz.de:/work/bb0820/ISIMIP/ISIMIP2b/InputData/GCM_atmosphere/biascorrected/landonly/rcp26/IPSL-CM5A-LR [LOCAL-DESTINATION-PATH]

or

rsync -auv [YOUR-USERNAME]@mistralpp.dkrz.de:/work/bb0820/ISIMIP/ISIMIP2b/InputData/GCM_atmosphere/biascorrected/landonly/rcp26/IPSL-CM5A-LR [LOCAL-DESTINATION-PATH]

You want to upload your simulations?

You will need to prepare your files according to these instructions, and place the files in the appropriate folders.

scp [LOCAL-PATH-TO-DATA]  [YOUR-USERNAME]@mistralpp.dkrz.de:/work/bb0820/ISIMIP/ISIMIP2b/UploadArea/[YOUR-SECTOR]/[YOUR-MODEL]/_tmp/[CLIMATE-FORCING-DATASET]/[PERIOD]/