Uploading your data


This information is relevant to the simulation round ISIMIP2a.

Simulation results should be uploaded to the DKRZ server (the same server where the input data are hosted). Each model has its own folder structure. 

The base path on vre2.dkrz.de is /work/bb0820/data/

Details about file formatting and meta data requirements can be found in the simulation protocol.  For example, remember to put your institution and contact as global meta data into the compressed NetCDF4 files. Also the specification of the several scenarios (harmonization, sensitivity, irrigation, co2, [where applied]) is crucial to clearly identify how your runs have been initialized. Splitting daily and monthly time series as described in the protocol also seems to be a common source of incorrect file formatting. If you are not sure about anything, you can upload just a small subset to _tmp/  below your model folder and let Matthias Büchner have a look at it before pushing the rest of your data.

Upload Area

upload_area_ISIMIP2.1/<sector>/<model>/_tmp/<simulation_type>/<GCM,OBS_dataset>/

Please note, you can only upload your data to the <MODEL>/_tmp/ structure. Data that have already been uploaded will be moved to _tmp with permission to alter files by every member of the user group isimip. Please do not move any data on your own. You no longer have write permissions for any folders other than the _tmp folder. A hidden copy of these files will be kept to avoid unintentional removal of the data in case of errors in the quality control (QC)  process.

The newly-implemented QC is integrated into the transfer of data from the _tmp structure to the main structure. The QC covers basic checks on file naming and NetCDF meta data and grid configuration. Minor issues will be fixed by us. _tmp files that have passed the QC will appear in the main structure (shown in blue below). For each file where major issues have been found, and where your assistance is needed, a corresponding QC log file will be written in a similar structure below _tmp/_qc_reports/. We will also inform you via email. Until these issues are fixed, your data will not appear in the main structure. The QC is regularly repeated regularly, ensuring that corrected files are transferred promptly to the main structure.

The following name space is prepared:

    • sector:
      water_regional
      water_global
      biomes
      forestry
      agriculture
      agro-eco
      permafrost
      marine-fishery
      energy
      infrastructure
      health
      biodiversity
      cge
    • model: if the sub structure is not created for your model yet, please contact Matthias Büchner
    • simulation_type:
      hist-obs (historical evaluation runs)
      catch-up (scenario runs with Fast Track input)
      cross-sec (cross-sectoral interaction runs with newer bias corrected GCM scenarios)
    • GCM (catch-up)
      hadgem2-es
      ipsl-cm5a-lr
      gfdl-esm2m
      miroc-esm-chem
      noresm1-m
    • GCM (cross-sec)
      hadgem2-es
      ipsl-cm5a-lr
    • OBS_dataset
      gswp3
      princeton (PGFv2)
      watch (WFD)
      wfdei (WFD+WFDEI)

Folder tree

/work/bb0820/data/upload_area_ISIMIP2.1/[SECTOR]/[MODEL]/
|-- _code
|-- _doc
|-- _tmp
|   |-- catch-up
|   |   |-- gfdl-esm2m
|   |   |-- hadgem2-es
|   |   |-- ipsl-cm5a-lr
|   |   |-- miroc-esm-chem
|   |   `-- noresm1-m
|   |-- cross-sec
|   |   |-- hadgem2-es
|   |   `-- ipsl-cm5a-lr
|   `-- hist-obs
|   |   |-- gswp3
|   |   |-- princeton
|   |   |-- watch
|   |   `-- wfdei
|   `-- _qc_reports
|       |-- catch-up ...

|       |-- cross-sec ...

|       `-- hist-obs ...
|-- catch-up
|   |-- gfdl-esm2m
|   |-- hadgem2-es
|   |-- ipsl-cm5a-lr
|   |-- miroc-esm-chem
|   `-- noresm1-m
|-- cross-sec
|   |-- hadgem2-es
|   `-- ipsl-cm5a-lr
`-- hist-obs
    |-- gswp3
    |-- princeton
    |-- watch
    `-- wfdei