All data uploaded will be quality checked by the ISIMIP data managers. The goal is to have all runs available with a consistent set of variables.
If you are not sure about something, you can upload just a small subset to _tmp/ below your model folder and let us know at isimip-data@pik-potsdam.de.
On request we offer consistency checks for your model.
Checking process
- Files in UploadArea will automatically get moved to a private folder 24 hrs after upload if they meet a basic naming scheme. Please refrain from re-uploading identical files that have already been copied to the DKRZ.
- Checks identify severe and fixable errors, correct files without severe errors and produce a report of the checks. Summary of checks:
- NetCDF format: NetCDF4 Classic ZIP and .nc4 extension
- File name: lower case, correct model and scenario names (severe), gcm/climate_driver/social_scenario/co2_scenario combinations according to protocol
- Variable: same file name and NetCDF variable name (severe), units according to protocol, only one variable per submitted file (severe; with some exceptions)
- Grid: names and units of grid coordinates, grid type, grid increments, grid mask (severe), missing_value, _FillValue, _ChunkSizes (only for global data)
- Time axis: name of time dimension (severe), time units, time increment (severe), reference date, number of time steps (severe), start and end year in agreement to simulation period and file name (severe)
- Global metadata: contact and institution, and metadata for variables with varying depth layers (when it applies)
- Log files are written to the DKRZ folder: /work/bb0820/ISIMIP/[SIMULATION-ROUND]/UploadArea/[SECTOR]/[MODEL]/_qc_reports/ . Non severe issues found (without “!!!”) will be fixed by the data managers. All others will need your assistance. Please check this folder on a regular basis and get in touch with Iliusi Vega or Matthias Büchner to discuss fixes on the files.
- We will check for any gcm/climate_driver/social_scenario/co2_scenario combination that has been uploaded and successfully passed the format checks. It then internally generates a list off any variable provided and afterwards goes through all the combinations found and checks if all those variables are there.
- Files that successfully pass the Quality Check will appear in the folder: /work/bb0820/ISIMIP/[SIMULATION-ROUND]/OutputData available for analysis by ISIMIP participants.
Quick formatting fixes
Some simple issues issues in your files –like wrong chunk sizes, wrong NetCDF format, inverted grid, or wrong variable and dimension names–, can be solved with the following fixes. For further instructions, please give a look at the NetCDF utilities mentioned above.
Chunk sizes
If your file does not have correct chunk sizes, try rewriting the data with: