URGENT ACTION NEEDED: ISIMIP DKRZ project


Posted by Martin Park on July 9, 2020

We have two urgent requests for you related to our ISIMIP DKRZ project. Please dedicate a few minutes to read this message, as in one case we need action from you latest by tomorrow, July 9th.

  1. Change passwords

As you may already know, there was a security incident on DKRZ on June 25th, 2020. Systems with the lustre file system mounted were hacked and confidential information, such as plain text passwords in files like ~/.netrc or ssh-keys, might have been stolen. If you have or had stored ssh-keys on Mistral, in particular those without passphrase, you should consider them stolen.

DKRZ temporarily deactivated login to DKRZ-servers with SSH-key, but access has now been re-enabled. However, for security reasons, SSH-keys are time-limited from now on and will expire after 6 weeks. Please register public SSH-keys at https://luv.dkrz.de/pubkeys/ prior to get access.

For your own safety please make sure to change your password on https://luv.dkrz.de until Thu 09.07.2020 if you use plain text passwords in .netrc or other scripts and you didn't change your password already after 25.06.2020. Otherwise your account will be blocked after 09.07.2020 until you have changed your password.

The entire story and instructions on changing passwords here:
- Password Change Enforced
- Time-limited SSH-Keys
- Security Incident Mistral

2. Check access to data on DKRZ

Some modelers have mentioned that they do not see any data listed within our DKRZ project after /mnt/lustre01/work/bb0820/. We have commented this to DKRZ Beratung and we got told that they have recently observed this behavior on another disk space as well.

In some cases, access was reestablished after setting a new login (as mentioned above). However, to fully investigate and solve this issue, they have asked for some information from us.

If you have encountered this problem, please respond to the following questions, so we can communicate a summary to DKRZ:

  • Can you see but not access directories and files listed after /mnt/lustre01/work/bb0820/, or you simply can not see anything listed after that point?
  • On which node/nodes and at what approximate time did you notice this? – It is nearly impossible for DKRZ to check the logs on all available nodes
  • What outcome do you get when typing the following commands:
    • ncdump -hs /work/bb0820/ISIMIP/ISIMIP2b/InputData/OBS_atmosphere/global/EWEMBI/historical/vas_ewembi1_2011_2016.nc4
    • find /work/bb0820/ISIMIP/ISIMIP2b/InputData/OBS_atmosphere/global/EWEMBI/historical -type f

Best regards,

the ISIMIP team at PIK