Purging Files from /scratch (Wed, Sept 18)
Tuesday, September 10, 2019 at 4:56 pm
OARC Research Computing Users,
This automated process will begin on Wednesday, Sept. 18 at 2:00pm ET.
Once this process begins, files sitting unused for 90 days in the Amarel and Perceval /scratch file systems will be deleted on an approximately weekly basis. Of course, the purging of files will only happen in the /scratch file system. No purging of files will take place anywhere else.
The /scratch filesystem on any shared computing system (not just ours) is a large temporary storage space intended only for actively running jobs. No data stored in /scratch are backed-up, so a hardware failure of the drives, controllers, or other support systems could cause unrecoverable loss of data at any time. That’s why important data must always be stored where it will be backed-up and many funded project data management plans now explicitly address this concern.
There are a variety of backed-up storage options available to Rutgers students, staff, and faculty:
• Unlimited storage in Box.Rutgers.edu, https://box.rutgers.edu for all students, staff, faculty (15 GB file size limit, but larger files can be split before uploading)
• 5 TB of space in Office365 OneDrive, https://it.rutgers.edu/rutgers-connect/knowledgebase/onedrive-for-business for all staff, faculty, and RBHS students (10 GB file size limit, but larger files can be split before uploading)
• Unlimited storage in Google Drive for ScarletApps users, https://it.rutgers.edu/scarletapps (5 TB file size limit, but larger files can be split before uploading)
• Backed-up /projects storage on Amarel, $150/TB for 4 years for compute node owners
Please let us know if you have questions or need help.