CryoEM (Cryo-Electron Microscopy) HPC - How to transfer data to/from the shared Cryosparc Volume


Introduction

This article is written to help new users of the CryoSparc Cluster transfer data to be used on the shared Turbo volume via Globus. If you need additional help with the content of this article, feel free to reach out via our help form. If you need more information on what the CryoEM service is, start with our landing page. NOTE: You will not be able to do the below instructions until an account has been requested and made for you for Cryosparc.

NOTE: Because this is a shared system you can not use it as permanent storage.  In order to preserve your projects and results you will need to copy the project folder to your permanent storage once jobs have been run.

Instructions

Step 1: Navigate to Globus's website in a web browser.

Step 2: Click Log In in the top right corner.



Step 3: In the dropdown box that says Look-up your organization… find University of Michigan. (You can enter the name to make finding it easier.)

 

Step 4: Once Selected, click continue. You'll be redirected to a weblogin form like for other UMICH services. Use your level 1 credentials.

 

 

Step 5: Once you have logged in you will be presented with the globus file manager, which looks like the below:

Step 6: In the top right, there is the panels option, Click the middle option, this will give you two panels in the file explorer.



Step 7: On the right side we'll choose the Destination of the transfer (The shared CryoEM Volume), the left will be the Source, but first we need to pick the collection they are both in.
The shared volume is in the UMich ARC Non-Sensitive Turbo Volume Collection as shown below. You can do the reverse of this to transfer from the shared volume to your permanant storage instead.

Step 8: Next set the path for the Destination volume to /lsi-umms-cryoem/ if you are transfering to the shared volume or whichever permanant storage location you intend to move your project results too from the shared volume, You can enter this directly in the path box for the destination side like so:



Step 9. This is the root of the shared volume, if you already have a folder for yourself/project, you can enter it from here, or use the create a new folder button to make a folder for yourself/project if this is your first time.

Step 10. Repeat this process on the left side for the source, but navigating to where your data you are trying to move is (Or the shared volume if your moving data from it to your permanant storage). This can be the same collection if it's a non-sensitive Turbo Volume, or the Non-sensitive Data Den Collection (named UMich ARC Non-Sensitive Data Den Volume Collection) if it's archived data being pulled out of cold storage. It's important to note that Sensitive Data is not permitted on the CryoEM Cryosparc cluster. You can also transfer data in from a guest collection which can be your work PC or other location outside of a Globus collection. For more info on setting a Guest Collection up, see this article. In the example below another turbo volume is used.



Step 11: From here you can go into your folder on the Destination if you have not already, select any files that need to be transferred on the left side and then click start above the source:

Step 12: From here, Globus will copy the files into the destination directory automatically and after some time (depending on the size of the files) you should see it show up in the destination, after clicking the refresh button.



Step 13: Transfer complete! Repeat for any further files.