REDCap (Academic License) - Bulk Removal of Data


Introduction

Data captured in REDCap may periodically need to be removed from the application for various reasons. This could be due to IRB requirements to erase data at the conclusion of a study or after a set amount of time, participant requests, or other study-specific reasons.

While removing data from a small number of fields in a single record is generally a straightforward edit of a single record, removing or erasing data from a number of fields for a large number of records through this method can be time consuming.  The below instructions provide a method of removing data in bulk from multiple fields and / or multiple records at once.

Important Note: Removal of data entered into fields in REDCap may not entirely remove the logging history of the given record, nor will it remove the data from previous data exports.  Please contact michr-support@umich.edu if data that cannot be removed through normal means (e.g. logging) that must be removed due to legal or regulatory requirements such as GDPR remains in REDCap after following the instructions below.

Important Note: REDCap does not offer an 'undo' feature and if the below steps are followed the erased data cannot be recovered by MICHR Support. Please always exercise caution when making bulk changes to data and create a full Data Export backup prior to making changes if you are unsure of the impact of the changes.

Instructions

Note: The below instructions expect the user executing these steps have appropriate User Rights to access, export, and import data into the REDCap project as well as access to the appropriate data and data access groups.  If the user responsible for removing data lacks necessary privileges they should work with the appropriate study team member to obtain necessary User Rights.

  1. Determine the fields that need to be erased and if they need to be erased for all records or a subset of records.  In this example we are erasing the data in 'Sensitive Data Field 1' and 'Sensitive Data Field 2' for all records.
  2. Go to Data Exports, Reports, and Stats.  Choose 'Create New Report'.
  3. Title the report appropriately and in 'Fields to include in report' select each of the fields that needs to be erased / removed (following the Record ID field).
  4. Note: If only erasing data from a subset of records, creating additional filter options (e.g. by date range, completion status, etc)  in the 'Step 3: Filters' section of the report builder can limit the cleared fields only to the desired records.  In this example we are clearing all data in the field [sensitive_1] and [sensitive_2] for all records in this project so no additional filters are set.
  5. 'Save Report' then select 'View Report'.
  6. In the View Report page, verify that the correct fields, records, and data are shown that needs to be removed.
  7. Click 'Export Data' and choose 'CSV / Microsoft Excel (raw data) export format.  Verify De-identification and Advanced data formatting options are set appropriately.
  8. Click 'Export Data' and download and save the Excel CSV to a known location.
  9. Select the data from the rows and columns that needs to be erased from REDCap and erase the values in those cells.
  10. Save the modified .CSV file and close Excel.
  11. Return to REDCap and go into the Data Import tool.
  12. Important: Set 'Allow blank values to overwrite existing saved values?' to 'Yes, blank values in the file will overwrite existing values'.
  13. Choose the edited .CSV file then select 'Upload File'.
  14. Review the changes in the Data Display table to ensure that the only data being modified is the desired data that is being overwritten.
  15. Verify the data being overwritten is correct and there are no undesirable data changes.  WARNING: Once you click 'Import Data' the data is permanently overwritten and you can not 'Undo' the changes.
  16. Click 'Import Data' and confirm you receive an 'Import Successful' message with the appropriate number of modified records.
  17. Review your data to ensure the expected data has been cleared / removed from your REDCap project.