Modifying Reclaimed Files With Partial Upload May Result In A System Lockup Or Incorrect Cloud Update
Applies to:
Tiger Bridge Partial Upload enabled builds older than version 4.0.4.155.
IMPORTANT: This is considered a MANDATORY update release. Though the conditions triggering this issue may be rare, the potential risk to backup integrity exists.
Problem Summary
Files modified during a partial download and upload may incorrectly replicate changes to the cloud or cause the system to lockup. Only reclaimed files (indicated by a blue <Nearline> file icon tag in Windows File Explorer) are potentially affected.
Problem Description
An issue was observed on a file in a reclaimed state that is then modified using partial upload. Reclaimed files exist in the cloud target with a local stub-file. In this situation, the modified portion of the cloud file will be retrieved, modified by the application, then saved back to the cloud using partial upload.
This problem can occur when Veeam creates a Synthetic Full Backup using a previously reclaimed Full Backup. Users can verify the integrity of their backup files (.vbk files) through a Guest File restore or Instant VM Recovery. If the .vbk file was incorrectly modified, Veeam will report a checksum error. Users can also retrieve their files from the cloud to verify integrity. If Veeam issues a checksum error, a new full backup should be created and replicated to the cloud.
Update Tiger Bridge to the latest version listed below. Normal backup routines may be resumed following the update.
Cause
The cause of this issue has been identified as a rare race condition that may occur when a cloud reclaimed file is partially modified resulting in the partial retrieval of data being modified.
Resolution
Tiger Bridge build 4.0.4.155 (or later) includes a fix for this issue.
If you already have Tiger Bridge installed and configured with a Backblaze account, when the Configuration Wizard opens, simply close the wizard or re-enter your original credentials.
Workaround
Users that are not in a position to upgrade Tiger Bridge immediately should refrain from RECLAIMING SPACE on any file subject to partial modification (e.g. full .vbk backup files).
Contacting Tiger Support
https://www.tiger-technology.com/contact/support/
Tiger Bridge 4.0.4.155 also includes these features and improvements:
What’s new?
Partial Upload
Partial Download
ReFS file system support
Critical bug fixes and stability improvements
Partial Upload
Tiger Bridge now writes incremental changes to the cloud, instead of uploading the entire file again. This feature greatly enhances the Veeam workflow as it minimizes data transfers of Synthetic Full Backups.
NOTE: If a file is modified while being uploaded, Tiger Bridge needs to cancel and restart the upload process. Bridge can only start managing partial uploads after a file has been entirely uploaded to the cloud.
Partial Download
Tiger Bridge supports partial download for Veeam’s Guest File access and Instant VM Recovery operations.
ReFS File System Support
Tiger Bridge now supports ReFS in addition to the NTFS file system. ReFS is particularly well suited for managing Veeam backups in an efficient way. Combined with Tiger Bridge, workflows are greatly streamlined.
Note to Veeam users:
Because “full” backup files can grow extremely large, and because Veeam touches them each time it does an incremental backup, Tiger recommends pausing Veeam to ensure that the full backup file was fully uploaded to the cloud before Veeam runs its next incremental backup job. Once the full backup is in the cloud, you can resume normal Veeam operations.
Note to File Server users:
This version of Tiger Bridge is specifically optimized for Veeam use cases. If you would like to run Tiger Bridge on your file server, ftp server or laptop to extend your local drive into the cloud, please let us know and we will provide you with the proper version!