rss logo

Synchronize Files from Cloud Storage with Rclone

Rclone Logo

I was searching for a way to automatically synchronize a KeePass file between two computers that are not on the same network and don't have access to a shared drive. After some personal research, I came across the Rclone tool, which met all my requirements. It's open source, free (you can support the project here: https://rclone.org/sponsor/), and it supports a wide range of cloud storage services, including Proton Drive.

  • Here's what I do:
    • Synchronize a KeePass file between two Windows workstations to account for modifications made by both user01 and user02.
    • Utilize Proton Drive cloud storage.
    • Enable Bi-Sync mode, allowing the KeePass file to be modified on both computers.

⚠️Note: Be aware that the bi-synchronization option in Rclone is still in development. Therefore, it's essential to regularly make backups of your files to prevent any potential data loss.⚠️

Rclone architecture showing two workstations synchronizing a KeePass file

Create a Proton Account

Proton Account Creation Page
  • From the main menu, select Drive to enable Cloud Storage:
Proton Drive Activation

Rclone

Configure Rclone with Proton Drive on PC01 and PC02

Rclone official download page
  • Choose the appropriate download link, depending on your architecture:
Select the Right Download Link Based on Your Architecture
  • Extract the contents of the zip archive and rename the extracted folder to rclone:
Extract and Rename the Rclone Folder
  • Open the terminal from the extracted folder and run the command .\rclone.exe config:
Open Terminal and Run the Rclone Configuration Command
  • Follow the configuration instructions and provide answers to the questions asked:
PS C:\Users\user\Downloads\rclone> .\rclone.exe config 2023/10/07 14:46:36 NOTICE: Config file "C:\\Users\\user\\AppData\\Roaming\\rclone\\rclone.conf" not found - using defaults No remotes found, make a new one? n) New remote s) Set configuration password q) Quit config n/s/q> n Enter name for new remote. name> proton Option Storage. Type of storage to configure. Choose a number from below, or type in your own value. 1 / 1Fichier \ (fichier) 2 / Akamai NetStorage \ (netstorage) 3 / Alias for an existing remote \ (alias) […] 37 / Proton Drive \ (protondrive) 38 / Put.io \ (putio) 39 / QingCloud Object Storage \ (qingstor) 40 / Quatrix by Maytech \ (quatrix) 41 / SMB / CIFS \ (smb) […] Storage> 37 Option username. The username of your proton account Enter a value. username> YOUREMAILADDRESS@proton.me Option password. The password of your proton account. Choose an alternative below. y) Yes, type in my own password g) Generate random password y/g> y Enter the password: password: ******** Confirm the password: password: ******** Option 2fa. The 2FA code The value can also be provided with --protondrive-2fa=000000 The 2FA code of your proton drive account if the account is set up with two-factor authentication Enter a value. Press Enter to leave empty. 2fa> Edit advanced config? y) Yes n) No (default) y/n> n Configuration complete. Options: - type: protondrive - username: YOUREMAILADDRESS@proton.me - password: *** ENCRYPTED *** Keep this "proton" remote? y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y Current remotes: Name Type ==== ==== proton protondrive e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> q
  • Check the connection to Proton Drive:
PS C:\Users\user\Downloads\rclone> .\rclone.exe ls proton:
  • If you receive this error message, make sure that you have previously opened the Proton Drive application from the web interface, as demonstrated above:
2023/10/07 15:44:12.806871 ERROR RESTY 401 GET https://mail.proton.me/api/core/v4/users: Invalid access token (Code=401, Status=401), Attempt 1 2023/10/07 15:44:12.945153 ERROR RESTY 422 POST https://mail.proton.me/api/auth/v4/refresh: Invalid refresh token (Code=10013, Status=422), Attempt 1 2023/10/07 15:44:16 mainShare &proton.Share{ShareMetadata:proton.ShareMetadata{ShareID:"", LinkID:"", VolumeID:"", Type:0, State:0, CreationTime:0, ModifyTime:0, Creator:"", Flags:0, Locked:false, VolumeSoftDeleted:false}, AddressID:"", AddressKeyID:"", Key:"", Passphrase:"", PassphraseSignature:""} 2023/10/07 15:44:16 shares []proton.ShareMetadata{} 2023/10/07 15:44:16 Failed to create file system for "proton:": couldn't initialize a new proton drive instance: the main share assumption has failed

Using Rclone

Once the Cloud Storage is configured, let's explore what we can do with this tool.

  • Print the configured remotes:
PS C:\Users\user\Downloads\rclone> .\rclone.exe listremotes proton:
  • Create an empty TEST.txt file in Proton Drive:
PS C:\Users\user\Downloads\rclone> .\rclone.exe touch proton:TEST.txt a TEST.txt File from Proton Drive web interface
  • Synchronize Cloud Drive with the local folder C:\Users\user\Desktop\PROTON\:
PS C:\Users\user\Downloads\rclone> .\rclone.exe sync proton: C:\Users\user\Desktop\PROTON\ A TEST.txt File in a windows explorer
  • List files in your Cloud Drive:
PS C:\Users\user\Downloads\rclone> .\rclone.exe ls proton: 0 TEST.txt
  • Check the version and update Rclone to the latest version if one is available:
PS C:\Users\user\Downloads\rclone> .\rclone.exe selfupdate 2023/10/07 16:39:33 NOTICE: rclone is up to date
  • Display all available Rclone options:
PS C:\Users\user\Downloads\rclone> .\rclone.exe Usage: rclone [flags] rclone [command] Available Commands: about Get quota information from the remote. authorize Remote authorization. backend Run a backend-specific command. bisync Perform bidirectional synchronization between two paths. cat Concatenates any files and sends them to stdout. check Checks the files in the source and destination match. checksum Checks the files in the source against a SUM file. cleanup Clean up the remote if possible. completion Output completion script for a given shell. config Enter an interactive configuration session. copy Copy files from source to dest, skipping identical files. copyto Copy files from source to dest, skipping identical files. copyurl Copy url content to dest. cryptcheck Cryptcheck checks the integrity of an encrypted remote. cryptdecode Cryptdecode returns unencrypted file names. dedupe Interactively find duplicate filenames and delete/rename them. delete Remove the files in path. deletefile Remove a single file from remote. gendocs Output markdown docs for rclone to the directory supplied. hashsum Produces a hashsum file for all the objects in the path. help Show help for rclone commands, flags and backends. link Generate public link to file/folder. listremotes List all the remotes in the config file and defined in environment variables. ls List the objects in the path with size and path. lsd List all directories/containers/buckets in the path. lsf List directories and objects in remote:path formatted for parsing. lsjson List directories and objects in the path in JSON format. lsl List the objects in path with modification time, size and path. md5sum Produces an md5sum file for all the objects in the path. mkdir Make the path if it doesn't already exist. mount Mount the remote as file system on a mountpoint. move Move files from source to dest. moveto Move file or directory from source to dest. ncdu Explore a remote with a text based user interface. obscure Obscure password for use in the rclone config file. purge Remove the path and all of its contents. rc Run a command against a running rclone. rcat Copies standard input to file on remote. rcd Run rclone listening to remote control commands only. rmdir Remove the empty directory at path. rmdirs Remove empty directories under the path. selfupdate Update the rclone binary. serve Serve a remote over a protocol. settier Changes storage class/tier of objects in remote. sha1sum Produces an sha1sum file for all the objects in the path. size Prints the total size and number of objects in remote:path. sync Make source and dest identical, modifying destination only. test Run a test command touch Create new file or change file modification time. tree List the contents of the remote in a tree like fashion. version Show the version number. Use "rclone [command] --help" for more information about a command. Use "rclone help flags" for to see the global flags. Use "rclone help backends" for a list of supported services.

Configure Bisynchronization

Now that we've briefly explored how to use Rclone, let's return to our primary scenario: bisynchronization of a KeePass file between two hosts.

PC01

  • Create a C:\RCLONE\ directory:
PS C:\Users\user\Downloads\rclone> mkdir C:\RCLONE
  • Send the C:\RCLONE\pwd.kdbx file to Proton Drive:
PS C:\Users\user\Downloads\rclone> .\rclone.exe sync C:\RCLONE\pwd.kdbx proton: Diagram showing the Send of a pwd.kdbx File to Proton Drive
  • Enable bisync between C:\RCLONE\ and Proton Drive:
PS C:\Users\user\Downloads\rclone> .\rclone.exe bisync C:\RCLONE\ proton: --resync Diagram showing the Bisynchronization between C:\RCLONE\ and Proton Drive
  • Perform a bidirectional synchronization between C:\RCLONE\ and Proton Drive:
PS C:\Users\user\Downloads\rclone> .\rclone.exe bisync C:\RCLONE\ proton: --force

PC02

  • Create a C:\RCLONE\ directory:
PS C:\Users\user\Downloads\rclone> mkdir C:\RCLONE
  • Send the pwd.kdbx file from the Proton Drive to C:\RCLONE\:
PS C:\Users\user\Downloads\rclone> .\rclone.exe sync proton: C:\RCLONE\ Diagram showing the Send of a pwd.kdbx File from Proton Drive
  • Enable bisync between C:\RCLONE\ and Proton Drive:
PS C:\Users\user\Downloads\rclone> .\rclone.exe bisync C:\RCLONE\ proton: --resync Diagram showing the Bisynchronization between C:\RCLONE\ and Proton Drive
  • Perform a bidirectional synchronization between C:\RCLONE\ and Proton Drive:
PS C:\Users\user\Downloads\rclone> .\rclone.exe bisync C:\RCLONE\ proton: --force

Automate Bisynchronization with Task Scheduler

Now that our architecture is configured to work, we still need to manually run the .\rclone.exe bisync C:\RCLONE\ proton: --force command on each computer every time the pwd.kdbx file is modified. To automate this process, we'll use a batch script that will be executed by the Windows Task Scheduler.

Batch Script

Create a C:\rclone.bat script.

@echo off REM VARIABLES set SOURCE=C:\RCLONE set DESTINATION=proton: set DOWNLOADS=C:\Users\USERNAME\Downloads REM UPDATE RCLONE TO LATEST RELEASE %DOWNLOADS%"\rclone\rclone.exe" selfupdate REM FORCE BISYNC %DOWNLOADS%"\rclone\rclone.exe" bisync %SOURCE% %DESTINATION% --force

Windows Task Scheduler

  • Open the Windows Task Scheduler:
Opening Windows Task Scheduler
  • Create a new task:
Creating a New Windows Task
  • Give a name to the task and configure it:
Naming and Configuring the Task
  • In the Triggers tab, click New… and configure the task to run every 5 minutes (adjust this value according to your needs):
Setting Task Trigger to Run Every 5 Minutes
  • In the Actions tab, click New… and set the previously created batch script, adding C:\Users\user\Downloads\rclone\ in the Start in option:
Configuring Task Actions and Start Directory
  • Finally, in the Conditions tab, specify that the task should only start if a network connection is available:
Setting Task Condition to Start Only with Network Connection