Secure backups on OneDrive for Unige

Apr 5, 2021 10:59 Β· 1784 words Β· 9 minute read

This short tutorial describes how to deploy an encrypted rclone remote on a OneDrive Business account. It is aimed at University of Geneva setting.

Start by installing rclone . The instructions below where tested with version 1.54.1 on Linux, but they should work also with MacOS or Windows through WSL (not tested).

Configure OneDrive remote πŸ”—

$ rclone config
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n
name> onedrive-ug

Here onedrive-ug is an arbitrary name (OneDrive, University of Geneva)

Then scroll the list to find the correct storage type and input the number to answer the question:

...
26 / Microsoft OneDrive
   \ "onedrive
...
Storage> 26

I got 26, but the actual number will depend on the version of rclone you have insinstalled.

Next we choose the default option for each question (read the question and just press <return>):

Enter a string value. Press Enter for the default ("").
client_id>
OAuth Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret>
region>
Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n> 
Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine
y) Yes (default)
n) No
y/n> 

At this point, a Web browser should open a localhost page. You must sign in. You must use your unige email adress as user name.

The browser will be redirected to the standard ISIS login. Once done you may press stay signed on the MS page (don’t think it matters).

A success message will finally be printed in your browser. Move back to rclone and continue the remote creation:

Got code
Choose a number from below, or type in an existing value
 1 / OneDrive Personal or Business
   \ "onedrive"
 2 / Root Sharepoint site
   \ "sharepoint"
 3 / Sharepoint site name or URL (e.g. mysite or https://contoso.sharepoint.com/sites/mysite)
   \ "url"
 4 / Search for a Sharepoint site
   \ "search"
 5 / Type in driveID (advanced)
   \ "driveid"
 6 / Type in SiteID (advanced)
   \ "siteid"
 7 / Sharepoint server-relative path (advanced, e.g. /teams/hr)
   \ "path"
Your choice> 1
Found 1 drives, please select the one you want to use:
0: OneDrive (business) xxxxxxxxxxxxxxx
Chose drive to use:> 0
Found drive 'root' of type 'business', URL: https://unigech-my.sharepoint.com/personal/xxxxxx_unige_ch/Documents
Is that okay?
y) Yes (default)
n) No
y/n> y
--------------------
[onedrive-ug]
type = onedrive
token = {"access_token":xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
drive_id = b!xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
drive_type = business
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:

Name                 Type
====                 ====
onedrive-ug          onedrive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

Now your remote is created. You can test with:

$ rclone lsd onedrive-ug:
$ rclone mkdir onedrive-ug:test_dir
$ rclone lsd onedrive-ug:

It works, but the files will stored in cleartext. It may be useful if you want to use Office365 or share some data.

Configure Crypt πŸ”—

Now we create a encrypted remote inside the OneDrive remote. First of all create a directory with:

$ rclone mkdir onedrive-ug:crypt

Here onedrive-ug is the name of the remove created above and crypt is an arbitrary name.

We can now create the new remote:

$ rclone config
Current remotes:

Name                 Type
====                 ====
onedrive-ug          onedrive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n
name> onedrive-ug-secret

Again onedrive-ug-secret is arbitrary. Use whatever suits you.

This time we look for the crypt storage type:

...
11 / Encrypt/Decrypt a remote
   \ "crypt"
...
Storage> 11

Mine was at position 11, but scroll the list and use the actual number for your version of rclone.

Then we input the path we mdkir’ed above:

Remote to encrypt/decrypt.
Normally should contain a ':' and a path, e.g. "myremote:path/to/dir",
"myremote:bucket" or maybe "myremote:" (not recommended).
Enter a string value. Press Enter for the default ("").
remote> onedrive-ug:crypt

We now decide to encrypt everything by default:

How to encrypt the filenames.
Enter a string value. Press Enter for the default ("standard").
Choose a number from below, or type in your own value
 1 / Encrypt the filenames see the docs for the details.
   \ "standard"
 2 / Very simple filename obfuscation.
   \ "obfuscate"
 3 / Don't encrypt the file names.  Adds a ".bin" extension only.
   \ "off"
filename_encryption> 1
Option to either encrypt directory names or leave them intact.

NB If filename_encryption is "off" then this option will do nothing.
Enter a boolean value (true or false). Press Enter for the default ("true").
Choose a number from below, or type in your own value
 1 / Encrypt directory names.
   \ "true"
 2 / Don't encrypt directory names, leave them intact.
   \ "false"
directory_name_encryption> 1

At this point we must set the password. I used random passwords, stored in a password manager.

WARNING, WARNING, WARNING: If you loose the passwords, files are no more recoverable ! You can print them on paper, OCR software will allow you to recover them easily, or better, encode them in a QR Code and print it on paper .

Password or pass phrase for encryption.

y) Yes type in my own password
g) Generate random password
y/g> g
Password strength in bits.
64 is just about memorable
128 is secure
1024 is the maximum
Bits> 512
Your password is: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Use this password? Please note that an obscured version of this 
password (and not the password itself) will be stored under your 
configuration file, so keep this generated password in a safe place.
y) Yes (default)
n) No
y/n> 
Password or pass phrase for salt. Optional but recommended.
Should be different to the previous password.
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank (default)
y/g/n> g
Password strength in bits.
64 is just about memorable
128 is secure
1024 is the maximum
Bits> 512
Your password is: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Use this password? Please note that an obscured version of this 
password (and not the password itself) will be stored under your 
configuration file, so keep this generated password in a safe place.
y) Yes (default)
n) No
y/n> 
Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n>
Remote config
--------------------
[onedrive-ug-secret]
type = crypt
remote = onedrive-ug:crypt
filename_encryption = standard
directory_name_encryption = true
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>
Current remotes:

Name                 Type
====                 ====
onedrive-ug          onedrive
onedrive-ug-secret   crypt

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

And it is done, you can check if everything is ok:

$ rclone lsd onedrive-ug-secret:
$ rclone mkdir onedrive-ug-secret:test_dir
$ rclone lsd onedrive-ug-secret:
$ rclone lsd onedrive-ug:crypt

OneDrive Limitations and other remarks πŸ”—

We just address the points relevant to backups. For more info look at the complete OneDrive rclone doc .

  • Deleted files end in OneDrive Trash. You should purge them manually. No API is exposed for that.
  • Your MS credentials (the refresh token) will expire if you don’t access the remote for 90 days. You will have to use rclone config reconnect onedrive-ug:. No need to worry if you use a backup cronjob, the token will be refreshed automatically.
  • Maximum size for a single file: 250GB (this may vary). In case you need it, there is a /chunking/ extension (still beta).
  • Avoid storing more than 50'000 files per directory.
  • OneDrive file and dir paths have a lot of weird limitations (inherited from Win95 ???). It should not be a problem if you use the encryted file names. You can check the actual rules in the OneDrive official doc .
  • By default the OneDrive content is versioned. While it is great, the version counts toward the account quota ! If you do a rclone copy you will create to version of a single file, because changing file attributes count as a version. You can solve this problem by either:
    • Purge excess version by running: rclone cleanup onedrive-ug-secret:subdir (use -i for interactive execution. May be slow.
    • Set the no_versions=true parameter in the config, wich forces rclone to remove any excess version. But it increases the transactions (and seems to work only with OneDrive Business). To set it, run rclone config file to retrieve the file path. Then edit directly the file, adding the setting to the onedrive-ug section (not the one-drive-ug-secret).
    • Change the OneDrive setting (almost impossible to find by yourself). The trick is explained in the appropriate section but you must use the old setting page (in french revenir Γ  l’ancienne page des paramΓ¨tres des sites)…

Backup Operations πŸ”—

Replace onedrive-ug-secret by the actual name of your remote.

Backuping data πŸ”—

Two options here:

  • You can sync your data to the remote. If you delete a file on your local machine, sync will also delete it from the remote.
  • You can copy your data to the remote. Existing files will not be erased.

OneDrive supports versions and has a trash, but I suspect that the encryption will render those functions useless). So use sync if you want to mirror the local machine, or copy if you want to be immune against accidental deletions.

Of course, both approach can be used…

The command is just:

$ rclone <METHOD> --log-file=<LOG> <SOURCE> onedrive-ug:<DEST>

where:

  • <METHOD> is either sync or copy
  • <LOG> is the log file name (useful if automated or the execution is long)
  • <SOURCE> is the directory to backup
  • <DEST> is the directory that will receive the backup on the remote

In case of doubt, use interactive mode -i or dry run -n.

You may want to prepend nohup or setsid if the operation is long, especially during the first backup.

Restoring data πŸ”—

Of course copy or sync works in both directions. Just reverse the arguments order to restore data.

$ rclone <METHOD> --log-file=<LOG> onedrive-ug:<PATH> <LOCAL_PATH>

Mounting the backup πŸ”—

We can directly mount the remote thanks to FUSE:

$ mkdir backup
$ rclone mount --vfs-cache-mode writes onedrive-ug-secret: backup/ &
$ cd backup

Don’t forget the ampersand & at the end.

You can then perform filesystem operations. You can cherry pick files to restore, erase unwanted files, etc.

Once done, unmount after leaving the mountpoint:

$ cd ..
$ umount backup/

Maintenance πŸ”—

Checking storage usage:

$ rclone about onedrive-ug-secret:

Cleaning past versions:

$ rclone cleanup onedrive-ug-secret:

References πŸ”—