在 Proxmox VE 中直接備份至 Google Drive (vzdump + rclone)

最近在幫自己的網站從 Vultr 搬到自己家裡的主機,因為家裡的主機是拿老桌機當 Server 用,怕隨時硬碟會炸掉XD,於是就有了這個備份需求,直接丟到學校提供的Google Drive 教育板,就不用太擔心了。

目前這個需求已經有人開發了工具,可以直接結合 Proxmox VE 內部的備份工具,可以在 Backup Log 中看到 Rclone 的相關 Log,搭配 Proxmoe VE 內建 CRON 自動備份到各種網路空間,非常方便:

 

安裝 Rclone

在 Proxmox VE 中以 root 身份登入,直接使用指令安裝 rclone

# apt update
# apt install rclone -y

亦或到 Rclone 的官網下載安裝:

初始化  Rclone

請務必仔細跟著設定,或是你自己去研究 rclone 的 config

# rclone config
2021/10/08 02:03:49 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> gd-backup_crypt # 務必使用這個名字,或是之後改腳本
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
~~
15 / Google Drive
\ "drive"
~~
Storage> 15
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a string value. Press Enter for the default ("").
client_id> 123456789-xxxxxxxxxxxxxx.apps.googleusercontent.com
OAuth Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Scope that rclone should use when requesting access from drive.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / Full access all files, excluding Application Data Folder.
\ "drive"
~~
scope> 1
ID of the root folder
Leave blank normally.

Fill in to access "Computers" folders (see docs), or for rclone to use
a non root folder as its starting point.

Enter a string value. Press Enter for the default ("").
root_folder_id> xxxx_123456789
# 在 Google Drive 建一個資料夾,網址最後一串就是 root_folder_id
# https://drive.google.com/drive/u/1/folders/xxxx_123456789
Service Account Credentials JSON file path
Leave blank normally.
Needed only if you want use SA instead of interactive login.

Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.

Enter a string value. Press Enter for the default ("").
service_account_file> [Enter]
Edit advanced config?
y) Yes
n) No (default)
y/n> [Enter]
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine

y) Yes (default)
n) No
y/n> n
Verification code

Go to this URL, authenticate then paste the code here.

https://accounts.google.com/o/oauth2/auth?xxxxxxxxxx
# 複製上面網址到瀏覽器登入帳號,取得驗證碼填入下方
Enter a string value. Press Enter for the default ("").
config_verification_code> 123454321
2021/10/08 02:16:42 NOTICE: Make sure your Redirect URL is set to "urn:ietf:wg:oauth:2.0:oob" in your custom config.
Configure this as a Shared Drive (Team Drive)?

y) Yes
n) No (default)
y/n> [Enter]
--------------------
[gd-backup_crypt]
type = drive
client_id = 123456789-xxxxxxxxxxxxxx.apps.googleusercontent.com
client_secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
scope = drive
root_folder_id = xxxx_123456789
token = {"access_token":"123454321","token_type":"Bearer","refresh_token":"123454321","expiry":"2021-10-08T03:16:42.034218011+08:00"}
team_drive =
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>
Current remotes:

Name Type
==== ====
gd-backup_crypt drive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

 

安裝 proxmox-vzbackup-rclone

下載

以 root 身分執行,這是我修改過的版本:

# apt-get install git
# cd /root
# git clone https://github.com/fuyuanli/proxmox-vzbackup-rclone.git
# git checkout pve7
# chmod +x /root/proxmox-vzbackup-rclone/vzbackup-rclone.sh

修改 vzbackup-rclone.sh

修改第 5 行至你的備份儲存路徑

1 #!/bin/bash
 2 # ./vzbackup-rclone.sh rehydrate YYYY/MM/DD file_name_encrypted.bin
 3 
 4 ############ /START CONFIG
 5 dumpdir="/data/pve/dump" # Set this to where your vzdump files are stored

備份保留的份數會依據你在 Proxmox VE 的 Backup 選項那邊來刪除,這個腳本只負責上傳檔案到雲端空間

Proxmox VE 7 修正

參考 Issue#7 中提到的方法修正,否則會上傳失敗,我 Fork 後修改的版本已經修正了:

修改第 18 行 tarfile=${TARFILE} 改成 tarfile=${TARGET}

15 if [ ! -z "${3}" ];then
16     CMDARCHIVE=$(echo "/${3}" | sed -e 's/\(.bin\)*$//g')
17 fi
18 tarfile=${TARGET}
19 exten=${tarfile#*.}
20 filename=${tarfile%.*.*}

設定 vzdump.conf

編輯 /etc/vzdump.conf 第 14 行後加入

script:/root/proxmox-vzbackup-rclone/vzbackup-rclone.sh

後存檔

1  # vzdump default settings
 2
 3  #tmpdir: DIR
 4  #dumpdir: DIR
 5  #storage: STORAGE_ID
 6  #mode: snapshot|suspend|stop
 7  #bwlimit: KBPS
 8  #ionice: PRI
 9  #lockwait: MINUTES
10  #stopwait: MINUTES
11  #stdexcludes: BOOLEAN
12  #mailto: ADDRESSLIST
13  #prune-backups: keep-INTERVAL=N[,...]
14  #script: FILENAME
15  script:/root/proxmox-vzbackup-rclone/vzbackup-rclone.sh
16  #exclude-path: PATHLIST
17  #pigz: N

設定完畢後就可以在 Proxmox VE 的界面備份了,每次備份都會觸發 vzbackup-rclone.sh ,再透過 rclone 上傳,執行 log 如下:

INFO: starting new backup job: vzdump 110 --storage local --node home --compress zstd --remove 0 --mode snapshot
INFO: Deleting backups older than 3 days.
INFO: filesystem type on dumpdir is 'zfs' -using /var/tmp/vzdumptmp2067793_110 for temporary files
INFO: Starting Backup of VM 110 (lxc)
INFO: Backup started at 2021-10-08 18:42:05
INFO: status = running
INFO: CT Name: pykms
INFO: including mount point rootfs ('/') in backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: create storage snapshot 'vzdump'
INFO: creating vzdump archive '/var/lib/vz/dump/vzdump-lxc-110-2021_10_08-18_42_05.tar.zst'
INFO: Total bytes written: 962498560 (918MiB, 65MiB/s)
INFO: archive file size: 274MB
INFO: Backing up /var/lib/vz/dump/vzdump-lxc-110-2021_10_08-18_42_05.tar.zst to remote storage
INFO: rcloning /var/lib/vz/dump/rclone/2021/10
INFO: 2021/10/08 18:43:21 INFO  : 
INFO: Transferred:   	   44.496Mi / 274.777 MiByte, 16%, 763.432 KiByte/s, ETA 5m8s
INFO: Transferred:            0 / 1, 0%
INFO: Elapsed time:       1m1.3s
INFO: Transferring:
INFO:  *    vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 16% /274.777Mi, 763.432Ki/s, 5m8s
INFO: 2021/10/08 18:44:21 INFO  : 
INFO: Transferred:   	   88.684Mi / 274.777 MiByte, 32%, 750.874 KiByte/s, ETA 4m13s
INFO: Transferred:            0 / 1, 0%
INFO: Elapsed time:       2m1.3s
INFO: Transferring:
INFO:  *    vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 32% /274.777Mi, 750.874Ki/s, 4m13s
INFO: 2021/10/08 18:45:21 INFO  : 
INFO: Transferred:   	  132.840Mi / 274.777 MiByte, 48%, 769.235 KiByte/s, ETA 3m8s
INFO: Transferred:            0 / 1, 0%
INFO: Elapsed time:       3m1.3s
INFO: Transferring:
INFO:  *    vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 48% /274.777Mi, 769.235Ki/s, 3m8s
INFO: 2021/10/08 18:46:21 INFO  : 
INFO: Transferred:   	  177.402Mi / 274.777 MiByte, 65%, 768.902 KiByte/s, ETA 2m9s
INFO: Transferred:            0 / 1, 0%
INFO: Elapsed time:       4m1.3s
INFO: Transferring:
INFO:  *    vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 64% /274.777Mi, 768.902Ki/s, 2m9s
INFO: 2021/10/08 18:47:21 INFO  : 
INFO: Transferred:   	  218.777Mi / 274.777 MiByte, 80%, 645.420 KiByte/s, ETA 1m28s
INFO: Transferred:            0 / 1, 0%
INFO: Elapsed time:       5m1.3s
INFO: Transferring:
INFO:  *    vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 79% /274.777Mi, 645.420Ki/s, 1m28s
INFO: 2021/10/08 18:48:21 INFO  : 
INFO: Transferred:   	  262.652Mi / 274.777 MiByte, 96%, 742.687 KiByte/s, ETA 16s
INFO: Transferred:            0 / 1, 0%
INFO: Elapsed time:       6m1.3s
INFO: Transferring:
INFO:  *    vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 95% /274.777Mi, 742.687Ki/s, 16s
INFO: 2021/10/08 18:48:40 INFO  : vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: Copied (new)
INFO: 2021/10/08 18:48:40 INFO  : 
INFO: Transferred:   	  274.777Mi / 274.777 MiByte, 100%, 668.665 KiByte/s, ETA 0s
INFO: Transferred:            1 / 1, 100%
INFO: Elapsed time:      6m20.2s
INFO: cleanup temporary 'vzdump' snapshot
INFO: Finished Backup of VM 110 (00:06:35)
INFO: Backup finished at 2021-10-08 18:48:40
INFO: Backing up main PVE configs
INFO: Tar files
INFO: rcloning /var/tmp/proxmox-cC0xj8WC/pveConfig.2021-10-08.18.48.40.tgz
INFO: 2021/10/08 18:48:43 INFO  : pveConfig.2021-10-08.18.48.40.tgz: Copied (new)
INFO: 2021/10/08 18:48:43 INFO  : pveConfig.2021-10-08.18.48.40.tgz: Deleted
INFO: 2021/10/08 18:48:43 INFO  : 
INFO: Transferred:   	   34.916Ki / 34.916 KiByte, 100%, 34.910 KiByte/s, ETA 0s
INFO: Checks:                 2 / 2, 100%
INFO: Deleted:                1 (files), 0 (dirs)
INFO: Renamed:                1
INFO: Transferred:            1 / 1, 100%
INFO: Elapsed time:         3.0s
INFO: Cleaning up
INFO: Backup job finished successfully
TASK OK

 

跑完的檔案會在這兩個位置

GoogleDrive/Year/Month/vzdump:VM/LXC 備份

GoogleDrive/Year/Month/pveconfig/etc/pve/ 備份

檔案還原

使用指令

~/proxmox-vzbackup-rclone/vzbackup-rclone.sh rehydrate

會把上面的備份都下載下來,建議未來有需要時再手動下載就好,我覺得沒有很好用

 


See also

comments powered by Disqus