amanda仮想テープでバックアップ/リストア

※実際には8/18のネタです。


■テープ情報の指定について

$ man amanda.conf | grep -A 8 "tapetype string"
tapetype string
Default: EXABYTE. The type of tape drive associated with tapedev or
tpchanger. This refers to one of the defined tapetypes in the
config file (see below), which specify various tape parameters,
like the length, filemark size, and speed of the tape media and
device.

First character of a tapetype string must be an alphabetic
character

■設定には、length 5 gbytes を指定してある。

# grep -A 8 "^tapetype" /etc/amanda/DailySet1/amanda.conf
tapetype DISK
tpchanger "chg-disk"
changerfile "/etc/amanda/DailySet1/changer" # needed by amlabel
tapedev "file:/space/vtapes/DailySet1/slots"

define tapetype DISK {
comment "Backup to HD"
length 5 gbytes
}

■既読になった最後のメールの取り出し

$ grep "Status: RO
" mbox -A 100 > report.log
$ tail -101 report.log | less

$ tail -101 report.log | grep "STATISTICS" -A 23
STATISTICS:
Total Full Incr.
-------- -------- --------
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:07
Dump Time (hrs:min) 0:05 0:00 0:05
Output Size (meg) 3461.7 0.0 3461.7
Original Size (meg) 3490.8 0.0 3490.8
Avg Compressed Size (%) 99.2 -- 99.2 (level:#disks ...)
Filesystems Dumped 1 0 1 (2:1)
Avg Dump Rate (k/s) 11180.7 -- 11180.7

Tape Time (hrs:min) 0:01 0:00 0:01
Tape Size (meg) 3461.7 0.0 3461.7
Tape Used (%) 67.6 0.0 67.6 (level:#disks ...)
Filesystems Taped 1 0 1 (2:1)

Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) 48143.0 -- 48143.0

USAGE BY TAPE:
Label Time Size % Nb Nc
DailySet1-02 0:01 3544768k 67.6 1 02^10=1000計算だと70%の使用率。

$ let x=3544768/50000;echo $x
702^10=1024計算だと67%の使用率。

$ let x=1024*1024*5;echo $x
5242880
$ let y=3544768/52428;echo $y
67

⇒5GBを基準にしていることは間違いないようだ。

■ファイル単位、またはディレクトリ単位でバックアップを取る事を考える。
 いずれも5GB以下。

※仮想テープサイズ以下のパーティションに分ける事も視野に入れるが、
 これは「/」でテストし、出来る事は分かっているので、最終手段とする。

$ df -h | head -1 ;df -h | grep /home
Filesystem サイズ 使用 残り 使用% マウント位置
/dev/sda3 870G 105G 721G 13% /home

$ du -hs /home/username/vmware/debianlenny
1.4G /home/username/vmware/debianlenny

$ du -hs /home/username/vmware/debianlenny/debianlenny.vmdk
918M /home/username/vmware/debianlenny/debianlenny.vmdk

■バックアップの確認

$ /usr/sbin/amadmin DailySet1 find | grep -v FAILED

date host disk lv tape or file file part status
2011-08-18 localhost / 2 DailySet1-01 1 -- OK
2011-08-18 localhost / 2 DailySet1-02 1 -- OK
2011-08-18 localhost / 1 DailySet1-08 1 -- OK
2011-08-18 localhost / 1 DailySet1-09 1 -- OK
2011-08-18 localhost / 0 DailySet1-07 1 -- OK

$ /usr/sbin/amadmin DailySet1 find | grep FAILED | sed s/"\["/"\n&"/g 
2011-08-18 localhost /home 0 DailySet1-01 0 -- FAILED (planner)
[dump larger than available tape space, 54170335 KB, but cannot incremental dump new disk]
2011-08-18 localhost /home 0 DailySet1-02 0 -- FAILED (planner)
[dump larger than available tape space, 54170335 KB, but cannot incremental dump new disk]
2011-08-18 localhost /home 0 DailySet1-03 0 -- FAILED (planner)
[dump larger than available tape space, 54170335 KB, but cannot incremental dump new disk]
2011-08-18 localhost /home 0 DailySet1-04 0 -- FAILED (planner)
[dump larger than available tape space, 54544735 KB, but cannot incremental dump new disk]
2011-08-18 localhost /home 0 DailySet1-05 0 -- FAILED (planner)
[dump larger than available tape space, 54544735 KB, but cannot incremental dump new disk]
2011-08-18 localhost /home 0 DailySet1-06 0 -- FAILED (planner)
[dump larger than available tape space, 54544735 KB, but cannot incremental dump new disk]

■アクセス権限の変更
 ※amrecoverにFQDNでログイン後にsethost localhostの移動が必要なのは困るのでFQDNで統一。

# sudo cat /var/backups/.amandahosts
full.domain.name root amindexd amidxtaped
full.domain.name backup amdump
full.domain.name root amindexd amidxtaped

■現状の状態

$ /usr/sbin/amrecover
amrecover> sethost localhost
200 Dump host set to localhost.
amrecover> listdisk
200- List of disk for host localhost
201- /
201- /home
200 List of disk for host localhost
amrecover> exit
200 Good bye.
■除外と取込の違い

$ man amanda.conf | grep "exclude \[" -A 26
exclude [ list|file ][[optional][ append ][ string ]+]
Default: file. There are two exclude lists, exclude file and
exclude list. With exclude file , the string is a GNU-tar exclude
expression. With exclude list , the string is a file name on the
client containing GNU-tar exclude expressions. The path to the
specified exclude list file, if present (see description of
'optional' below), must be readable by the Amanda user.

All exclude expressions are concatenated in one file and passed to
GNU-tar as an --exclude-from argument.

Exclude expressions must always be specified as relative to the
head directory of the DLE.

With the append keyword, the string is appended to the current
list, without it, the string overwrites the list.

If optional is specified for exclude list, then amcheck will not
complain if the file doesn't exist or is not readable.

For exclude list, if the file name is relative, the disk name being
backed up is prepended. So if this is entered:

exclude list ".amanda.excludes"
the actual file used would be /var/.amanda.excludes for a backup of
/var, /usr/local/.amanda.excludes for a backup of /usr/local, and
so on.

$ man amanda.conf | grep -A 25 "include \["
include [ list|file ][[optional][ append ][ string ]+]
Default: file ".". There are two include lists, include file and
include list. With include file , the string is a glob expression.
With include list , the string is a file name on the client
containing glob expressions.

All include expressions are expanded by Amanda, concatenated in one
file and passed to GNU-tar as a --files-from argument. They must
start with "./" and contain no other "/".

Include expressions must always be specified as relative to the
head directory of the DLE.

Note
For globbing to work at all, even the limited single level, the top
level directory of the DLE must be readable by the Amanda user.

With the append keyword, the string is appended to the current
list, without it, the string overwrites the list.

If optional is specified for include list, then amcheck will not
complain if the file doesn't exist or is not readable.

For include list, If the file name is relative, the disk name being
backed up is prepended.
■取込用の設定を追加

# grep include /etc/amanda/DailySet1/amanda.conf
include list "/etc/amanda/DailySet1/global-debian-include.list"

★バックアップに取り込むディレクトリ/ファイル
 ※ディスクパス「/home」からの相対パスに「./」を付けて記述する

$ cat /etc/amanda/DailySet1/global-debian-include.list
./username/vmware/debianlenny/debianlenny.vmdk

★バックアップから除外するディレクトリ/ファイル
 ※ディスクパス「/home」からの相対パスに「./」を付けて記述する

$ cat /etc/amanda/DailySet1/global-debian-exclude.list
cat /etc/amanda/DailySet1/global-debian-exclude.list | sed s/labunix/username/g
./lost+found
./username/iso_system
./username/mail
./username/xorgtest

■バックアップを実行する

$ su backup
$ /usr/sbin/amdump DailySet1

■バックアップが出来ていることを確認する

$ su root
# /usr/sbin/amrecover
AMRECOVER Version 2.5.2p1. Contacting server on ibm-amddebian ...
220 fqdn AMANDA index server (2.5.2p1) ready.
Setting restore date to today (2011-08-18)
200 Working date set to 2011-08-18.
200 Config set to DailySet1.
501 Host fqdn is not in your disklist.
Trying host fqdn.domain.name ...
200 Dump host set to fqdn.domain.name.
Use the setdisk command to choose dump disk to recover
amrecover> listdisk
200- List of disk for host fqdn.domain.name
201- /home
200 List of disk for host fqdn.domain.name
amrecover> setdisk /home /tmp
200 Disk set to /home.
amrecover> pwd
/tmp
amrecover> ls
2011-08-18 username/
amrecover> cd username
/tmp/username
amrecover> ls
2011-08-18 vmware/
amrecover> cd vmware
/tmp/username/vmware
amrecover> ls
2011-08-18 debianlenny/
amrecover> cd debianlenny
/tmp/username/vmware/debianlenny
amrecover> ls
2011-08-18 debianlenny.vmdk
amrecover> exit

※ディレクトリ、ファイル、共に問題なくバックアップが出来るようになった。
 メール通知は、username宛てに送信される。
 対象はディレクトリに変更。
 ⇒課題はCLOSE。

■仮想テープをリセットする
 ※関連ファイルを削除、クリアして、ラベルを付け直す。

$ su backup
$ ls -A /dumps/
$ rm -fr /dumps/*
$ rm -fr /etc/amanda/DailySet1/index/*
$ rm -fr /etc/amanda/DailySet1/logs/*
$ rm -fr /etc/amanda/DailySet1/curinfo/*
$ rm /etc/amanda/DailySet1/chan*
$ rm /etc/amanda/DailySet1/tapelist.*

$ ls /etc/amanda/DailySet1/tapelist | xargs less
$ echo "" > /etc/amanda/DailySet1/tapelist

$ ls /space/vtapes/DailySet1/slots/slot*/*
$ rm -fr /space/vtapes/DailySet1/slots/slot*/*
$ ls -l /space/vtapes/DailySet1/slots/slot* | grep "合計" | uniq -c
9 合計 0

$ ls -l /var/lib/amanda/gnutar-lists/*
$ rm -fr /var/lib/amanda/gnutar-lists/*

$ ls /var/lib/amanda/amandates | xargs cat
$ echo "" > /var/lib/amanda/amandates

$ cd /space/vtapes/DailySet1/slots
$ ln -s slot1 data
$ /usr/sbin/ammt -f file:/space/vtapes/DailySet1/slots status
$ $ for i in `seq 1 9`; do /usr/sbin/amlabel DailySet1 DailySet1-0$i slot $i; done

$ /usr/sbin/amtape DailySet1 show
$ /usr/sbin/amtape DailySet1 reset
$ /usr/sbin/amtape DailySet1 show

■バックアップを実行します。

$ /usr/sbin/amcheck DailySet1
$ /usr/sbin/amdump DailySet1


$ env EDITOR=vim crontab -e
$ crontab -l
# m h dom mon dow command
0 1 * * 1-5 /usr/sbin/amdump DailySet1

⇒1回目がフル、2回目が増分になっていればOK。

■リストアしてみる

# mkdir /tmp/restore
# chown backup:backup /tmp/restore
# amrecover

amrecover> listdisk
200- List of disk for host ibm-amddebian.example.local
201- /home
200 List of disk for host ibm-amddebian.example.local
amrecover> setdisk /home
200 Disk set to /home.
amrecover> pwd
/home
amrecover> cd username/vmware/debianlenny
/home/username/vmware/debianlenny
amrecover> ls
2011-08-18 .
2011-08-18 vmware.log
2011-08-18 vmware-0.log
2011-08-18 debianlenny.vmxf
2011-08-18 debianlenny.vmx
2011-08-18 debianlenny.vmss
2011-08-18 debianlenny.vmsd
2011-08-18 debianlenny.vmem
2011-08-18 debianlenny.vmdk
2011-08-18 debianlenny.nvram
amrecover> cd ..
/home/username/vmware
amrecover> lcd /tmp/restore/
amrecover> add debianlenny
Added dir /username/vmware/debianlenny/ at date 2011-08-18
Added dir /username/vmware/debianlenny/ at date 2011-08-18
amrecover> expire
Invalid command: expire
amrecover> extract

Extracting files using tape drive chg-disk on host ibm-amddebian.
The following tapes are needed: DailySet1-09
DailySet1-01
Restoring files into directory /tmp/restore
Continue [?/Y/n]? Y

Extracting files using tape drive chg-disk on host ibm-amddebian.
Load tape DailySet1-09 now
Continue [?/Y/n/s/t]? Y
Label mismatch, got DailySet1-02 and expected DailySet1-09
Looking for tape DailySet1-09...
./username/vmware/debianlenny/
./username/vmware/debianlenny/debianlenny.nvram
./username/vmware/debianlenny/debianlenny.vmdk
./username/vmware/debianlenny/debianlenny.vmem
./username/vmware/debianlenny/debianlenny.vmsd
./username/vmware/debianlenny/debianlenny.vmss
./username/vmware/debianlenny/debianlenny.vmx
./username/vmware/debianlenny/debianlenny.vmxf
./username/vmware/debianlenny/vmware-0.log
./username/vmware/debianlenny/vmware.log
Extracting files using tape drive chg-disk on host ibm-amddebian.
Load tape DailySet1-01 now
Continue [?/Y/n/s/t]? Y
Label mismatch, got DailySet1-09 and expected DailySet1-01
Looking for tape DailySet1-01...
./username/vmware/debianlenny/
amrecover> exit
200 Good bye.

■チェックサムも同じ状態で別ディレクトリに修復されました。

# ls /tmp/restore/username/vmware/debianlenny/*
# for list in /tmp/restore/username/vmware/debianlenny/*;do md5sum "$list";done
144083f145618b16c22ea79e7586be85 /tmp/restore/username/vmware/debianlenny/debianlenny.nvram
5fb44098e0025e56f7accac9bb915e5a /tmp/restore/username/vmware/debianlenny/debianlenny.vmdk
bf7d9764433a9053277090201b9b3a4b /tmp/restore/username/vmware/debianlenny/debianlenny.vmem
d41d8cd98f00b204e9800998ecf8427e /tmp/restore/username/vmware/debianlenny/debianlenny.vmsd
ede1767228d636ba5cab21e9b8802326 /tmp/restore/username/vmware/debianlenny/debianlenny.vmss
b24e4a9d97ad15b08e2c736e46ce1a46 /tmp/restore/username/vmware/debianlenny/debianlenny.vmx
a1c7d0f5b455d53fbfd14de0a55e0d3d /tmp/restore/username/vmware/debianlenny/debianlenny.vmxf
ac8949eefef8d9804aeaaf2680ba270a /tmp/restore/username/vmware/debianlenny/vmware-0.log
dec4e9c1a17b87caccbfda891349b314 /tmp/restore/username/vmware/debianlenny/vmware.log

$ cd ~/;for next in vmware/debianlenny/*;do /usr/bin/md5sum "$next";done
144083f145618b16c22ea79e7586be85 vmware/debianlenny/debianlenny.nvram
5fb44098e0025e56f7accac9bb915e5a vmware/debianlenny/debianlenny.vmdk
bf7d9764433a9053277090201b9b3a4b vmware/debianlenny/debianlenny.vmem
d41d8cd98f00b204e9800998ecf8427e vmware/debianlenny/debianlenny.vmsd
ede1767228d636ba5cab21e9b8802326 vmware/debianlenny/debianlenny.vmss
b24e4a9d97ad15b08e2c736e46ce1a46 vmware/debianlenny/debianlenny.vmx
a1c7d0f5b455d53fbfd14de0a55e0d3d vmware/debianlenny/debianlenny.vmxf
ac8949eefef8d9804aeaaf2680ba270a vmware/debianlenny/vmware-0.log
dec4e9c1a17b87caccbfda891349b314 vmware/debianlenny/vmware.log

■リストア後もフルと増分のバックアップが残ってます。

$ /usr/sbin/amadmin DailySet1 find
date host disk lv tape or file file part status
2011-08-18 ibm-amddebian.example.local /home 1 DailySet1-01 1 -- OK
2011-08-18 ibm-amddebian.example.local /home 0 DailySet1-09 1 -- OK