[LON-CAPA-admin] notes on backing up and restoring

[Scott Harrison]

Dear All:

Caveat Admin, this could happen to you:

> **** WARNING: Last week I had tried to tar up /home/httpd/lonUsers on
> ... - THIS FAILED!!! Without much of an error message,
> tar threw an "unrecoverable error" that the tar-file was too big. This was NOT
> for lack of disk space, but appears to be a restriction of tar-files. A tar
> file was actually generated, but it was INCOMPLETE.

Minor issue:
cpio is a better tool for large backups in terms of verification,
performance, etc.  (But in the long term, tar and cpio will become
one command according to the GNU folks.)

Larger issue:
many filesystem formats only support 2gb or 4gb file sizes;
however, cpio and tar often utilize C library functions which only support
2gb.

As we (or at least I) work on a formal backup/restore strategy for
LON-CAPA servers, two thoughts come to mind:

1. CRC/checksum/verify the backup files IMMEDIATELY after generation
2. Irrespective of 2gb/4gb file size limitations, there is a practical
   issue that large files are at times "just too bulky".
   I might suggest that backup archives consist of multiple
   large-yet-small files.  For instance, perhaps we generate
   multiple archive files which are <750Mb; these multiple archive
   files simple contain mutually exclusive sets of different files
   related to the same backup attempt.

   The advantage here might allow for speedily backing up any LON-CAPA server
   so long as it had 750Mb of free disk space (the archive files could
   then be piped to another server during generation).

Regards,
Scott

-- 
Scott Harrison, sharrison at users.sourceforge.net