[LON-CAPA-admin] notes on backing up and restoring

Scott Harrison sharrison at users.sourceforge.net
Thu Nov 14 11:25:36 EST 2002

Dear All:

Caveat Admin, this could happen to you:

> **** WARNING: Last week I had tried to tar up /home/httpd/lonUsers on
> ... - THIS FAILED!!! Without much of an error message,
> tar threw an "unrecoverable error" that the tar-file was too big. This was NOT
> for lack of disk space, but appears to be a restriction of tar-files. A tar
> file was actually generated, but it was INCOMPLETE.

Minor issue:
cpio is a better tool for large backups in terms of verification,
performance, etc.  (But in the long term, tar and cpio will become
one command according to the GNU folks.)

Larger issue:
many filesystem formats only support 2gb or 4gb file sizes;
however, cpio and tar often utilize C library functions which only support

As we (or at least I) work on a formal backup/restore strategy for
LON-CAPA servers, two thoughts come to mind:

1. CRC/checksum/verify the backup files IMMEDIATELY after generation
2. Irrespective of 2gb/4gb file size limitations, there is a practical
   issue that large files are at times "just too bulky".
   I might suggest that backup archives consist of multiple
   large-yet-small files.  For instance, perhaps we generate
   multiple archive files which are <750Mb; these multiple archive
   files simple contain mutually exclusive sets of different files
   related to the same backup attempt.

   The advantage here might allow for speedily backing up any LON-CAPA server
   so long as it had 750Mb of free disk space (the archive files could
   then be piped to another server during generation).


Scott Harrison, sharrison at users.sourceforge.net

More information about the LON-CAPA-admin mailing list