Our digital division has been working on PC and server remote backups for some time now. They have excellent services and provide remote, fully distributed, mirrored computer backup solutions for small and large organizations.
Very impressive technology. The concept of per-file backups has been done before but it was the first time I had been exposed to the concept and it started me thinking of how to improve my current application. Yes, I know, I am re-inventing the wheel with this and others have done this before. But backups are important and I had particular requirements that I didn't see solved anywhere else.
So I reorganized my backup scripts to treat each file separately instead of a tar-ball. Any file can be a part of any number of backups. The files are stored compressed in a directory hierarchy by their digest signature default is SHA and organized using a SQL database. There is one database row per file per backup so if a file exists in 10 different backups, it has 10 different rows in the file table. The Perl backup script walks your filesystem using the File::Find package and backs up each file into the content directory.
If a previous version of the same file exists then all it needs to do is add a line for it in the file database table. Other Perl dependencies are:.
The scripts also depend on these more standard Perl packages:. All of the standard files are compressed into a directory hierarchy. The directory is specified to all of the scripts with the -c flag. When the file is backed up, its digest is first calculated and looked up in the database to see if it exists already. If it does then an entry is written into the files database table -- nothing is written to the content directory. If a file doesn't exist then it is compressed into the content directory.
To avoid race conditions it is read again, a new digest is calculated, and it is compressed into a file. Once this is done, the compressed file is moved into the directory hierarchy based on the new digest. If the file's digest is:. More levels would have been overkill with most being unpopulated and less levels would have resulted in s of files in the lowest directory which would have been less optimal for most filesystems. NOTE: Only standard files are stored in the content hierarchy.
Special files which include directories, devices, zero-length files, and symbolic links are recorded in the files database table. For my backup database, I run a Postgres instance on the backup drive on a different port under my UID. This allows me to run my main Postgres database servicing all of my webaps on my main drive and my database for the backup files on a the backup drive without getting them confused.
There are 2 tables in my backup database:. The backups table records each backup instance. This ID is used in the "files" table to assign a file to a specific backup. It also has a comment field in case you need to better describe a backup. The files table contains an entry for each file stored in each backup. If a file with the same exact content digest has been backed up 20 times over the years, it will have one entry in the content hierarchy but 20 entries in the files table. The files table records the type of file see below , full path, backup ID, linkpath if a symlink , size, modes, modification times, node information if a device , owner, group, and content digest for normal files only.
There are 5 different types of files:. This is useful for rotating files that were not created by this module. Makes the backup -- creates the backed up directory and archive file.
If it is a string, it is expected to be the path to a directory that is to be backed up. If the element is an array, the first element is expected to be a directory that is to be backed up, and the second should be the name the directory is called once it has been copied to the backup directory.
Rotates the backup directories and archive files. The number of archive files to keep and the number of directories to keep are specified in the new constructor. Set the path to the compression program you want to use when creating the archive files in the call to backup. The given compression program must provide the same API as gzip and bzip2, at least to the extent that it will except input from stdin and will write output to stdout when no file names are provided.
This defaults to 'bzip2' no explicit path. This sets the extension given to the archive name after the. This defaults to. For more information on module installation, please visit the detailed CPAN module installation guide.
Collectives on Stack Overflow. Learn more. How to use Perl to Backup Host file? Asked 9 years, 2 months ago. Active 9 years, 1 month ago. Viewed times. Improve this question. LordRazon LordRazon 91 7 7 bronze badges. Using the backtick operator in void context isn't wrong, but a poor choice. Backticks are for capturing a command's output. Use system instead. Add a comment.
Active Oldest Votes.
0コメント