I like to keep logs and since space is cheap, why not just keep 2 or 3, or maybe 10 years of logs? That’s all fine, but soon the logging directories will become cluttered. Well, there are plenty of ways to keep everything tidy and clean, but even a separate backup isn’t very suitable for me, as backups tend to cycle quicker, and I do want to keep my logs a lot longer. So, how can we go about moving anything older than a certain date into a subdirectory of our logs and let it sit there?

One way I like is a combination of find and rsync.

cd /var/logs/
find .                                                 \ 
     -name BACKUP -prune                               \ 
     -o                                                \ 
     -newermt 20080101 ! -newermt 20101231             \ 
     -type f                                           \ 
     -exec                                             \ 
     rsync -avR --remove-source-files {} ./BACKUP/ \;

It’s quite simple really, ahem… once you know it, but lets decompose the command and try to understand what each part of the command does:

  1. find . — Search into the current directory
  2. -name BACKUP -prune — exclude files/directories matching the pattern BACKUP
  3. -o — expr1 -o expr2 means if expr1 is true do not evaluate expr2, hence whichever file/dir does not match BACKUP will be subjected to the actions of the options next
  4. -newermt 20080101 ! -newermt 20101231 — the modification of the files is between 20080101 and 20101231
  5. -type f — process files only
  6. -exec rsync — execute the following command, in our case rsync. {} is the matched file
  7. -avR --remove-source-files — These are options to rsync!!
    • -a — archive
    • -v — verbose
    • -R — recursive
    • --remove-source-files — delete files once copied
  8. {} ./BACKUP/ — since {} is the filename matched, this is the file that will be copied to the destination ./BACKUP/

If you want to exclude more subdirectories/files, except BACKUP, then you could change step 2 above with:

  1. \( -name BACKUP -o -name mysql \) -prune — this will exclude both BACKUP and mysql
0