Question: Short question: I have two dynamically generated tar archives (so they have different timestamps), how can I compare them, ignoring any different in time?

Backgrounds...

I am doing some backup, in which I use a script to generate things that needs to be backed up, put them in to a directory, then tar the directory and keep several old versions. The backup script needs to run every 30 minutes to make sure we don't lose hours of work.

Now I realize that there are periods of time that the data doesn't change, so it doesn't make sense to store duplicates of the same thing over and over again. I would like to compare the archives before saving. My attempt was to run `cmp newdata.tar.gz olddata.tar.gz` and only store `newdata.tar.gz` if it contains new data. Apparently that didn't work, because there are different timestamps.

Answer: 
Here are three approaches. The first has my preference.

### Change the file generation process

Instead of regenerating the files indiscriminately (`dump_table foo >foo.dump`), keep the old file if it's identical to the new one.

```
dump_table foo >foo.dump.new
if cmp foo.dump foo.dump.new; then
  rm foo.dump.new
else
  mv foo.dump.new foo.dump
fi

```

### Mount the archives as directories

Use [AVFS](http://sourceforge.net/projects/avf/) to create a view of your entire directory hierarchy where all archives have an associated directory (same name with `#` tacked on at the end) that appears to hold the archive content.

```
mountavfs
d=$(date -d %Y%m%d%H%M%S); mkdir $d; tar czf /path/to/$d.tgz $d; rmdir $d
cd /path/to/back/up
rsync -ac --compare-dest=~/.avfs/path/to/yesterday.tgz . ~/.avfs/path/to/$d.tgz

```

### Use a backup tool with the requisite feature

Backup programs need to be extremely reliable. It's hard for a home-grown solution to cover all cases. So consider a full-blown backup program. I think [duplicity](http://www.nongnu.org/duplicity/) would do what you require: it performs incremental backups, and it uses the rsync algorithm, so it shouldn't waste any space on data that's already there.
Try this, then:

```
OLDSUM=`tar zxOf olddata.tar.gz | md5sum | awk '{print $1}'`
NEWSUM=`tar zxOf newdata.tar.gz | md5sum | awk '{print $1}'`
if test "${NEWSUM}" != "${OLDSUM}"
then
  echo save newdata.tar.gz
else
  echo nothing changed
fi

```

The capital `O` extracts the archive contents to `stdout`.

It will not work, however, unless the files are added to the archive in the exact same order (which they probably are, though).