Question: Short question: I have two dynamically generated tar archives (so they have different timestamps), how can I compare them, ignoring any different in time?

Backgrounds...

I am doing some backup, in which I use a script to generate things that needs to be backed up, put them in to a directory, then tar the directory and keep several old versions. The backup script needs to run every 30 minutes to make sure we don't lose hours of work.

Now I realize that there are periods of time that the data doesn't change, so it doesn't make sense to store duplicates of the same thing over and over again. I would like to compare the archives before saving. My attempt was to run `cmp newdata.tar.gz olddata.tar.gz` and only store `newdata.tar.gz` if it contains new data. Apparently that didn't work, because there are different timestamps.

Answer: 
Here are three approaches. The first has my preference.

### Change the file generation process

Instead of regenerating the files indiscriminately (`dump_table foo >foo.dump`), keep the old file if it's identical to the new one.

```
dump_table foo >foo.dump.new
if cmp foo.dump foo.dump.new; then
  rm foo.dump.new
else
  mv foo.dump.new foo.dump
fi

```

### Mount the archives as directories

Use [AVFS](http://sourceforge.net/projects/avf/) to create a view of your entire directory hierarchy where all archives have an associated directory (same name with `#` tacked on at the end) that appears to hold the archive content.

```
mountavfs
d=$(date -d %Y%m%d%H%M%S); mkdir $d; tar czf /path/to/$d.tgz $d; rmdir $d
cd /path/to/back/up
rsync -ac --compare-dest=~/.avfs/path/to/yesterday.tgz . ~/.avfs/path/to/$d.tgz

```

### Use a backup tool with the requisite feature

Backup programs need to be extremely reliable. It's hard for a home-grown solution to cover all cases. So consider a full-blown backup program. I think [duplicity](http://www.nongnu.org/duplicity/) would do what you require: it performs incremental backups, and it uses the rsync algorithm, so it shouldn't waste any space on data that's already there.
Do the files inside the archives change, or are just new ones added?

If you only add files, and don't change any, try something like this:

```
tar ztf olddata.tar.gz | sort > tmpfile1
tar ztf newdata.tar.gz | sort > tmpfile2
diff tmpfile{1,2} > /dev/null
if test $? -eq 0
then
  echo ignore newdata.tar.gz
else
  echo keep olddata.tar.gz
fi
rm tmpfile{1,2}

```

Note the absence of `v` from the tar table of contents commands.

If you just want to skip archives with identical contents, add a `v` to the mix, as in:

```
tar ztvf olddata.tar.gz | sort > tmpfile1

```

and so on.