Incremental backups?

[Ryan Ashline]

I am trying to set up Red Hat Linux to use as a file server. I have
used Windows in the past with a program called Dantz Retrospect for
incremental backups to another hard drive in the server. Everything I
have read so far deals with booting up and making a full back up of
the server. What software (less than $75.00) is availible to backup my
Linux File Server to a Hard drive on the same Server.

Thanks for your help
Ryan

[Peter T. Breuer]

In comp.os.linux.help Ryan Ashline <ashline@yahoo.com> wrote:

> the server. What software (less than $75.00) is availible to backup my
> Linux File Server to a Hard drive on the same Server.

You don't need any "software". Just use tar. Same as always.


Peter

[Nico Kadel-Garcia]

Peter T. Breuer wrote:

> In comp.os.linux.help Ryan Ashline <ashline@yahoo.com> wrote:
>

>>the server. What software (less than $75.00) is availible to backup my
>>Linux File Server to a Hard drive on the same Server.

>
>
> You don't need any "software". Just use tar. Same as always.

And once again Peter gets it dead wrong. Use "rsync": that way, you
don't wind up duplicating files that are already there, and you can far
more easily set it more gracefully to ignore certain files, manipulate
ownership, or delete files on the target system that have been deleted
locally. You can even bandwidth limit it trivially to reduce load on
your systems, unlike tar.

[johna@onevista.com]

In comp.os.linux.help Ryan Ashline <ashline@yahoo.com> wrote:

>the server. What software (less than $75.00) is availible to backup my
>Linux File Server to a Hard drive on the same Server.

I have some home written code that keeps up to ten generations of files.
It writes to a local disk and will backup an entire system. I have been using it
for some time to backup my NT and Linux systems. If you want to try it, it's
available at [url]http://www.onevista.com/delayed_mirror.c[/url]

It's not a real-time mirror. It runs from a cron job. Every two hours for
/home and /local. Every 24 hours for the rest of the system.

Documentation? Look at the source, try the -h option or send mail.

Of course, nothing beats tape. I use that as well but having online copies of
files that I have just destroyed is a life saver.

johna

[Peter T. Breuer]

In comp.os.linux.help Nico Kadel-Garcia <nkadel@verizon.net> wrote:

> Peter T. Breuer wrote:

> > In comp.os.linux.help Ryan Ashline <ashline@yahoo.com> wrote:
> >

> >>the server. What software (less than $75.00) is availible to backup my
> >>Linux File Server to a Hard drive on the same Server.

> >
> >
> > You don't need any "software". Just use tar. Same as always.

> And once again Peter gets it dead wrong. Use "rsync": that way, you

Look, kook, I don't go insulting you. Simply keep your insults to
yourself.

Yes, *I* use rsync. It certainly updates only an increment each time,
but it doesn't *produce* incremental backups, which is what the guy
wants, it produces a mirror. In particular, you can't rollback to a
given day with rsync.

So it is not what the guy means by "backup", and hence not the answer
for him.


Peter

[Nico Kadel-Garcia]

Peter T. Breuer wrote:

> In comp.os.linux.help Nico Kadel-Garcia <nkadel@verizon.net> wrote:
>

>>Peter T. Breuer wrote:

>
>

>>>In comp.os.linux.help Ryan Ashline <ashline@yahoo.com> wrote:
>>>
>>>
>>>>the server. What software (less than $75.00) is availible to backup my
>>>>Linux File Server to a Hard drive on the same Server.
>>>
>>>
>>>You don't need any "software". Just use tar. Same as always.

>
>

>>And once again Peter gets it dead wrong. Use "rsync": that way, you

>
>
> Look, kook, I don't go insulting you. Simply keep your insults to
> yourself.

OK, I'll let you insult yourself. No problem.

> Yes, *I* use rsync. It certainly updates only an increment each time,
> but it doesn't *produce* incremental backups, which is what the guy
> wants, it produces a mirror. In particular, you can't rollback to a
> given day with rsync.
>
> So it is not what the guy means by "backup", and hence not the answer
> for him.

Like I said, it only took you a few lines to get it dead wrong. He
didn't say backup to tape, or to a file: he said backup to disk.

Using tar for incremental backups does not *delete* files that have been
removed. Rsync does. If he's backing up to disk, he should use live disk
to do full file system backukps: that way he has an immediately
bootable OS image (maybe with an /etc/fstab or grub.conf edit) rather
than spending the laborious time doing sequential extractions from tar
archives.

If he needs sequential recover backups to extract things from the
archive, there's an old standard trick where you create a full mirror,
duplicate the mirror using hard links for all the files, and rsync to
the mirror of the mirror. You can get away with storing quite a few
incremental mirrors this way without using up significantly more disk
space, although the extra directories do use up some inodes: it's
similar to the NetApp trick where they have the ".backup" directory
hidden in each directory, where incremental read-only backups are
accessible.

[Mark Miksis]

Have you considered using rsync? Check out a great article at
[url]http://www.mikerubel.org/computers/rsync_snapshots/[/url]


[email]ashline@yahoo.com[/email] (Ryan Ashline) wrote in message news:<54109d7d.0309151423.218f41b8@posting.google. com>...

> I am trying to set up Red Hat Linux to use as a file server. I have
> used Windows in the past with a program called Dantz Retrospect for
> incremental backups to another hard drive in the server. Everything I
> have read so far deals with booting up and making a full back up of
> the server. What software (less than $75.00) is availible to backup my
> Linux File Server to a Hard drive on the same Server.
>
> Thanks for your help
> Ryan

[Doug Laidlaw]

Peter T. Breuer wrote:

> In comp.os.linux.help Nico Kadel-Garcia <nkadel@verizon.net> wrote:

>> Peter T. Breuer wrote:

>

>> > In comp.os.linux.help Ryan Ashline <ashline@yahoo.com> wrote:
>> >
>> >>the server. What software (less than $75.00) is availible to backup my
>> >>Linux File Server to a Hard drive on the same Server.
>> >
>> >
>> > You don't need any "software". Just use tar. Same as always.

>

>> And once again Peter gets it dead wrong. Use "rsync": that way, you

>
> Look, kook, I don't go insulting you. Simply keep your insults to
> yourself.
>
> Yes, *I* use rsync. It certainly updates only an increment each time,
> but it doesn't *produce* incremental backups, which is what the guy
> wants, it produces a mirror. In particular, you can't rollback to a
> given day with rsync.
>
> So it is not what the guy means by "backup", and hence not the answer
> for him.
>
>
> Peter

I don't want to enter the insulting match, but, Peter, "Just use 'tar'" is a
bit vague. Tar has to find what to back up. I use "backerupper," which I
found on Freshmeat. It does a full backup of user-defined
directories/files on the first of the month, and incremental backups in
between. It uses zip as the default, or bzip2 optionally. There is no
provision as it stands for deleting old backups, but it is just a
collection of scripts, so anyone with a bit more know-how than me could
probably arrange this. There is a line for a "final command" where it
could go. I use the final command to copy the zip file to my Windows
directory then shut down the computer.

Doug.
--
Registered Linux User No. 277548.
They say lightning never strikes twice in the same place. My typing is
about as accurate. Apologies for any typos that slip in. - Doug.

[Wayne Throop]

: Doug Laidlaw <laidlaws@myaccess.com.au>
: I don't want to enter the insulting match, but, Peter, "Just use
: 'tar'" is a bit vague. Tar has to find what to back up. I use
: "backerupper," which I found on Freshmeat. It does a full backup of
: user-defined directories/files on the first of the month, and
: incremental backups in between. It uses zip as the default, or bzip2
: optionally.

Yes, tar has to find what to back up. Both classic tar and GNU tar have
several parameters designed to do just that. I use GNU tar's
--listed-incremental.

`--listed-incremental=SNAPSHOT-FILE'
`-g SNAPSHOT-FILE'
During a `--create' operation, specifies that the archive that
`tar' creates is a new GNU-format incremental backup, using
SNAPSHOT-FILE to determine which files to backup. With other
operations, informs `tar' that the archive is in incremental
format.

along with a cron job to clean up the snapshot so that a full
backup is done monthly. More normal methods might involve

`--newer=DATE'
`--after-date=DATE'
`-N'
When creating an archive, `tar' will only add files that have
changed since DATE. If DATE begins with `/' or `.', it is taken
to be the name of a file whose last-modified time specifies the
date.

and so on and so forth. GNU tar is a normal GNU "kitchen sink"
tool, and can handle most tasks by careful selection of its
many options.


Wayne Throop [email]throopw@sheol.org[/email] [url]http://sheol.org/throopw[/url]

[Wayne Throop]

:: Peter T. Breuer
:: Yes, *I* use rsync. It certainly updates only an increment each
:: time, but it doesn't *produce* incremental backups, which is what the
:: guy wants, it produces a mirror. In particular, you can't rollback
:: to a given day with rsync. So it is not what the guy means by
:: "backup", and hence not the answer for him.

: Nico Kadel-Garcia <nkadel@verizon.net>
: If he needs sequential recover backups to extract things from the
: archive, there's an old standard trick where you create a full mirror,
: duplicate the mirror using hard links for all the files, and rsync to
: the mirror of the mirror.

Well... actually you want to rsync to a "latest" mirror, and then
at convenient intervals, snapshot that mirror with hard links.
If you next rsync to the duplicate of the mirror, you'll end up
copying some things that could be links.

Unless of course I'm misunderstanding what is meant by
"rsync to the mirror of the mirror".

But anyways, the basic notion is to use rsync to keep a mirror
up to date, and then use cp -al to create incremental snapshots.

It would be possible to use cp -al to make snapshots directly,
if they were on the same filesystem... but that's not very robust
against disk failures, and use of cp -al for this purpose depends
on all updates to the tree being cp -al'ed occur by unlink without
any overwrites. Fortuitously, rsync always unlinks, so that's
allright then.

On the other hand, I use rsync to create a mirror, and tar to
create a separate incrmental archive. The incremental archives can
be compressed, and if I got short of space, I could dispense with
the mirror and still have compressed tarballs.

On the other hand, use of rsync && cp -al might work well
with a compressed filesystem; there are several implementations
of those hanging around.

So as usual, there are zillions of ways to do it.


Wayne Throop [email]throopw@sheol.org[/email] [url]http://sheol.org/throopw[/url]

[Juha Siltala]

In article <54109d7d.0309151423.218f41b8@posting.google.com >, Ryan Ashline
wrote:

> I am trying to set up Red Hat Linux to use as a file server. I have
> used Windows in the past with a program called Dantz Retrospect for
> incremental backups to another hard drive in the server. Everything I
> have read so far deals with booting up and making a full back up of
> the server. What software (less than $75.00) is availible to backup my
> Linux File Server to a Hard drive on the same Server.

There are lots of good backup scripts that use basic Unix tools, and it's
not too hard to write another one if you know a bit of bash or perl. I
like Reoback enough not to bother thinking about it myself:
[url]http://reoback.sourceforge.net/[/url] . The price is well within your budget :)

--
Juha Siltala
[url]http://www.edu.helsinki.fi/activity/people/jsiltala/[/url]

[Nico Kadel-Garcia]

Wayne Throop wrote:

> :: Peter T. Breuer
> :: Yes, *I* use rsync. It certainly updates only an increment each
> :: time, but it doesn't *produce* incremental backups, which is what the
> :: guy wants, it produces a mirror. In particular, you can't rollback
> :: to a given day with rsync. So it is not what the guy means by
> :: "backup", and hence not the answer for him.
>
> : Nico Kadel-Garcia <nkadel@verizon.net>
> : If he needs sequential recover backups to extract things from the
> : archive, there's an old standard trick where you create a full mirror,
> : duplicate the mirror using hard links for all the files, and rsync to
> : the mirror of the mirror.
>
> Well... actually you want to rsync to a "latest" mirror, and then
> at convenient intervals, snapshot that mirror with hard links.
> If you next rsync to the duplicate of the mirror, you'll end up
> copying some things that could be links.
>
> Unless of course I'm misunderstanding what is meant by
> "rsync to the mirror of the mirror".

You described what I meant.

> But anyways, the basic notion is to use rsync to keep a mirror
> up to date, and then use cp -al to create incremental snapshots.

cp -al is precisely what I meant, thank you.

> It would be possible to use cp -al to make snapshots directly,
> if they were on the same filesystem... but that's not very robust
> against disk failures, and use of cp -al for this purpose depends
> on all updates to the tree being cp -al'ed occur by unlink without
> any overwrites. Fortuitously, rsync always unlinks, so that's
> allright then.

That's true. If you need *tape* backup, which the original poster did
not mention, though, this allows you to do the tape and chance of disk
writes during the backup off of the primary server. This is
*particularly* useful for systems that run databases: the database can
be built on the mirror using a specific snapshot data dump of the
original disk, rather than from a potentially "in the midst of an atomic
operation" file, as well.

> On the other hand, I use rsync to create a mirror, and tar to
> create a separate incrmental archive. The incremental archives can
> be compressed, and if I got short of space, I could dispense with
> the mirror and still have compressed tarballs.
>
> On the other hand, use of rsync && cp -al might work well
> with a compressed filesystem; there are several implementations
> of those hanging around.
>
> So as usual, there are zillions of ways to do it.

Yup. That's why I get so irritated at Peter's snapshot answers: they're
snappish, often pretend as if the questioner was an idiot for not
knowing the answer already, and often frankly wrong for the situation
the questioner described.

[Doug Laidlaw]

Nico Kadel-Garcia wrote:

> Peter T. Breuer wrote:
>

>> In comp.os.linux.help Ryan Ashline <ashline@yahoo.com> wrote:
>>

>>>the server. What software (less than $75.00) is availible to backup my
>>>Linux File Server to a Hard drive on the same Server.

>>
>>
>> You don't need any "software". Just use tar. Same as always.

>
> And once again Peter gets it dead wrong. Use "rsync": that way, you
> don't wind up duplicating files that are already there, and you can far
> more easily set it more gracefully to ignore certain files, manipulate
> ownership, or delete files on the target system that have been deleted
> locally. You can even bandwidth limit it trivially to reduce load on
> your systems, unlike tar.

With deference to Peter (and BTW thanks Peter for your explanation of tar)

I had /usr/src as the only partition on /dev/hdb, and I had space for it on
/dev/hda. I used rsync to move it, and everything rebooted like a charm.
The only error message was to the effect of "mount: /usr/src doesn't exist
as a mount point," but mtab showed that the mount had succeeded. The line
in fstab was identical to the previous one, with only the partition ID
changed. It was probably that I was trying to mount /usr/src before /usr,
but anyway, it worked. The real attraction of rsync was that it preserves
linking.

Doug.

--
Registered Linux User No. 277548.
"They" say lightning never strikes twice in the same place. My typing is
about as accurate. Apologies for any typos that slip in. - Doug.