restarting wget?

[Michael D. Crawford]

If wget fails partway through recursively downloading a big website, is there
some way to start it over again, to have it download the rest of the site
without downloading everything a second time?

What I envision is for wget to see that I already have some HTML files in its
download directory, and to follow the links in them, rather than fetching the
files from the website. Only if it comes across a file that has not already
been downloaded would it get the file from the web.

I've experimented with lots of wget's command line options, and it seems to me
like it ought to be able to do this, but I haven't been able to get it to work.

Alternatively, is there another recursive website downloader that can do what I
want? I just want something I can restart in the event of failure.

Thanks,

Mike
--
Michael D. Crawford
GoingWare Inc. - Expert Software Development and Consulting
[url]http://www.goingware.com[/url]
[email]crawford@goingware.com[/email]

Tilting at Windmills for a Better Tomorrow.

"I give you this one rule of conduct. Do what you will, but speak
out always. Be shunned, be hated, be ridiculed, be scared,
be in doubt, but don't be gagged."
-- John J. Chapman, "Make a Bonfire of Your Reputations"
[url]http://www.goingware.com/reputation/[/url]


--
To UNSUBSCRIBE, email to [email]debian-user-request@lists.debian.org[/email]
with a subject of "unsubscribe". Trouble? Contact [email]listmaster@lists.debian.org[/email]

[Johannes Zarl]

On Sunday 03 August 2003 11:39, Michael D. Crawford wrote:

> If wget fails partway through recursively downloading a big website, is
> there some way to start it over again, to have it download the rest of
> the site without downloading everything a second time?
>

Perhaps the -m (--mirror) option can help you.
From man wget:
--mirror
Turn on options suitable for mirroring. This option turns on recursion
and time-stamping, sets infinite recursion depth and keeps FTP directory
listings. It is currently equivalent to -r -N -l inf -nr.

The part relevant to you should be the '-N -nr'. Haven't tried it, though..

HTH,
Johannes

--
"More than machinery we need humanity" -- Charlie Chaplin, The Great
Dictator


--
To UNSUBSCRIBE, email to [email]debian-user-request@lists.debian.org[/email]
with a subject of "unsubscribe". Trouble? Contact [email]listmaster@lists.debian.org[/email]