Web Spider Component

[wiseduck]

Does anyone know of a good web spidering component available to crawl and extract llinks in coldfusion?

[mpwoodward *TMM*]

wiseduck wrote:

> Does anyone know of a good web spidering component available to crawl and extract llinks in coldfusion?

Do you mean for your own local sites or for remote sites? If local, CF
comes with Verity spider--google for that and you'll find some info. If
you mean external sites CF doesn't do this natively so you'd have to
roll your own by using CFHTTP. A few folks have done this so again
google is your friend. ;-)

Matt

--
Matt Woodward
[email]mpwoodward@gmail.com[/email]
Team Macromedia - ColdFusion

[wiseduck]

No I mean remote sites, not verity...

I need a crawler that I can point to an initial site, and will then crawl
through it and collect the links that are only within that host .....

I am aware that I may have to build my own, but I was wondering If anyone can
reccomend something already out there?

[BKBK]

>... I am aware that I may have to build my own
No need to re-invent the wheel

>... I was wondering If anyone can reccomend something already out there?

Oh, yeah. It doesn't get better than this one, by Xavier Roche. It's free, too.

[BKBK]

[url]http://www.httrack.com[/url]

It's Open Source, so you can also get the source code,
if you want.