URL checking

[A. Sinan Unur]

[email]mandy100@ihug.com.au[/email] (Mandy) wrote in news:6522b540.0306281641.4bbf5b23
@posting.google.com:

> Hi.
> I can't seem to find any examples that will help me learn how to check
> 2 things using a perl script.
>
> 1. Does a given page exist on a remote site
> e.g. If i want to check that my homepage's pictures.html page exists.

see [url]http://www.perldoc.com/perl5.8.0/lib/LWP.html#An-Example[/url]

> 2. Does a page contain a url that I specify
> e.g. does my pictures.html page contain a link to my diary.html page

HTML::Parser is your friend.

--
A. Sinan Unur
[email]asu1@c-o-r-n-e-l-l.edu[/email]
Remove dashes for address
Spam bait: mailto:uce@ftc.gov

[Tad McClellan]

Mandy <mandy100@ihug.com.au> wrote:

> 1. Does a given page exist on a remote site

Try and "fetch" it, if you get a good-looking response, then
the resource exists.

> 2. Does a page contain a url that I specify

^^^
^^^

> I know there must be examples out there but my choice of keywords
> hasn't been too productive yet.

Errr, OK.


perldoc -q fetch

How do I fetch an HTML file?

perldoc -q url

How do I extract URLs?


--
Tad McClellan SGML consulting
[email]tadmc@augustmail.com[/email] Perl programming
Fort Worth, Texas

[Mark Jason Dominus]

In article <6522b540.0306281641.4bbf5b23@posting.google.com >,
Mandy <mandy100@ihug.com.au> wrote:

>Hi.
>I can't seem to find any examples that will help me learn how to check
>2 things using a perl script.
>
>1. Does a given page exist on a remote site
>e.g. If i want to check that my homepage's pictures.html page exists.

I usually use the LWP::Simple module for that. First install the
module by running the command:

perl -MCPAN -e 'install LWP::Simple'


Once you have LWP::Simple, you can use it to check for a page this
way:

use LWP::Simple;

if (head($url)) {
# it seems to exist
} else {
# not
}

>2. Does a page contain a url that I specify
>e.g. does my pictures.html page contain a link to my diary.html page

Similarly, you can use LWP::Simple to fetch the page's content, and
then search through it looking for the link. Try

use LWP::Simple;
use HTML::LinkExtor; # "Link Extractor"

my $content = get($url);
if (defined $content) {
my $found = 0;
HTML::LinkExtor->new(sub { my $tag, %links = @_;
for (values %links) {
$found = 1 if $_ eq "diary.html";
}
}, $url)
->parse($content);
if ($found) {
# found it
} else {
# didn't find it
}
}



Good luck.