Using HTTPS with LWP::UserAgent = Bad Service

[Richard Lawrence]

Hi all,

I'm trying to access [url]https://www.nodeworks.com/[/url] through a proxy which
requires a proxy username and password. The code I have is as follows:

#!/usr/bin/perl
use strict;
use LWP::UserAgent;
use Data::Dumper qw(Dumper);
# If they provide an argument, connect to https else http
my $url = "http://www.nodeworks.com";
$url = "https://www.nodeworks.com" if ($ARGV[0]);
print "Going to get $url\n";
my $ua = new LWP::UserAgent;
# According to Crypt::SSLeay:
# "Use of the HTTPS_PROXY environment variable in this way is
# similar to LWP::UserAgent->env_proxy() usage, but calling that
# method will likely override or break the Crypt::SSLeay support,
# so do not mix the two."
# so only define $ua->proxy when using http.
if ($url =~ /^https/i)
{
print "Using HTTPS proxy $ENV{HTTPS_PROXY}\n";
}
else
{
$ua->env_proxy();
print "Using HTTP proxy $ENV{HTTP_PROXY}\n";
}
$ua->timeout(15);
my $req = new HTTP::Request('HEAD', $url);
my $res = $ua->request($req);
print "Result is:\n";
print Dumper($res);

My environment variable for http_proxy is set up like so:

http_proxy=http://richard:mypassword@10.10.200.44:8080/
HTTP_PROXY=http://richard:mypassword@10.10.200.44:8080/
export ...all of the above...

and running the code above with no arguments produces the desired
result.

However, if I try the code above with an argument (so it trys https)
with the following environment variables:

https_proxy=http://richard:mypassword@10.10.200.44:8080/
HTTPS_PROXY=http://richard:mypassword@10.10.200.44:8080/
export ...all of the above...

or even:

https_proxy=http://10.10.200.44:8080/
HTTPS_PROXY=http://10.10.200.44:8080/
https_proxy_username=richard
HTTPS_PROXY_USERNAME=richard
https_proxy_password=mypassword
HTTPS_PROXY_PASSWORD=mypassword
export ...all of the above...

then I always get the following error:

$VAR1 = bless( {
'_content' => '500 Can\'t connect to [url]www.nodeworks.com:443[/url] (Bad service
\'8080/\')',
'_rc' => 500,
'_headers' => bless( {
'client-warning' => 'Internal response',
'client-date' => 'Wed, 30 Mar 2005 13:41:12 GMT',
'content-type' => 'text/plain'
}, 'HTTP::Headers' ),
'_msg' => 'Can\'t connect to [url]www.nodeworks.com:443[/url] (Bad service
\'8080/\')',
'_request' => bless( {
'_content' => '',
'_uri' => bless( do{\(my $o = 'https://www.nodeworks.com')},
'URI::https' ),
'_headers' => bless( {
'user-agent' => 'libwww-perl/5.803'
}, 'HTTP::Headers' ),
'_method' => 'HEAD'
}, 'HTTP::Request' )
}, 'HTTP::Response' );

Can anyone offer any suggestions?

Many thanks,

Richard

[Ilmari Karonen]

["Followup-To:" set to comp.lang.perl.misc.]

Richard Lawrence <richard.lawrence@gmail.com> kirjoitti 30.03.2005:

>
> I'm trying to access [url]https://www.nodeworks.com/[/url] through a proxy which
> requires a proxy username and password. The code I have is as follows:

I don't have a proxy that would require a username and a password, but
I've tested your code as far as I've can. As far as I can tell, your
code works just as it should -- the problem is elsewhere.

> my $req = new HTTP::Request('HEAD', $url);
> my $res = $ua->request($req);
> print "Result is:\n";
> print Dumper($res);

Just so you know, you can get a much cleaner dump of the response by
using $res->as_string(). In fact, those four lines can be reduced to:

print "Result is:\n", $ua->head($url)->as_string;

> However, if I try the code above with an argument (so it trys https)
> with the following environment variables:
>
> https_proxy=http://richard:mypassword@10.10.200.44:8080/

You don't need this; the uppercase version is the proper one.

> HTTPS_PROXY=http://richard:mypassword@10.10.200.44:8080/

It appears that Crypt::SSLeay doesn't support this syntax, and will
fail with an error similar to what you report. So this is likely to
be at least a part of your problem.

> HTTPS_PROXY=http://10.10.200.44:8080/
> HTTPS_PROXY_USERNAME=richard
> HTTPS_PROXY_PASSWORD=mypassword

This, however, seems to work just fine. Of course, I can't really
test if the authentication works or not, but at least there is no
error. You're reporting that you get the same "Bad service" error
each time, but I'd really suggest you try it one more time, and make
sure $ENV{HTTPS_PROXY} really contains what you think it does. You
could add something like this to your code to dump the relevant
environment variables:

/^HTTPS?_PROXY/ and print "$_=$ENV{$_}\n" for sort keys %ENV;


Hope this helps.

--
Ilmari Karonen
To reply by e-mail, please replace ".invalid" with ".net" in address.