using Perl to auto-login to capitalone.com

[Andrew Gaffney]

I am attempting to write a Perl script that can login to my account at capitalone.com,
grab my balance and available credit and email them to me. I did this successfully with my
online banking, but Capitalone's login is a pain in the ass to wrap my head around. It
redirects you through 3 or pages before an actual login screen and sets cookies at each
one. On one of the last pages, it has JavaScript that generates some more cookies. So far,
I've got:


#!/usr/bin/perl

use LWP::UserAgent;
use HTTP::Cookies;

my $res, $req;
my $ua = LWP::UserAgent->new;
$ua->cookie_jar(HTTP::Cookies->new(file => "bankcookies.txt", autosave => 1));

$req = HTTP::Request->new(GET => 'http://www.capitalone.com/indexrfa.php');
$res = $ua->request($req);
#$req = HTTP::Request->new(GET =>
'http://www.capitalone.com/redirect.html?Log=1&linkid=WWW_Z_Z_Z_HMLFT_L1_01_G _OASL&dest=https://service.capitalone.com/cgi/Home?Login');
#$res = $ua->request($req);
#print $res->as_string;
$req = HTTP::Request->new(GET => 'https://service.capitalone.com/cgi/Home?Login');
$res = $ua->request($req);
#print $res->as_string;
$req = HTTP::Request->new(GET =>
'https://service.capitalone.com/oas_secure/oas/login.do?objectclicked=LoginSplash');
$res = $ua->request($req);
print $res->as_string;


I'm not sure how much of the above is actually necessary, but those are the pages that you
go through when manually logging in. That last page is where it starts using JavaScript to
generate cookies just to make sure that you have JavaScript and cookies enabled in the
browser. Can anyone help me out? Thanks.

--
Andrew Gaffney
System Administrator
Skyline Aeronautics, LLC.
776 North Bell Avenue
Chesterfield, MO 63005
636-357-1548

[Luinrandir Hernsen]

IS this how I capture a web page code to a variable?

> #!/usr/bin/perl
>
> use LWP::UserAgent;
>
> my $req;
> my $ua = LWP::UserAgent->new;
> $req = HTTP::Request->new(GET => 'http://www.yahoo.com');

print $req; ## ?????????????
from here I can print at the code for the web page?


################################################## ######3

[Kevin Old]

Have you look at WWW::Mechanize on CPAN. It seems to make navigating
sites (virtually) and using forms, etc. much easier.

It's worth a shot.

HTH,
Kevin

On Wed, 2004-01-28 at 02:37, Andrew Gaffney wrote:

> I am attempting to write a Perl script that can login to my account at capitalone.com,
> grab my balance and available credit and email them to me. I did this successfully with my
> online banking, but Capitalone's login is a pain in the ass to wrap my head around. It
> redirects you through 3 or pages before an actual login screen and sets cookies at each
> one. On one of the last pages, it has JavaScript that generates some more cookies. So far,
> I've got:
>
>
> #!/usr/bin/perl
>
> use LWP::UserAgent;
> use HTTP::Cookies;
>
> my $res, $req;
> my $ua = LWP::UserAgent->new;
> $ua->cookie_jar(HTTP::Cookies->new(file => "bankcookies.txt", autosave => 1));
>
> $req = HTTP::Request->new(GET => 'http://www.capitalone.com/indexrfa.php');
> $res = $ua->request($req);
> #$req = HTTP::Request->new(GET =>
> 'http://www.capitalone.com/redirect.html?Log=1&linkid=WWW_Z_Z_Z_HMLFT_L1_01_G _OASL&dest=https://service.capitalone.com/cgi/Home?Login');
> #$res = $ua->request($req);
> #print $res->as_string;
> $req = HTTP::Request->new(GET => 'https://service.capitalone.com/cgi/Home?Login');
> $res = $ua->request($req);
> #print $res->as_string;
> $req = HTTP::Request->new(GET =>
> 'https://service.capitalone.com/oas_secure/oas/login.do?objectclicked=LoginSplash');
> $res = $ua->request($req);
> print $res->as_string;
>
>
> I'm not sure how much of the above is actually necessary, but those are the pages that you
> go through when manually logging in. That last page is where it starts using JavaScript to
> generate cookies just to make sure that you have JavaScript and cookies enabled in the
> browser. Can anyone help me out? Thanks.
>
> --
> Andrew Gaffney
> System Administrator
> Skyline Aeronautics, LLC.
> 776 North Bell Avenue
> Chesterfield, MO 63005
> 636-357-1548

--
Kevin Old <kold@kold.homelinux.com>

[Rob Dixon]

Kevin Old wrote:

>
> Have you look at WWW::Mechanize on CPAN. It seems to make navigating
> sites (virtually) and using forms, etc. much easier.
>
> It's worth a shot.

Seconded.

It's definitely worthwhile for well-behaved sites, but can be tricky
to debug if the site doesn't behave as you expect.

Rob

[Wiggins D Anconia]

> I am attempting to write a Perl script that can login to my account at
capitalone.com,

> grab my balance and available credit and email them to me. I did this

successfully with my

> online banking, but Capitalone's login is a pain in the ass to wrap my

head around. It

> redirects you through 3 or pages before an actual login screen and

sets cookies at each

> one. On one of the last pages, it has JavaScript that generates some

more cookies. So far,

> I've got:
>

<snip code>

>
> I'm not sure how much of the above is actually necessary, but those

are the pages that you

> go through when manually logging in. That last page is where it starts

using JavaScript to

> generate cookies just to make sure that you have JavaScript and

cookies enabled in the

> browser. Can anyone help me out? Thanks.
>

Just a couple thoughts. Make sure you aren't violating user agreements,
etc. Some instituitions could view this as a denial of service
depending on how often you intend to run it, etc.

Because you are looking to wrangle the HTTP request/response headers,
etc. there is a new (to me at least) extension for the Firebird
(Mozilla) browser, Live HTTP Headers, that allows you to view and
capture as you are browsing all of the actual response/request headers.
This might help show exactly what is happening in the browser making it
easier to mimic in LWP or whatever.

More here:
[url]http://www.texturizer.net/firebird/extensions/#livehttpheaders[/url]

Very cool for web app developers...

[url]http://danconia.org[/url]

[John McKown]

On Wed, 28 Jan 2004, Luinrandir Hernsen wrote:

> IS this how I capture a web page code to a variable?
>

> > #!/usr/bin/perl
> >
> > use LWP::UserAgent;
> >
> > my $req;
> > my $ua = LWP::UserAgent->new;
> > $req = HTTP::Request->new(GET => 'http://www.yahoo.com');

>
> print $req; ## ?????????????
> from here I can print at the code for the web page?
>

print $req->as_string; #print the html

--
Maranatha!
John McKown

[Andrew Gaffney]

It does not appear that this module would solve my problem. It won't handle the JavaScript
generation of cookies.

Kevin Old wrote:

> Have you look at WWW::Mechanize on CPAN. It seems to make navigating
> sites (virtually) and using forms, etc. much easier.
>
> It's worth a shot.
>
> HTH,
> Kevin
>
> On Wed, 2004-01-28 at 02:37, Andrew Gaffney wrote:
>

>>I am attempting to write a Perl script that can login to my account at capitalone.com,
>>grab my balance and available credit and email them to me. I did this successfully with my
>>online banking, but Capitalone's login is a pain in the ass to wrap my head around. It
>>redirects you through 3 or pages before an actual login screen and sets cookies at each
>>one. On one of the last pages, it has JavaScript that generates some more cookies. So far,
>>I've got:
>>
>>
>>#!/usr/bin/perl
>>
>>use LWP::UserAgent;
>>use HTTP::Cookies;
>>
>>my $res, $req;
>>my $ua = LWP::UserAgent->new;
>>$ua->cookie_jar(HTTP::Cookies->new(file => "bankcookies.txt", autosave => 1));
>>
>>$req = HTTP::Request->new(GET => 'http://www.capitalone.com/indexrfa.php');
>>$res = $ua->request($req);
>>#$req = HTTP::Request->new(GET =>
>>'http://www.capitalone.com/redirect.html?Log=1&linkid=WWW_Z_Z_Z_HMLFT_L1_01_G _OASL&dest=https://service.capitalone.com/cgi/Home?Login');
>>#$res = $ua->request($req);
>>#print $res->as_string;
>>$req = HTTP::Request->new(GET => 'https://service.capitalone.com/cgi/Home?Login');
>>$res = $ua->request($req);
>>#print $res->as_string;
>>$req = HTTP::Request->new(GET =>
>>'https://service.capitalone.com/oas_secure/oas/login.do?objectclicked=LoginSplash');
>>$res = $ua->request($req);
>>print $res->as_string;
>>
>>
>>I'm not sure how much of the above is actually necessary, but those are the pages that you
>>go through when manually logging in. That last page is where it starts using JavaScript to
>>generate cookies just to make sure that you have JavaScript and cookies enabled in the
>>browser. Can anyone help me out? Thanks.

--
Andrew Gaffney
System Administrator
Skyline Aeronautics, LLC.
776 North Bell Avenue
Chesterfield, MO 63005
636-357-1548

[Andrew Gaffney]

Wiggins d Anconia wrote:

>>I am attempting to write a Perl script that can login to my account at

>
> capitalone.com,
>

>>grab my balance and available credit and email them to me. I did this

>
> successfully with my
>

>>online banking, but Capitalone's login is a pain in the ass to wrap my

>
> head around. It
>

>>redirects you through 3 or pages before an actual login screen and

>
> sets cookies at each
>

>>one. On one of the last pages, it has JavaScript that generates some

>
> more cookies. So far,
>

>>I've got:
>>

>
>
> <snip code>
>

>>I'm not sure how much of the above is actually necessary, but those

>
> are the pages that you
>

>>go through when manually logging in. That last page is where it starts

>
> using JavaScript to
>

>>generate cookies just to make sure that you have JavaScript and

>
> cookies enabled in the
>

>>browser. Can anyone help me out? Thanks.
>>

>
>
> Just a couple thoughts. Make sure you aren't violating user agreements,
> etc. Some instituitions could view this as a denial of service
> depending on how often you intend to run it, etc.
>
> Because you are looking to wrangle the HTTP request/response headers,
> etc. there is a new (to me at least) extension for the Firebird
> (Mozilla) browser, Live HTTP Headers, that allows you to view and
> capture as you are browsing all of the actual response/request headers.
> This might help show exactly what is happening in the browser making it
> easier to mimic in LWP or whatever.
>
> More here:
> [url]http://www.texturizer.net/firebird/extensions/#livehttpheaders[/url]
>
> Very cool for web app developers...

That is a very cool plugin. It appears to work with Mozilla, also. I think that will help
a lot. Thanks.

--
Andrew Gaffney
System Administrator
Skyline Aeronautics, LLC.
776 North Bell Avenue
Chesterfield, MO 63005
636-357-1548