ongoing 'ping and parse' project

[Erich Linder]

NEWBIE ALERT * NEWBIE ALERT * NEWBIE ALERT

I'm starting here as PHP is the only scripting language that I know other
than VBA for Excel. But believe me, I am very weak in PHP as I'm sure
you'll see very soon.

I want to do a project where I'm compiling A LOT of data in a MySQL DB.
The data will be fairly straightforward, actually it might be just one
table rather than a real DB. I need a lot of data as this is a statistics
problem and I want to get my confidence intervals as accurate as possible.

What I want to do is run a script that

1) reads another site's page -- this page will have a unique address
2) parses the data on that page so that it can be
3) posted to the MySQL DB then
4) starts the whole process over again -- with a new page with another
unique address

SO, I've worked with scripts that run one operation, however many
pages/scripts that takes, and come back with a result page and/or post data
etc. But I've never worked with anything that once it has been kicked off
will run and run for a very long time, compiling data along the way.

Is PHP even the right place to start with this? I'm going to research
using VBA to do this, but I haven't done anything that really uses web
based data in a meaningful way before.

Thanks for reading this and ANY insight into any aspect of this project
would be mucho appreciated.

Erich Linder
[email]thunderball_NOSPAM_fists@hotmail.com[/email]

[Milambar]

Erich Linder wrote:

> NEWBIE ALERT * NEWBIE ALERT * NEWBIE ALERT
>
> I'm starting here as PHP is the only scripting language that I know other
> than VBA for Excel. But believe me, I am very weak in PHP as I'm sure
> you'll see very soon.
>
> I want to do a project where I'm compiling A LOT of data in a MySQL DB.
> The data will be fairly straightforward, actually it might be just one
> table rather than a real DB. I need a lot of data as this is a statistics
> problem and I want to get my confidence intervals as accurate as possible.
>
> What I want to do is run a script that
>
> 1) reads another site's page -- this page will have a unique address
> 2) parses the data on that page so that it can be
> 3) posted to the MySQL DB then
> 4) starts the whole process over again -- with a new page with another
> unique address
>
> SO, I've worked with scripts that run one operation, however many
> pages/scripts that takes, and come back with a result page and/or post data
> etc. But I've never worked with anything that once it has been kicked off
> will run and run for a very long time, compiling data along the way.
>
> Is PHP even the right place to start with this? I'm going to research
> using VBA to do this, but I haven't done anything that really uses web
> based data in a meaningful way before.
>
> Thanks for reading this and ANY insight into any aspect of this project
> would be mucho appreciated.
>
> Erich Linder
> [email]thunderball_NOSPAM_fists@hotmail.com[/email]

Ummm.

Parse the site's page for what?

It sounds suspiciously like you want a program to harvest email
addresses from webpages.. In which case, I suspect your appeal for help
will fall on deaf ears.