how does one trap for out-of-memory errors?

[lawrence]

I'm not sure how this is normally done, on a large site, perhaps one
running Phorum. Occassionally a thread will have hundreds of entries,
perhaps a meg or two worth of data. You won't necessarily print all
that to the screen, but PHP has to hold it in memory. Run some
operations on it, or, more likely, have an array that you keep adding
things to, and very soon you run into the 8 meg limit that is the
default limit for PHP scripts.

How do you trap for out-of-memory errors? I want to do something like:

if (php is now using more than 8 megs) {
print "Sorry, but this operation requires more memory than PHP is
allowed.";
} else {
printOutEntryes($allEntries);
}


PHP has a command that checks the current memory usage of PHP, but
PHP, near as I know, has no command to estimate how much the next
operation is going to drive up memory usuage.

I'm sure this problem comes up for people using Phorum, or PostNuke,
or phpSlash, or phpMyAdmin or any of a lot of programs. How is it
normally handled?

[Nikolai Chuvakhin]

[email]lkrubner@geocities.com[/email] (lawrence) wrote in message
news:<da7e68e8.0310181002.6fbf830b@posting.google. com>...

>
> How do you trap for out-of-memory errors? I want to do something like:
>
> if (php is now using more than 8 megs) {
> print "Sorry, but this operation requires more memory than PHP is
> allowed.";
> } else {
> printOutEntryes($allEntries);
> }

Not going to work. PHP will display an out-of-memory error and stop
running BEFORE it gets to executing your if() statement.

> I'm sure this problem comes up for people using Phorum, or PostNuke,
> or phpSlash, or phpMyAdmin or any of a lot of programs. How is it
> normally handled?

By preventing it from happening. More specifically, by avoiding
sucking into memory things of unknown and potentially very large
size. Even more specifically, by writing proper database queries,
so that the resulting datasets can be processed and output strictly
one record at a time, without accumulating them in arrays on the
client side.

Cheers,
NC

[Perttu Pulkkinen]

"Nikolai Chuvakhin" <nc@iname.com> wrote in message

> Not going to work. PHP will display an out-of-memory error and stop
> running BEFORE it gets to executing your if() statement.

Are you also saying that memory errors are OUTSIDE of possibilities of php
error handling systems (notices, warnings, errors and fatal errors). Because
normally PHP errors CAN be handled and it is possible to execute code after
error. You just need to write error handlers and set them to use.

perttu pulkkinen, Finland

[DvDmanDT]

You know, try to read a 100mb file into memory, do some operations on it,
then see what happends... It's strange really, but all that happends is
slight slowness... There was some thread comparing PHP with ASP where
someone wanted to something with big XML files...
ASP+40mb XML file... Script ran for 60 mins or so unless my memory fails
me... Then it just said out of memory or something...

PHP + 100mb: 15 mins... He read entire file into memory, then did some
operations on it... PHP handled the file and the memory without problems...

--
// DvDmanDT
MSN: [email]dvdmandt@hotmail.com[/email]
Mail: [email]dvdmandt@telia.com[/email]
"lawrence" <lkrubner@geocities.com> skrev i meddelandet
news:da7e68e8.0310181002.6fbf830b@posting.google.c om...

> I'm not sure how this is normally done, on a large site, perhaps one
> running Phorum. Occassionally a thread will have hundreds of entries,
> perhaps a meg or two worth of data. You won't necessarily print all
> that to the screen, but PHP has to hold it in memory. Run some
> operations on it, or, more likely, have an array that you keep adding
> things to, and very soon you run into the 8 meg limit that is the
> default limit for PHP scripts.
>
> How do you trap for out-of-memory errors? I want to do something like:
>
> if (php is now using more than 8 megs) {
> print "Sorry, but this operation requires more memory than PHP is
> allowed.";
> } else {
> printOutEntryes($allEntries);
> }
>
>
> PHP has a command that checks the current memory usage of PHP, but
> PHP, near as I know, has no command to estimate how much the next
> operation is going to drive up memory usuage.
>
> I'm sure this problem comes up for people using Phorum, or PostNuke,
> or phpSlash, or phpMyAdmin or any of a lot of programs. How is it
> normally handled?

[DvDmanDT]

To simply allocate way to much memory in a very idiotic way (to test what
happends):

<?php
/**** Memory killer 1.0 ****
* Will allocate some memory.
* Case you wonder: No, I
* don't have a life. I
* suggest using this on cgi
* version, PHP 4 or 5. Apache
* might crash otherwise.
*/
set_time_limit(0); // This WILL take some time, up to 30 mins, so disable
time limit
define("BYTES_TO_ALLOCATE",20*1024*1024); // 20 mb
ob_implicit_flush();
header("Content-Type: text/plain");
function pmem(){
global $str;
echo "Current lenght: ".strlen($str)."\r\n";
// Uncomment next line to show usage
#echo "Memory useage: ".(memory_get_usage()/1024)." kb\r\n";
}
register_tick_function('pmem');
declare(ticks=1024);
$str="";
srand(time());
for($i=0;$i<BYTES_TO_ALLOCATE;$i++)$str.=chr(rand( 32,100)); // Add a random
char
?>

--
// DvDmanDT
MSN: [email]dvdmandt@hotmail.com[/email]
Mail: [email]dvdmandt@telia.com[/email]
"DvDmanDT" <dvdmandt@telia.com> skrev i meddelandet
news:u7Dkb.30206$mU6.79346@newsb.telia.net...

> You know, try to read a 100mb file into memory, do some operations on it,
> then see what happends... It's strange really, but all that happends is
> slight slowness... There was some thread comparing PHP with ASP where
> someone wanted to something with big XML files...
> ASP+40mb XML file... Script ran for 60 mins or so unless my memory fails
> me... Then it just said out of memory or something...
>
> PHP + 100mb: 15 mins... He read entire file into memory, then did some
> operations on it... PHP handled the file and the memory without

problems...

>
> --
> // DvDmanDT
> MSN: [email]dvdmandt@hotmail.com[/email]
> Mail: [email]dvdmandt@telia.com[/email]
> "lawrence" <lkrubner@geocities.com> skrev i meddelandet
> news:da7e68e8.0310181002.6fbf830b@posting.google.c om...

> > I'm not sure how this is normally done, on a large site, perhaps one
> > running Phorum. Occassionally a thread will have hundreds of entries,
> > perhaps a meg or two worth of data. You won't necessarily print all
> > that to the screen, but PHP has to hold it in memory. Run some
> > operations on it, or, more likely, have an array that you keep adding
> > things to, and very soon you run into the 8 meg limit that is the
> > default limit for PHP scripts.
> >
> > How do you trap for out-of-memory errors? I want to do something like:
> >
> > if (php is now using more than 8 megs) {
> > print "Sorry, but this operation requires more memory than PHP is
> > allowed.";
> > } else {
> > printOutEntryes($allEntries);
> > }
> >
> >
> > PHP has a command that checks the current memory usage of PHP, but
> > PHP, near as I know, has no command to estimate how much the next
> > operation is going to drive up memory usuage.
> >
> > I'm sure this problem comes up for people using Phorum, or PostNuke,
> > or phpSlash, or phpMyAdmin or any of a lot of programs. How is it
> > normally handled?

>
>

[Nikolai Chuvakhin]

"Perttu Pulkkinen" <Perttu.Pulkkinen@co.jyu.fi> wrote in message
news:<1yrkb.51$BY1.45@read3.inet.fi>...

>
> "Nikolai Chuvakhin" <nc@iname.com> wrote in message

> > PHP will display an out-of-memory error and stop running
> > BEFORE it gets to executing your if() statement.

>
> Are you also saying that memory errors are OUTSIDE of possibilities
> of php error handling systems (notices, warnings, errors and fatal
> errors).

No. All I'm saying is that what the original poster wanted cannot
be done the way he wants it. In particular, if my understanding
is correct, a memory allocation problem causes a fatal error
(E_ERROR), but until the error is handled (possibly, by a user-
defined error handling function), it is impossible to know whether
the error is caused by memory allocation problem or something else.

Cheers,
NC

[lawrence]

[email]nc@iname.com[/email] (Nikolai Chuvakhin) wrote in message news:<32d7a63c.0310212041.6780b34e@posting.google. com>...

> "Perttu Pulkkinen" <Perttu.Pulkkinen@co.jyu.fi> wrote in message
> news:<1yrkb.51$BY1.45@read3.inet.fi>...

> >
> > "Nikolai Chuvakhin" <nc@iname.com> wrote in message

> > > PHP will display an out-of-memory error and stop running
> > > BEFORE it gets to executing your if() statement.

> >
> > Are you also saying that memory errors are OUTSIDE of possibilities
> > of php error handling systems (notices, warnings, errors and fatal
> > errors).

>
> No. All I'm saying is that what the original poster wanted cannot
> be done the way he wants it. In particular, if my understanding
> is correct, a memory allocation problem causes a fatal error
> (E_ERROR), but until the error is handled (possibly, by a user-
> defined error handling function), it is impossible to know whether
> the error is caused by memory allocation problem or something else.

Yes, thank you. You understood my intention. I was asking about
trapping for errors.

If you can open a 100mb file and process it and then write it to disk,
then what does the 8 mb limit refer to?

[lawrence]

[email]nc@iname.com[/email] (Nikolai Chuvakhin) wrote in message news:<32d7a63c.0310182055.442f5236@posting.google. com>...

> > I'm sure this problem comes up for people using Phorum, or PostNuke,
> > or phpSlash, or phpMyAdmin or any of a lot of programs. How is it
> > normally handled?

>
> By preventing it from happening. More specifically, by avoiding
> sucking into memory things of unknown and potentially very large
> size. Even more specifically, by writing proper database queries,
> so that the resulting datasets can be processed and output strictly
> one record at a time, without accumulating them in arrays on the
> client side.

The right kind of queries? So something like this is bad:

SELECT * FROM content WHERE type = 'weblogEntries'

I get this back and then try to read it into an array. If there are
enough entries, then I run into the 8 meg limit. I take it that your
approach would avoid putting the whole return into an array?

[Nikolai Chuvakhin]

[email]lkrubner@geocities.com[/email] (lawrence) wrote in message
news:<da7e68e8.0310231140.2867c947@posting.google. com>...

>

> > > How is it normally handled?

> >
> > By preventing it from happening. More specifically, by avoiding
> > sucking into memory things of unknown and potentially very large
> > size. Even more specifically, by writing proper database queries,
> > so that the resulting datasets can be processed and output strictly
> > one record at a time, without accumulating them in arrays on the
> > client side.

>
> The right kind of queries? So something like this is bad:
>
> SELECT * FROM content WHERE type = 'weblogEntries'

No (except for the fact that `type` is a text, and thus likely
unindexed, field), but your handling of it is definitely bad:

> I get this back and then try to read it into an array.

Why would you want to do this? With very few exceptions, any data
manipulation you are about to do with PHP is done faster and with
less overhead by MySQL. And if you are not going to manipulate
the data, why bother reading it into a huge array?

If you explain what exactly you are trying to achieve, I'll gladly
help you find a better way of getting it done.

> I take it that your approach would avoid putting the whole
> return into an array?

Of course. The usual

while ($record = mysql_fetch_row ($result)) {}

only keeps in memory one record at a time. Remember, $result
is a resource (sort of like a file pointer), so it has the same
memory footprint regardless of the number of records in the
returned dataset.

Cheers,
NC

[Zurab Davitiani]

Nikolai Chuvakhin wrote on Friday 24 October 2003 00:25:

> Of course. The usual
>
> while ($record = mysql_fetch_row ($result)) {}
>
> only keeps in memory one record at a time.

I'm not so sure about this. My understanding is that

$my_var = "something";

$my_var = "something else";

does not free the original memory occupied by "something", it just allocates
new memory for "something else". All memory is only freed when script
execution or PHP process ends. I agree though, maybe some garbage
collection would be great in PHP.

--
Business Web Solutions
ActiveLink, LLC
[url]www.active-link.com/intranet/[/url]

[Nikolai Chuvakhin]

Zurab Davitiani <agt@mindless.com> wrote in message
news:<fW4mb.3044$dv3.1630@newssvr14.news.prodigy.c om>...

>
> Nikolai Chuvakhin wrote on Friday 24 October 2003 00:25:
>

> > The usual
> > while ($record = mysql_fetch_row ($result)) {}
> > only keeps in memory one record at a time.

>
> I'm not so sure about this. My understanding is that
>
> $my_var = "something";
> $my_var = "something else";
>
> does not free the original memory occupied by "something",
> it just allocates new memory for "something else". All memory
> is only freed when script execution or PHP process ends.

I honestly doubt it. If you were correct, one of my largest
applications couldn't possibly work, because one of its
functionalities is like this:

1. Retrieve about 10 fields per record from the entire
table (currently, ~200,000 records).
2. Based on those 10 fields and some extra inputs, compute
about 70 more fields for each record.
3. Write the results back into the table.

In other words, the

while ($record = mysql_fetch_row ($result)) {}

cycle was repeated 200,000 times. Given that $record was an
array including a date and a collection of floating-point
numbers, I'd say its memory requirement was about 100 bytes.
This would mean that the script would trash about 20 megabytes
of memory only when handling this one variable. But what about
the 70 values that were computed for each of the 200,000 records?
They (being mostly integers, with a few floats) would trash
at least another 20 megabytes. The UPDATE query string formed
for writing the output into the databse was about 100 characters
long, so it, too, would trash 20 megabytes over 200,000
reassignments. Yet the whole thing never ever hit the default
memory limit...

I am not saying you're wrong, but if you're right, I'd like to
understand how it is possible for you to be right while I see
what I see. No pun intended; I really would like to learn
something here...

Cheers,
NC

[lawrence]

[email]nc@iname.com[/email] (Nikolai Chuvakhin) wrote in message

> > I get this back and then try to read it into an array.

>
> Why would you want to do this? With very few exceptions, any data
> manipulation you are about to do with PHP is done faster and with
> less overhead by MySQL. And if you are not going to manipulate
> the data, why bother reading it into a huge array?
>
> If you explain what exactly you are trying to achieve, I'll gladly
> help you find a better way of getting it done.
>

> > I take it that your approach would avoid putting the whole
> > return into an array?

>
> Of course. The usual
>
> while ($record = mysql_fetch_row ($result)) {}
>
> only keeps in memory one record at a time. Remember, $result
> is a resource (sort of like a file pointer), so it has the same
> memory footprint regardless of the number of records in the
> returned dataset.

I am putting database returns in PHP arrays to mask the client code
from the underlying database. That is, I don't want my code to know
what kind of database or datastore is being dealt with.

I am storing all actual SQL in objects that I am calling queryObjects.
But the only place the query objects are called is in what I call Get,
Update, Delete, and Insert objects. The Get, Update, Delete, and
Insert objects are meant to mask, from the rest of the code, what kind
of database or datastore is in use. In other words, if I want to get
all weblog entries out of a database, I might have a method called
getAllWeblogEntries, and this function would contain actual SQL, and
return a PHP array. So I might have a query object for MySql and a
query object for PostGre and a query object for XML streams. Each of
these query objects would have a method called getAllWeblogEntries,
and each would return a PHP array.

The Get object knows what kind of database or datastore the website
uses (this is something set in the config file for that website). All
the other code simply makes a call to the Get object. Thus, the other
code, I mean the client code, never needs to know what kind of
database or datastore is in use.

I can't imagine how I can hide the database from the client code if I
use database return resources, as you suggest above. But I now realize
how limiting these arrays can be, in the presence of PHP's 8 meg
limit. So I'm open to any graceful ideas for otherwise masking the
database from the client code.

[Nikolai Chuvakhin]

[email]lkrubner@geocities.com[/email] (lawrence) wrote
in message <da7e68e8.0310241718.3fcdc191@posting.google.com >:

>
> I am putting database returns in PHP arrays to mask the client
> code from the underlying database. That is, I don't want my code
> to know what kind of database or datastore is being dealt with.

Haven't you heard that abstraction doesn't work? :) I am not
trying to start a flame war here, but you need to understand that
higher level of abstraction ALWAYS implies a performance penalty.
In your case, you are shooting for capital abstraction and what
you get for it is the performance equivalent of capital
punishment.

> I am storing all actual SQL in objects that I am calling
> queryObjects. But the only place the query objects are called
> is in what I call Get, Update, Delete, and Insert objects. The
> Get, Update, Delete, and Insert objects are meant to mask, from
> the rest of the code, what kind of database or datastore is in
> use. In other words, if I want to get all weblog entries out of
> a database, I might have a method called getAllWeblogEntries,
> and this function would contain actual SQL, and return a PHP
> array. So I might have a query object for MySql and a query
> object for PostGre and a query object for XML streams. Each of
> these query objects would have a method called
> getAllWeblogEntries, and each would return a PHP array.

And that's the root of your problem. You want to return something
of potentially very large size, and you are hell-bent on storing
it in memory. But let me ask you: what will the application do
with a PHP array that contains all entries in the weblog? Output
them? Then the entries don't have to be in memory at all, they
can be output straight from the DB buffer. Sort them? Again,
they don't have to be in the memory for that, you should have
taken care of it on the database level. Search through them?
Same thing; too late to do it efficiently, had to be done on the
database level.

So my suggestion to you is not to have a getAllWeblogEntries
method at all. Rather, you should have a getWeblogEntry method
that returns a single weblog entry in an array and an entriesLeft
method that returns a boolean value. This way, you can retrieve
all entries one by one:

$qo = new queryObjectMySQL ('SELECT * FROM entries');
// the constructor establishes the connection, retrieves data,
// and keeps the resulting resource for the object's internal
// use...
while ($qo->entriesLeft ()) {
$entry = $qo->getWeblogEntry ();
// now, do what you want with the $entry...
}

> I can't imagine how I can hide the database from the client code
> if I use database return resources, as you suggest above.

You make it sound like it's a good thing... :)

Cheers,
NC

[lawrence]

[email]nc@iname.com[/email] (Nikolai Chuvakhin) wrote in message news:<32d7a63c.0310251605.258cfb2b@posting.google. com>...

> [email]lkrubner@geocities.com[/email] (lawrence) wrote
> in message <da7e68e8.0310241718.3fcdc191@posting.google.com >:

> >
> > I am putting database returns in PHP arrays to mask the client
> > code from the underlying database. That is, I don't want my code
> > to know what kind of database or datastore is being dealt with.

>
> Haven't you heard that abstraction doesn't work? :) I am not
> trying to start a flame war here, but you need to understand that
> higher level of abstraction ALWAYS implies a performance penalty.
> In your case, you are shooting for capital abstraction and what
> you get for it is the performance equivalent of capital
> punishment.

This runs counter to most of what I've been taught. I'm not sure what
you mean when you say abstraction doesn't work. Much software work
involves abstraction. You could argue that PHP itself is an
abstraction, hiding the complexity of the underlying C code. Some
might say that anything other than machine code is an abstraction. I
can't think of a software project that doesn't involve abstraction.
As for OO design, to hide the datastore seems like an obvious move.
Hasn't the team working on PEAR done the same?

>

> > I am storing all actual SQL in objects that I am calling
> > queryObjects. But the only place the query objects are called
> > is in what I call Get, Update, Delete, and Insert objects. The
> > Get, Update, Delete, and Insert objects are meant to mask, from
> > the rest of the code, what kind of database or datastore is in
> > use. In other words, if I want to get all weblog entries out of
> > a database, I might have a method called getAllWeblogEntries,
> > and this function would contain actual SQL, and return a PHP
> > array. So I might have a query object for MySql and a query
> > object for PostGre and a query object for XML streams. Each of
> > these query objects would have a method called
> > getAllWeblogEntries, and each would return a PHP array.

>
> And that's the root of your problem. You want to return something
> of potentially very large size, and you are hell-bent on storing
> it in memory.

No, I am not hell bent. I asked for alternatives.

> But let me ask you: what will the application do
> with a PHP array that contains all entries in the weblog? Output
> them? Then the entries don't have to be in memory at all, they
> can be output straight from the DB buffer. Sort them? Again,
> they don't have to be in the memory for that, you should have
> taken care of it on the database level. Search through them?
> Same thing; too late to do it efficiently, had to be done on the
> database level.
>
> So my suggestion to you is not to have a getAllWeblogEntries
> method at all. Rather, you should have a getWeblogEntry method
> that returns a single weblog entry in an array and an entriesLeft
> method that returns a boolean value. This way, you can retrieve
> all entries one by one:
>
> $qo = new queryObjectMySQL ('SELECT * FROM entries');
> // the constructor establishes the connection, retrieves data,
> // and keeps the resulting resource for the object's internal
> // use...
> while ($qo->entriesLeft ()) {
> $entry = $qo->getWeblogEntry ();
> // now, do what you want with the $entry...
> }

I think your suggestion is a good one. Somehow I must get just one
entry at a time. The query object also needs to remember which entry
it got last, so it can get the next one. The SELECT object would have
to wrap that interger and pass it along to the final code, especially
if the final code has a loop that needs to know how long it is
supposed to keep running.

Off hand, it sounds terribly complicated, and the whole of the code
would have to be rewritten. Every loop would have to be rewritten. But
it would eventually get the code free of working with dangerously
large arrays. It is a project that, I think, will take a few months,
but I think the direction you suggest is a good one.

> > I can't imagine how I can hide the database from the client code
> > if I use database return resources, as you suggest above.

>
> You make it sound like it's a good thing... :)

Well, this code is being written first to work with MySql, but I'm
supposed to write it in such a way that would make it easy to switch
to PostGre. So I've written it the way I have to save myself some
future work. And saving oneself future work is the aim of most OO
abstraction.

[Nikolai Chuvakhin]

[email]lkrubner@geocities.com[/email] (lawrence) wrote in message
news:<da7e68e8.0310261251.1d71189b@posting.google. com>...

> [email]nc@iname.com[/email] (Nikolai Chuvakhin) wrote in message
> news:<32d7a63c.0310251605.258cfb2b@posting.google. com>...
>

> > In your case, you are shooting for capital abstraction and what
> > you get for it is the performance equivalent of capital
> > punishment.

>
> This runs counter to most of what I've been taught. I'm not sure
> what you mean when you say abstraction doesn't work.

My apologies for vague language. I should have said "database
abstraction layers" rather than just "abstraction".

> As for OO design, to hide the datastore seems like an obvious move.

Which is exactly the reason I generally object to using OO design
in the first place.

> Hasn't the team working on PEAR done the same?

Yes, they have. And, according to phplens, this resulted in 150-170%
increase in time it takes to complete a series of SELECT queries:

[url]http://phplens.com/lens/adodb/[/url]

> I am not hell bent. I asked for alternatives.

I think looking for an altermative in your situation was the right
thing to do. Note also that you didn't have a compelling reason
to do it "your" way in the first place; at least, my question to
that extent:

> > But let me ask you: what will the application do
> > with a PHP array that contains all entries in the weblog? Output
> > them? Then the entries don't have to be in memory at all, they
> > can be output straight from the DB buffer. Sort them? Again,
> > they don't have to be in the memory for that, you should have
> > taken care of it on the database level. Search through them?
> > Same thing; too late to do it efficiently, had to be done on the
> > database level.

went unanswered...

> Somehow I must get just one entry at a time.

I'd say this is the preferred way of dealing with the problem.

> The query object also needs to remember which entry it got last,
> so it can get the next one.

It already does, since it has a resource with an internal pointer
to the next available entry...

> Well, this code is being written first to work with MySql, but I'm
> supposed to write it in such a way that would make it easy to switch
> to PostGre.

This is yet another reason why database abstraction layers are
such a bad idea... PostgreSQL has a whole slew of features
that are not available in MySQL: views, rules, triggers, stored
procedures, etc. Database abstraction forces you to stick to
the lowest common denominator, which robs you of opportunities
to use the more advanced, higher-performance features...

Cheers,
NC

[lawrence]

[email]nc@iname.com[/email] (Nikolai Chuvakhin) wrote in message

> > I am not hell bent. I asked for alternatives.

>
> I think looking for an altermative in your situation was the right
> thing to do. Note also that you didn't have a compelling reason
> to do it "your" way in the first place; at least, my question to
> that extent:
>

> > > But let me ask you: what will the application do
> > > with a PHP array that contains all entries in the weblog? Output
> > > them? Then the entries don't have to be in memory at all, they
> > > can be output straight from the DB buffer. Sort them? Again,
> > > they don't have to be in the memory for that, you should have
> > > taken care of it on the database level. Search through them?
> > > Same thing; too late to do it efficiently, had to be done on the
> > > database level.

>
> went unanswered...

I'm not sure what you mean when you say I didn't answer your question.
I explained that I was trying to hide the database from the client
code. Putting the database return into a PHP array had been my answer
to that. You can, and have, argued that it is the wrong answer, but I
certainly answered your question about what it was that I was trying
to accomplish.

> > Well, this code is being written first to work with MySql, but I'm
> > supposed to write it in such a way that would make it easy to switch
> > to PostGre.

>
> This is yet another reason why database abstraction layers are
> such a bad idea... PostgreSQL has a whole slew of features
> that are not available in MySQL: views, rules, triggers, stored
> procedures, etc. Database abstraction forces you to stick to
> the lowest common denominator, which robs you of opportunities
> to use the more advanced, higher-performance features...

Engineering of any kind is often about goals and tradeoffs.
Programmers would, in general, be delighted if, by magic, there was
some way to both take advantage of all the features of any database
and yet also easily port software from one database to another.
However, there is no easy way to achieve both these goals, so one
generally has to choose one. Either software can take advantage of the
special features of one piece of software, or it can be easily ported
from one database to another.

In our case, portability trumps advanced feature use. For another
project the opposite decision might be a wise one.

[lawrence]

[email]nc@iname.com[/email] (Nikolai Chuvakhin) wrote in message

> > The query object also needs to remember which entry it got last,
> > so it can get the next one.

>
> It already does, since it has a resource with an internal pointer
> to the next available entry...

It strikes me as breathtakingly complicated. If I want my code to do
one thing if the database has returned less than 50 rows, but
something else if the database has returned 50 or more rows, then a
request to count the number of rows returned needs to be sent down the
line, through the selectObject to the queryObject, and then returned
back up the line. And both objects need a getNextRow method, wrapping
(for MySQL) mysql_fetch_row. Or, if I want sometimes use
mysql_fetch_object, or any of the other commands for getting rows,
then I'd have to write a separate method for each of those commands.
If I want the API of every queryObject to conform to the contract of a
standard interface, then I need to write these methods with that in
mind.

This may be the best way of doing things, I can't think of anything
better, but it strikes me as a large project, especially as I have
about 100 SQL statements for MySql alone.

[Zurab Davitiani]

Nikolai Chuvakhin wrote on Friday 24 October 2003 15:08:

> I am not saying you're wrong, but if you're right, I'd like to
> understand how it is possible for you to be right while I see
> what I see. No pun intended; I really would like to learn
> something here...

This is OT from the original discussion. Sorry for taking it in this
direction, but I'd like to clear up my original claim.

You are right. The issue doesn't affect primitive variables and arrays
thereof. It only affects objects. Objects don't seem to be destroyed (and
memory freed) until the script execution ends. e.g., if you have:

for($i=0; $i<10000; $i++) {
$myObj = new LogParser("SomeBigLogFile.log");
}

$myObj objects are not destroyed (to free the memory) even though the
variable with the same name is being reassigned. So, total memory
consumption goes up with each pass. And unset() doesn't help either in this
case.

But, if you have:

for($i=0; $i<10000; $i++) {
$contents = file_get_contents("SomeBigLogFile.log");
}

the memory consumption will stay the same.

By going over some comments in previous posts on this topic, it seems like
PHP5 does have a better memory management with regard to objects. I will
test it when I have time, or if anyone has verified, feel free to post.

--
Business Web Solutions
ActiveLink, LLC
[url]www.active-link.com/intranet/[/url]

[Alan Little]

Carved in mystic runes upon the very living rock, the last words of
lawrence of comp.lang.php make plain:

> I can't imagine how I can hide the database from the client code if I
> use database return resources, as you suggest above. But I now realize
> how limiting these arrays can be, in the presence of PHP's 8 meg
> limit. So I'm open to any graceful ideas for otherwise masking the
> database from the client code.

Unless I'm misunderstanding, it seems like you're overlooking an obvious
solution. You have methods for transparently managing the queries, why
not have one for transparently retrieving the data from the resource
pointer? I have something like this in one of my applications, which
needs to be able to work with MySQL or Sybase. I simply wrote db_
functions to replace all the mysql_ and sybase_ functions, and made them
intelligent enough to handle the differences transparently.

--
Alan Little
Phorm PHP Form Processor
[url]http://www.phorm.com/[/url]

[Nikolai Chuvakhin]

Zurab Davitiani <agt@mindless.com> wrote in message
news:<Lk6nb.1319$X76.1047@newssvr14.news.prodigy.c om>...

> Nikolai Chuvakhin wrote on Friday 24 October 2003 15:08:
>

> > I am not saying you're wrong, but if you're right, I'd like to
> > understand how it is possible for you to be right while I see
> > what I see. No pun intended; I really would like to learn
> > something here...

>
> This is OT from the original discussion.

I don't think so; the original discussion was about preventing
out-of-memory errors. The issue of how memory is allocated is,
I believe, relevant to that discussion.

> You are right. The issue doesn't affect primitive variables and arrays
> thereof. It only affects objects. Objects don't seem to be destroyed (and
> memory freed) until the script execution ends.

Thank you, this indeed clears things up.

Cheers,
NC