Ask a Question related to PHP Bugs, Design and Development.
-
foster dot graeme at gmail dot com #1
#40494 [NEW]: Memory problem with ZipArchive::addFile()
From: foster dot graeme at gmail dot com
Operating system: Linux
PHP version: 5.2.1
PHP Bug Type: Zip Related
Bug description: Memory problem with ZipArchive::addFile()
Description:
------------
When adding files to an archive, (using successive ZipArchive::addFile()
commands) the compression doesn't happen until the file is closed. This
can result in an out of memory error, a temporary fix is to close the
archive and then reopen it within the php code.
An idea solution would be to compress the file when it is added, probably
in function _zip_replace(), but I don't know what the implications of this
would be. It would certainly require a rewrite of the ugly function
zip_close().
--
Edit bug report at [url]http://bugs.php.net/?id=40494&edit=1[/url]
--
Try a CVS snapshot (PHP 4.4): [url]http://bugs.php.net/fix.php?id=40494&r=trysnapshot44[/url]
Try a CVS snapshot (PHP 5.2): [url]http://bugs.php.net/fix.php?id=40494&r=trysnapshot52[/url]
Try a CVS snapshot (PHP 6.0): [url]http://bugs.php.net/fix.php?id=40494&r=trysnapshot60[/url]
Fixed in CVS: [url]http://bugs.php.net/fix.php?id=40494&r=fixedcvs[/url]
Fixed in release: [url]http://bugs.php.net/fix.php?id=40494&r=alreadyfixed[/url]
Need backtrace: [url]http://bugs.php.net/fix.php?id=40494&r=needtrace[/url]
Need Reproduce Script: [url]http://bugs.php.net/fix.php?id=40494&r=needscript[/url]
Try newer version: [url]http://bugs.php.net/fix.php?id=40494&r=oldversion[/url]
Not developer issue: [url]http://bugs.php.net/fix.php?id=40494&r=support[/url]
Expected behavior: [url]http://bugs.php.net/fix.php?id=40494&r=notwrong[/url]
Not enough info: [url]http://bugs.php.net/fix.php?id=40494&r=notenoughinfo[/url]
Submitted twice: [url]http://bugs.php.net/fix.php?id=40494&r=submittedtwice[/url]
register_globals: [url]http://bugs.php.net/fix.php?id=40494&r=globals[/url]
PHP 3 support discontinued: [url]http://bugs.php.net/fix.php?id=40494&r=php3[/url]
Daylight Savings: [url]http://bugs.php.net/fix.php?id=40494&r=dst[/url]
IIS Stability: [url]http://bugs.php.net/fix.php?id=40494&r=isapi[/url]
Install GNU Sed: [url]http://bugs.php.net/fix.php?id=40494&r=gnused[/url]
Floating point limitations: [url]http://bugs.php.net/fix.php?id=40494&r=float[/url]
No Zend Extensions: [url]http://bugs.php.net/fix.php?id=40494&r=nozend[/url]
MySQL Configuration Error: [url]http://bugs.php.net/fix.php?id=40494&r=mysqlcfg[/url]
foster dot graeme at gmail dot com Guest
-
#39784 [NEW]: Can't make an instance of ZipArchive
From: itsmegawtf at gmail dot com Operating system: Gentoo linux PHP version: 5.2.0 PHP Bug Type: Zip Related Bug... -
#39742 [NEW]: Class 'ZipArchive' not found
From: eric dot leseint at ac-poitiers dot fr Operating system: Windows 2003 server PHP version: 5.2.0 PHP Bug Type: ... -
#39506 [NEW]: Archive corrupt with ZipArchive::addFile method
From: hscheidegger at zumbach dot ch Operating system: Windows XP PHP version: 5.2.0 PHP Bug Type: Zip Related Bug... -
#38944 [NEW]: ZipArchive exits with SEGV
From: judas dot iscariote at gmail dot com Operating system: linux PHP version: 5CVS-2006-09-25 (CVS) PHP Bug Type: Zip... -
memory problem/memory leak
Hi I have a problem with shockwave player running in Internet Explorer. My program consists of a controller movie which loads in content files in... -
tony2001@php.net #2
#40494 [Opn->Asn]: Memory problem with ZipArchive::addFile()
ID: 40494
Updated by: [email]tony2001@php.net[/email]
Reported By: foster dot graeme at gmail dot com
-Status: Open
+Status: Assigned
-Bug Type: Zip Related
+Bug Type: Feature/Change Request
Operating System: Linux
PHP Version: 5.2.1
-Assigned To:
+Assigned To: pajoye
Previous Comments:
------------------------------------------------------------------------
[2007-02-15 10:22:25] foster dot graeme at gmail dot com
Description:
------------
When adding files to an archive, (using successive
ZipArchive::addFile() commands) the compression doesn't happen until
the file is closed. This can result in an out of memory error, a
temporary fix is to close the archive and then reopen it within the php
code.
An idea solution would be to compress the file when it is added,
probably in function _zip_replace(), but I don't know what the
implications of this would be. It would certainly require a rewrite of
the ugly function zip_close().
------------------------------------------------------------------------
--
Edit this bug report at [url]http://bugs.php.net/?id=40494&edit=1[/url]
tony2001@php.net Guest
-
pajoye@php.net #3
#40494 [Asn->Bgs]: Memory problem with ZipArchive::addFile()
ID: 40494
Updated by: [email]pajoye@php.net[/email]
Reported By: foster dot graeme at gmail dot com
-Status: Assigned
+Status: Bogus
Bug Type: Feature/Change Request
Operating System: Linux
PHP Version: 5.2.1
Assigned To: pajoye
New Comment:
"When adding files to an archive, (using successive
ZipArchive::addFile()
commands) the compression doesn't happen until the file is closed. "
Yes, we do it while finalizing the archive.
" This can result in an out of memory error, "
You will run out of file ID before running out of memory. It does not
really use many memory, only the file names and file handlers.
I suppose you are talking about the file handlers?
"It would certainly require a rewrite of the ugly function
zip_close()"
What is ugly in this function? Or do you have a portable way to lock a
file until the archive creation is done?
I think you refer to the file handlers limitation. There is already a
bug about it and I plan to add a special (less safe) mode. This mode
will allow one to add only the paths without checks, errors will occur
only when the archive is closed. But that's a feature addition not a
bug fix.
I close this bug (not a bug > bogus).
Thanks for your report!
Previous Comments:
------------------------------------------------------------------------
[2007-02-15 10:22:25] foster dot graeme at gmail dot com
Description:
------------
When adding files to an archive, (using successive
ZipArchive::addFile() commands) the compression doesn't happen until
the file is closed. This can result in an out of memory error, a
temporary fix is to close the archive and then reopen it within the php
code.
An idea solution would be to compress the file when it is added,
probably in function _zip_replace(), but I don't know what the
implications of this would be. It would certainly require a rewrite of
the ugly function zip_close().
------------------------------------------------------------------------
--
Edit this bug report at [url]http://bugs.php.net/?id=40494&edit=1[/url]
pajoye@php.net Guest
-
foster dot graeme at gmail dot com #4
#40494 [Bgs]: Memory problem with ZipArchive::addFile()
ID: 40494
User updated by: foster dot graeme at gmail dot com
Reported By: foster dot graeme at gmail dot com
Status: Bogus
-Bug Type: Feature/Change Request
+Bug Type: Zip Related
Operating System: Linux
PHP Version: 5.2.1
Assigned To: pajoye
New Comment:
Maybe I need to explain this problem a little more.
I am trying to archive a folder on the server, at the moment it
contains 5609 folders and 11,221 files. The script loops through the
files adding them to the archive using the addFile() method. After the
first 1002 files I get a ZIPARCHIVE::ER_OPEN. If I close the archive
and the open it again I still have that error. However, if I close the
archive and open it before I get that error then I can archive all
11,221 files.
Since closing the file and re-opening fixes the problem (so long as I
do that before I get the error) Then may I suggest that closing an
archive will clear the status. Obviously, it would be good if this
wasn't necessary, in thatthe code could catch the problem and allocate
extra file handles if that is the problem.
Previous Comments:
------------------------------------------------------------------------
[2007-02-15 11:41:24] [email]pajoye@php.net[/email]
"When adding files to an archive, (using successive
ZipArchive::addFile()
commands) the compression doesn't happen until the file is closed. "
Yes, we do it while finalizing the archive.
" This can result in an out of memory error, "
You will run out of file ID before running out of memory. It does not
really use many memory, only the file names and file handlers.
I suppose you are talking about the file handlers?
"It would certainly require a rewrite of the ugly function
zip_close()"
What is ugly in this function? Or do you have a portable way to lock a
file until the archive creation is done?
I think you refer to the file handlers limitation. There is already a
bug about it and I plan to add a special (less safe) mode. This mode
will allow one to add only the paths without checks, errors will occur
only when the archive is closed. But that's a feature addition not a
bug fix.
I close this bug (not a bug > bogus).
Thanks for your report!
------------------------------------------------------------------------
[2007-02-15 10:22:25] foster dot graeme at gmail dot com
Description:
------------
When adding files to an archive, (using successive
ZipArchive::addFile() commands) the compression doesn't happen until
the file is closed. This can result in an out of memory error, a
temporary fix is to close the archive and then reopen it within the php
code.
An idea solution would be to compress the file when it is added,
probably in function _zip_replace(), but I don't know what the
implications of this would be. It would certainly require a rewrite of
the ugly function zip_close().
------------------------------------------------------------------------
--
Edit this bug report at [url]http://bugs.php.net/?id=40494&edit=1[/url]
foster dot graeme at gmail dot com Guest
-
pajoye@php.net #5
#40494 [Bgs]: Memory problem with ZipArchive::addFile()
ID: 40494
Updated by: [email]pajoye@php.net[/email]
Reported By: foster dot graeme at gmail dot com
Status: Bogus
Bug Type: Zip Related
Operating System: Linux
PHP Version: 5.2.1
Assigned To: pajoye
New Comment:
See:
[url]http://pecl.php.net/bugs/bug.php?id=9443[/url]
"it would be good if this wasn't necessary, in thatthe code could catch
the problem and allocate extra file handles if that is the problem."
This is not something I can control. The operating system defines it
and there is no way for me to increase this value.
I suggest you to close and reopen it every 1000 or so (or even 255 if
you want to go on the safest way, ie old windows).
Future releases will have a different mode, where the checks will done
only when you close the archives.
Previous Comments:
------------------------------------------------------------------------
[2007-02-15 13:14:57] foster dot graeme at gmail dot com
Maybe I need to explain this problem a little more.
I am trying to archive a folder on the server, at the moment it
contains 5609 folders and 11,221 files. The script loops through the
files adding them to the archive using the addFile() method. After the
first 1002 files I get a ZIPARCHIVE::ER_OPEN. If I close the archive
and the open it again I still have that error. However, if I close the
archive and open it before I get that error then I can archive all
11,221 files.
Since closing the file and re-opening fixes the problem (so long as I
do that before I get the error) Then may I suggest that closing an
archive will clear the status. Obviously, it would be good if this
wasn't necessary, in thatthe code could catch the problem and allocate
extra file handles if that is the problem.
------------------------------------------------------------------------
[2007-02-15 11:41:24] [email]pajoye@php.net[/email]
"When adding files to an archive, (using successive
ZipArchive::addFile()
commands) the compression doesn't happen until the file is closed. "
Yes, we do it while finalizing the archive.
" This can result in an out of memory error, "
You will run out of file ID before running out of memory. It does not
really use many memory, only the file names and file handlers.
I suppose you are talking about the file handlers?
"It would certainly require a rewrite of the ugly function
zip_close()"
What is ugly in this function? Or do you have a portable way to lock a
file until the archive creation is done?
I think you refer to the file handlers limitation. There is already a
bug about it and I plan to add a special (less safe) mode. This mode
will allow one to add only the paths without checks, errors will occur
only when the archive is closed. But that's a feature addition not a
bug fix.
I close this bug (not a bug > bogus).
Thanks for your report!
------------------------------------------------------------------------
[2007-02-15 10:22:25] foster dot graeme at gmail dot com
Description:
------------
When adding files to an archive, (using successive
ZipArchive::addFile() commands) the compression doesn't happen until
the file is closed. This can result in an out of memory error, a
temporary fix is to close the archive and then reopen it within the php
code.
An idea solution would be to compress the file when it is added,
probably in function _zip_replace(), but I don't know what the
implications of this would be. It would certainly require a rewrite of
the ugly function zip_close().
------------------------------------------------------------------------
--
Edit this bug report at [url]http://bugs.php.net/?id=40494&edit=1[/url]
pajoye@php.net Guest
-
foster dot graeme at gmail dot com #6
#40494 [Bgs]: Memory problem with ZipArchive::addFile()
ID: 40494
User updated by: foster dot graeme at gmail dot com
Reported By: foster dot graeme at gmail dot com
Status: Bogus
Bug Type: Zip Related
Operating System: Linux
PHP Version: 5.2.1
Assigned To: pajoye
New Comment:
Okay thanks for the explanation, I understand the problem a little
better. I still think that it would be nice if there was some way for
the system to manage this.
I was thinking along the lines of a function to flush the files so that
the archive can be partially built prior to the ulimit being reached.
This could be set as 250, with the ability to overload it. Maybe this
would only be triggered if a flag was set when the archive was opened.
Previous Comments:
------------------------------------------------------------------------
[2007-02-15 13:23:36] [email]pajoye@php.net[/email]
See:
[url]http://pecl.php.net/bugs/bug.php?id=9443[/url]
"it would be good if this wasn't necessary, in thatthe code could catch
the problem and allocate extra file handles if that is the problem."
This is not something I can control. The operating system defines it
and there is no way for me to increase this value.
I suggest you to close and reopen it every 1000 or so (or even 255 if
you want to go on the safest way, ie old windows).
Future releases will have a different mode, where the checks will done
only when you close the archives.
------------------------------------------------------------------------
[2007-02-15 13:14:57] foster dot graeme at gmail dot com
Maybe I need to explain this problem a little more.
I am trying to archive a folder on the server, at the moment it
contains 5609 folders and 11,221 files. The script loops through the
files adding them to the archive using the addFile() method. After the
first 1002 files I get a ZIPARCHIVE::ER_OPEN. If I close the archive
and the open it again I still have that error. However, if I close the
archive and open it before I get that error then I can archive all
11,221 files.
Since closing the file and re-opening fixes the problem (so long as I
do that before I get the error) Then may I suggest that closing an
archive will clear the status. Obviously, it would be good if this
wasn't necessary, in thatthe code could catch the problem and allocate
extra file handles if that is the problem.
------------------------------------------------------------------------
[2007-02-15 11:41:24] [email]pajoye@php.net[/email]
"When adding files to an archive, (using successive
ZipArchive::addFile()
commands) the compression doesn't happen until the file is closed. "
Yes, we do it while finalizing the archive.
" This can result in an out of memory error, "
You will run out of file ID before running out of memory. It does not
really use many memory, only the file names and file handlers.
I suppose you are talking about the file handlers?
"It would certainly require a rewrite of the ugly function
zip_close()"
What is ugly in this function? Or do you have a portable way to lock a
file until the archive creation is done?
I think you refer to the file handlers limitation. There is already a
bug about it and I plan to add a special (less safe) mode. This mode
will allow one to add only the paths without checks, errors will occur
only when the archive is closed. But that's a feature addition not a
bug fix.
I close this bug (not a bug > bogus).
Thanks for your report!
------------------------------------------------------------------------
[2007-02-15 10:22:25] foster dot graeme at gmail dot com
Description:
------------
When adding files to an archive, (using successive
ZipArchive::addFile() commands) the compression doesn't happen until
the file is closed. This can result in an out of memory error, a
temporary fix is to close the archive and then reopen it within the php
code.
An idea solution would be to compress the file when it is added,
probably in function _zip_replace(), but I don't know what the
implications of this would be. It would certainly require a rewrite of
the ugly function zip_close().
------------------------------------------------------------------------
--
Edit this bug report at [url]http://bugs.php.net/?id=40494&edit=1[/url]
foster dot graeme at gmail dot com Guest
-
pajoye@php.net #7
#40494 [Bgs]: Memory problem with ZipArchive::addFile()
ID: 40494
Updated by: [email]pajoye@php.net[/email]
Reported By: foster dot graeme at gmail dot com
Status: Bogus
Bug Type: Zip Related
Operating System: Linux
PHP Version: 5.2.1
Assigned To: pajoye
New Comment:
"I still think that it would be nice if there was some way for the
system to manage this."
It is in the TODO list. As I said three times already in this
discussion. The solution is to add different modes:
- commit at the end when the archive is close
- immediate addition (will be much slower)
And again, it is in my TODOs already. I cannot tell when they will be
available (I do it on my free time).
In the meantime a simple:
if (($zip->numFiles % $yourlimit) == 0) {close; reopen;}
will do it.
"the archive can be partially built prior to the ulimit being reached.
This could be set as 250, with the ability to overload it. Maybe this
would only be triggered if a flag was set when the archive was
opened."
This solution does not work.The limit is arbitrary. There is no way to
get an exact value (and I doubt php is the only running process).
Previous Comments:
------------------------------------------------------------------------
[2007-02-15 14:02:51] foster dot graeme at gmail dot com
Okay thanks for the explanation, I understand the problem a little
better. I still think that it would be nice if there was some way for
the system to manage this.
I was thinking along the lines of a function to flush the files so that
the archive can be partially built prior to the ulimit being reached.
This could be set as 250, with the ability to overload it. Maybe this
would only be triggered if a flag was set when the archive was opened.
------------------------------------------------------------------------
[2007-02-15 13:23:36] [email]pajoye@php.net[/email]
See:
[url]http://pecl.php.net/bugs/bug.php?id=9443[/url]
"it would be good if this wasn't necessary, in thatthe code could catch
the problem and allocate extra file handles if that is the problem."
This is not something I can control. The operating system defines it
and there is no way for me to increase this value.
I suggest you to close and reopen it every 1000 or so (or even 255 if
you want to go on the safest way, ie old windows).
Future releases will have a different mode, where the checks will done
only when you close the archives.
------------------------------------------------------------------------
[2007-02-15 13:14:57] foster dot graeme at gmail dot com
Maybe I need to explain this problem a little more.
I am trying to archive a folder on the server, at the moment it
contains 5609 folders and 11,221 files. The script loops through the
files adding them to the archive using the addFile() method. After the
first 1002 files I get a ZIPARCHIVE::ER_OPEN. If I close the archive
and the open it again I still have that error. However, if I close the
archive and open it before I get that error then I can archive all
11,221 files.
Since closing the file and re-opening fixes the problem (so long as I
do that before I get the error) Then may I suggest that closing an
archive will clear the status. Obviously, it would be good if this
wasn't necessary, in thatthe code could catch the problem and allocate
extra file handles if that is the problem.
------------------------------------------------------------------------
[2007-02-15 11:41:24] [email]pajoye@php.net[/email]
"When adding files to an archive, (using successive
ZipArchive::addFile()
commands) the compression doesn't happen until the file is closed. "
Yes, we do it while finalizing the archive.
" This can result in an out of memory error, "
You will run out of file ID before running out of memory. It does not
really use many memory, only the file names and file handlers.
I suppose you are talking about the file handlers?
"It would certainly require a rewrite of the ugly function
zip_close()"
What is ugly in this function? Or do you have a portable way to lock a
file until the archive creation is done?
I think you refer to the file handlers limitation. There is already a
bug about it and I plan to add a special (less safe) mode. This mode
will allow one to add only the paths without checks, errors will occur
only when the archive is closed. But that's a feature addition not a
bug fix.
I close this bug (not a bug > bogus).
Thanks for your report!
------------------------------------------------------------------------
[2007-02-15 10:22:25] foster dot graeme at gmail dot com
Description:
------------
When adding files to an archive, (using successive
ZipArchive::addFile() commands) the compression doesn't happen until
the file is closed. This can result in an out of memory error, a
temporary fix is to close the archive and then reopen it within the php
code.
An idea solution would be to compress the file when it is added,
probably in function _zip_replace(), but I don't know what the
implications of this would be. It would certainly require a rewrite of
the ugly function zip_close().
------------------------------------------------------------------------
--
Edit this bug report at [url]http://bugs.php.net/?id=40494&edit=1[/url]
pajoye@php.net Guest
-
foster dot graeme at gmail dot com #8
#40494 [Bgs]: Memory problem with ZipArchive::addFile()
ID: 40494
User updated by: foster dot graeme at gmail dot com
Reported By: foster dot graeme at gmail dot com
Status: Bogus
Bug Type: Zip Related
Operating System: Linux
PHP Version: 5.2.1
Assigned To: pajoye
New Comment:
Would it be possible to add a brief description of this situation to
the documentation, for example the following could be added to the
description of ZipArchive::addFile
Description
bool ZipArchive::addFile ( string filename [, string localname] )
Adds a link to the ZIP archive from a given path. When the archive is
closed the link is checked to ensure that the file still exists and
will then be compressed and added to the archive. If a lot of files are
being added then the number of file handles permitted by the OS may be
exceeded, if that occurs then the status will be set to
ZIPARCHIVE::ER_OPEN. This can be avoided by closing the archive before
the limit is reached and then reopening the archive.
for example:
if ($zip->numfile % $limit == 0)
{
$zip->close();
$zip->open($filename,ZIPARCHIVE::CREATE);
}
Previous Comments:
------------------------------------------------------------------------
[2007-02-15 14:35:34] [email]pajoye@php.net[/email]
"I still think that it would be nice if there was some way for the
system to manage this."
It is in the TODO list. As I said three times already in this
discussion. The solution is to add different modes:
- commit at the end when the archive is close
- immediate addition (will be much slower)
And again, it is in my TODOs already. I cannot tell when they will be
available (I do it on my free time).
In the meantime a simple:
if (($zip->numFiles % $yourlimit) == 0) {close; reopen;}
will do it.
"the archive can be partially built prior to the ulimit being reached.
This could be set as 250, with the ability to overload it. Maybe this
would only be triggered if a flag was set when the archive was
opened."
This solution does not work.The limit is arbitrary. There is no way to
get an exact value (and I doubt php is the only running process).
------------------------------------------------------------------------
[2007-02-15 14:02:51] foster dot graeme at gmail dot com
Okay thanks for the explanation, I understand the problem a little
better. I still think that it would be nice if there was some way for
the system to manage this.
I was thinking along the lines of a function to flush the files so that
the archive can be partially built prior to the ulimit being reached.
This could be set as 250, with the ability to overload it. Maybe this
would only be triggered if a flag was set when the archive was opened.
------------------------------------------------------------------------
[2007-02-15 13:23:36] [email]pajoye@php.net[/email]
See:
[url]http://pecl.php.net/bugs/bug.php?id=9443[/url]
"it would be good if this wasn't necessary, in thatthe code could catch
the problem and allocate extra file handles if that is the problem."
This is not something I can control. The operating system defines it
and there is no way for me to increase this value.
I suggest you to close and reopen it every 1000 or so (or even 255 if
you want to go on the safest way, ie old windows).
Future releases will have a different mode, where the checks will done
only when you close the archives.
------------------------------------------------------------------------
[2007-02-15 13:14:57] foster dot graeme at gmail dot com
Maybe I need to explain this problem a little more.
I am trying to archive a folder on the server, at the moment it
contains 5609 folders and 11,221 files. The script loops through the
files adding them to the archive using the addFile() method. After the
first 1002 files I get a ZIPARCHIVE::ER_OPEN. If I close the archive
and the open it again I still have that error. However, if I close the
archive and open it before I get that error then I can archive all
11,221 files.
Since closing the file and re-opening fixes the problem (so long as I
do that before I get the error) Then may I suggest that closing an
archive will clear the status. Obviously, it would be good if this
wasn't necessary, in thatthe code could catch the problem and allocate
extra file handles if that is the problem.
------------------------------------------------------------------------
[2007-02-15 11:41:24] [email]pajoye@php.net[/email]
"When adding files to an archive, (using successive
ZipArchive::addFile()
commands) the compression doesn't happen until the file is closed. "
Yes, we do it while finalizing the archive.
" This can result in an out of memory error, "
You will run out of file ID before running out of memory. It does not
really use many memory, only the file names and file handlers.
I suppose you are talking about the file handlers?
"It would certainly require a rewrite of the ugly function
zip_close()"
What is ugly in this function? Or do you have a portable way to lock a
file until the archive creation is done?
I think you refer to the file handlers limitation. There is already a
bug about it and I plan to add a special (less safe) mode. This mode
will allow one to add only the paths without checks, errors will occur
only when the archive is closed. But that's a feature addition not a
bug fix.
I close this bug (not a bug > bogus).
Thanks for your report!
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
[url]http://bugs.php.net/40494[/url]
--
Edit this bug report at [url]http://bugs.php.net/?id=40494&edit=1[/url]
foster dot graeme at gmail dot com Guest
-
pajoye@php.net #9
#40494 [Bgs]: Memory problem with ZipArchive::addFile()
ID: 40494
Updated by: [email]pajoye@php.net[/email]
Reported By: foster dot graeme at gmail dot com
Status: Bogus
Bug Type: Zip Related
Operating System: Linux
PHP Version: 5.2.1
Assigned To: pajoye
New Comment:
Yes, it can be added to the doc.
However your explanation is not correct. The files are open and kept
open until the archive is closed (that's why you reach the handlers
limit). It is the only safe way to lock a file and be sure it exists
when when we finalize the archive.
Previous Comments:
------------------------------------------------------------------------
[2007-02-15 15:15:13] foster dot graeme at gmail dot com
Would it be possible to add a brief description of this situation to
the documentation, for example the following could be added to the
description of ZipArchive::addFile
Description
bool ZipArchive::addFile ( string filename [, string localname] )
Adds a link to the ZIP archive from a given path. When the archive is
closed the link is checked to ensure that the file still exists and
will then be compressed and added to the archive. If a lot of files are
being added then the number of file handles permitted by the OS may be
exceeded, if that occurs then the status will be set to
ZIPARCHIVE::ER_OPEN. This can be avoided by closing the archive before
the limit is reached and then reopening the archive.
for example:
if ($zip->numfile % $limit == 0)
{
$zip->close();
$zip->open($filename,ZIPARCHIVE::CREATE);
}
------------------------------------------------------------------------
[2007-02-15 14:35:34] [email]pajoye@php.net[/email]
"I still think that it would be nice if there was some way for the
system to manage this."
It is in the TODO list. As I said three times already in this
discussion. The solution is to add different modes:
- commit at the end when the archive is close
- immediate addition (will be much slower)
And again, it is in my TODOs already. I cannot tell when they will be
available (I do it on my free time).
In the meantime a simple:
if (($zip->numFiles % $yourlimit) == 0) {close; reopen;}
will do it.
"the archive can be partially built prior to the ulimit being reached.
This could be set as 250, with the ability to overload it. Maybe this
would only be triggered if a flag was set when the archive was
opened."
This solution does not work.The limit is arbitrary. There is no way to
get an exact value (and I doubt php is the only running process).
------------------------------------------------------------------------
[2007-02-15 14:02:51] foster dot graeme at gmail dot com
Okay thanks for the explanation, I understand the problem a little
better. I still think that it would be nice if there was some way for
the system to manage this.
I was thinking along the lines of a function to flush the files so that
the archive can be partially built prior to the ulimit being reached.
This could be set as 250, with the ability to overload it. Maybe this
would only be triggered if a flag was set when the archive was opened.
------------------------------------------------------------------------
[2007-02-15 13:23:36] [email]pajoye@php.net[/email]
See:
[url]http://pecl.php.net/bugs/bug.php?id=9443[/url]
"it would be good if this wasn't necessary, in thatthe code could catch
the problem and allocate extra file handles if that is the problem."
This is not something I can control. The operating system defines it
and there is no way for me to increase this value.
I suggest you to close and reopen it every 1000 or so (or even 255 if
you want to go on the safest way, ie old windows).
Future releases will have a different mode, where the checks will done
only when you close the archives.
------------------------------------------------------------------------
[2007-02-15 13:14:57] foster dot graeme at gmail dot com
Maybe I need to explain this problem a little more.
I am trying to archive a folder on the server, at the moment it
contains 5609 folders and 11,221 files. The script loops through the
files adding them to the archive using the addFile() method. After the
first 1002 files I get a ZIPARCHIVE::ER_OPEN. If I close the archive
and the open it again I still have that error. However, if I close the
archive and open it before I get that error then I can archive all
11,221 files.
Since closing the file and re-opening fixes the problem (so long as I
do that before I get the error) Then may I suggest that closing an
archive will clear the status. Obviously, it would be good if this
wasn't necessary, in thatthe code could catch the problem and allocate
extra file handles if that is the problem.
------------------------------------------------------------------------
The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
[url]http://bugs.php.net/40494[/url]
--
Edit this bug report at [url]http://bugs.php.net/?id=40494&edit=1[/url]
pajoye@php.net Guest



Reply With Quote

