out of memory doing an database scan

peterle · April 19th, 2010, 03:39 PM

Hi Chuck,

a few months ago I asked for a similar problem:
mod_fcgid: read data timeout in 40 seconds - workaround?

The problem was caused by an apache worker configuration with an standard fcgimod having some problems using its own config file.

Since this system is quite complicated updating it, changed back to apache prefork with a phpmod.

Now I tried a "Check for and create missing thumbnails" and got stuck with this message:
Checking photo #51760: done.
Checking photo #51761: done.
Checking photo #51762: done.
Checking photo #51763: done.
Checking photo #51764:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 24 bytes) in /var/www/portal/galerie/pp-inc.php on line 373

Looks to my like a script sucking up all the memory I have, which is:

Code:

Content visible to verified customers only.

Any suggestions on that one?
Yes, I know, we have more than 2 or 3 pictures ...

Code:

Content visible to verified customers only.

Thanks for any help.

Chuck S · April 19th, 2010, 06:09 PM

well one suggestion I would say is to install Imagemagick on the server and use that as GD2 is part of PHP and uses PHP memory.

peterle · April 20th, 2010, 04:05 AM

I am afraid, I am doing so ...

Code:

Content visible to verified customers only.

Chuck S · April 20th, 2010, 06:40 AM

what version of photopost you using?

Code:

Content visible to verified customers only.

that is 373 for me in the mysql_query function.

peterle · April 20th, 2010, 09:54 AM

Current Version 6.11

older one, but before updating I would like to have this version working again without any problems.

VirtualBulletin is 3.8.2

Chuck S · April 20th, 2010, 10:23 AM

6.11 is a little old yes I really dont know the code in that older version so I doubt there is much to comment on here.

Timeouts are PHP settings however it is clear your already using 128 mb of php memory there. You can try upping that but this is a server memory issue here.

You can update your code and see if that helps in the overall overhead.

peterle · April 21st, 2010, 04:37 AM

Sounds a bit like "lay back and think of England"

I will tell you what happend then ...

Chuck S · April 21st, 2010, 08:24 AM

Hard for me to make suggestions on old out dated code.

peterle · April 21st, 2010, 10:40 AM

Ok, you got me.
The latest version will still work with vb 3.8?

peterle · March 13th, 2011, 07:54 PM

Hi Chuck,

I updated to 7.1 a few days ago and still have the same problem with database scans

Checking photo #44249: done.
Checking photo #44250: done.
Checking photo #44251: done.
Checking photo #44252: done.
Checking photo #44253: done.
Checking photo #44254: done.
Checking photo #44255: done.

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) in /var/www/portal/galerie/adm-misc.php on line 1452

the php.ini says:

Code:

Content visible to verified customers only.

and

Code:

Content visible to verified customers only.

here are the statistics:

Code:

Content visible to verified customers only.

It's a root server running only as a test system, with nothing else to do except running that script.

I would be very glad to find a solution for that problem, even if we have to use shell scripts instead of the php scripts.

Chuck S · March 14th, 2011, 08:07 AM

Your using GD2 to process images so no real workaround other than to use Imagemagick.

Your getting a PHP memory error which is related to GD2 needing more memory to process an image

peterle · March 14th, 2011, 09:24 AM

Quote:

Originally Posted by Chuck S

Your using GD2 to process images so no real workaround other than to use Imagemagick.

Your getting a PHP memory error which is related to GD2 needing more memory to process an image

[Sammy Davis Jr.]
Yes I can!
[/Sammy Davis Jr.]

- at least I hope so.

Chuck S · March 14th, 2011, 09:37 AM

That specific error is well documented here. One can give more memory to PHP and set max height width parameters in Upload Options in Admin to say 1200 or less but GD2 with larger images is very prone to cause PHP memory issues. Imagemagick uses server memory not PHP so it is not prone specifically to crash like GD2 causes PHP to do. Imagemagick can regularly handle larger images with much more reliability.

peterle · March 14th, 2011, 12:22 PM

Quote:

Originally Posted by peterle

[Sammy Davis Jr.]
Yes I can!
[/Sammy Davis Jr.]

- at least I hope so.

Quote:

Originally Posted by Chuck S

That specific error is well documented here. One can give more memory to PHP and set max height width parameters in Upload Options in Admin to say 1200 or less but GD2 with larger images is very prone to cause PHP memory issues. Imagemagick uses server memory not PHP so it is not prone specifically to crash like GD2 causes PHP to do. Imagemagick can regularly handle larger images with much more reliability.

Well, like Sinatra said in a remarkable show: No you can't.

I installed imagemagick, checked that the options in PPpro are set to Imagemagick and tried again, but I am back at the starting point one year ago:

Checking photo #44250: done.
Checking photo #44251: done.
Checking photo #44252: done.
Checking photo #44253: done.
Checking photo #44254: done.
Checking photo #44255: done.
Checking photo #44256:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 81 bytes) in /var/www/portal/galerie/pp-inc.php on line 385

Chuck S · March 14th, 2011, 12:44 PM

Does your server have an issue with the PHP function microtime?

Code:

Content visible to verified customers only.

The last line is line 385 of the pp-inc.php file in the current distribution that I see, I am by no means a server tech and know lots on server setup. Your host may be more familiar with what might cause that issue. I do not see anything code related to our software that would cause an issue.

You can try commenting out that totaltime line with // to see if that helps but it all goes back to what on the server is triggering the issue.

peterle · March 15th, 2011, 06:57 AM

There seems to be a problem using microtime on servers which run ntpd - what all my servers do.
microtime & ntp
So I turned it off and updated php to php 5.3.3.7.

The new error is:

Checking photo #42579: done.
Checking photo #42580: done.
Checking photo #42581: done.
Checking photo #42582: done.
Checking photo #42583: done.
Checking photo #42584: done.
Checking photo #42585:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) in /var/www/portal/galerie/pp-inc.php on line 389

line 389 of pp-inc.php says:

Code:

Content visible to verified customers only.

Chuck S · March 15th, 2011, 09:47 AM

Your still crashing PHP.

microtime is a standard accepted and widely used PHP command. You can try time instead of microtime but without some backend server log info on the true error it is hard to comment.

Can your host give you an info on what is triggering a PHP timeout?

You can also try and edit adm-misc.php and change the $maxcount line from 200 to say 100. It is going to take lots of testing on your part on that server to determine the issue.

peterle · March 15th, 2011, 10:50 AM

Thanks for your assistance - what level of error logging would you like?

Chuck S · March 15th, 2011, 10:59 AM

I am really just asking if any of your logs have any detailed information other than a PHP memory issue. Basically PHP shuts down when that happens which in turn shuts down the script. I do not know of any type of replacement for PHP microtime. Like I said you can try time. The only difference is going to be micro seconds would not be passed just down to the second.

We know your server setup does no appear to support some widely used PHP function(s) so I am just seeing if there is anything else here that we can comment on. It looks to me like your PHP memory is set to what 128mb? Thats really high so whatever is balking things down has to have some kind of trail here to follow I would think.

peterle · March 15th, 2011, 11:18 AM

I will try to narrow it down and come back - hope not in a year.