0

First of all sorry to post a question that seems to have been flogged to death on SO before. However, none of the questions I have reviewed helped me to solve my specific problem.

I have built a web application that runs an extensive data processing routine in PHP (i.e. MySQL queries, calculations, etc.).

Depending on the amount of data fed to the app this processing can take quite a long time so the script needs to run server-side and independently from the web front-end.

There is a problem, however. It seems I cannot control the script execution time limit as long as the script is invoked via cgi.

When I run the script via SSH and the command line it works fine for however long it takes to process the data.

But if I use the exec() command in a php script called via the webserver I always ends up with the error End of script output before headers after approximately 45 seconds.

Rather than having to fiddle with server settings (a nightmare in terms of portability) I would like to find a solution that kicks off the script independently from cgi.

Any suggestions?

Improve this question
8
  • " independently from cgi." so mod_php ? Commented Jan 9, 2015 at 1:50
  • Yes, but no. It's a hosted solution with limited access to server configuration. So I have to find a solution that works when mod_fastcgi is default Commented Jan 9, 2015 at 1:55
  • could you show us the php code? and did you check any other command exec() that doesn't need much time for execution, does it work? only time limit is an obstacle? Commented Jan 9, 2015 at 1:57
  • Well, the code is massive and I'm not sure that it helps. As I said, it works fine as long as I execute the script via SSH. Only if invoked via HTTP I end up with the End of script issue Commented Jan 9, 2015 at 1:59
  • Long running jobs should have nothing to do with a web server at all. There should be an infrastructure independent of the web server which can run long running jobs; e.g. a queue-worker system such as Gearman, ØMQ, cron jobs or such. If your server is so limited as to disallow such infrastructure, it may simply not be possible on that system. Commented Jan 9, 2015 at 2:00

1 Answer 1

Reset to default
2

Don't execute the long script directly from the website (AKA, directly from Apache) because, as you've mentioned, it will block until it finishes and potentially time out. Instead, use the website to schedule a job (an execution of the long script) to be run immediately.

Here is a basic outline of how you can potentially do this:

  1. Create a new, small database to store job requests, including fields job_id, processing_status, run_start_time, and more relevant fields
  2. Create some Ajax that hits your server and writes a "job request" to this jobs database, set to execute immediately.
  3. Add a crontab script or bot that periodically watches for new jobs. If it finds a job that is yet to be processed but has passed the run_start_time, run it using exec() or some other command executor. This way the command won't timeout because it is not being run by Apache, but by the cron daemon.
  4. When the command finishes, update the jobs database saying that processing is finished.
  5. From your website, write a frontend that allows the user to see if the requested job is finished yet. Once it finishes, it displays some kind of "Done" indicator or something similar.
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.