News Feed
Sections




News Archive
Looking for more information on how to do PHP the right way? Check out PHP: The Right Way

James Cohen's Blog:
Poor Man's Parallelization for Batch Processing Jobs
May 18, 2011 @ 11:56:31

James Cohen has a quick post about what he calls a "poor man's parallelization" for working with batch jobs. It takes in parameters that tell it which set of jobs to run through when looping.

One common problem that I've seen time and time again with batch processing jobs (generally cronjobs) is that when they're written they run quickly. Over time their workload grows until eventually it's unacceptably slow. [...] To create a simple of way of separating the jobs in a consistent way we can use the modulus operator. It just calculates the remainder of two numbers. It's a common arithmetic operator in almost all languages so this technique is pretty portable.

His proof-of-concept script takes in two parameters, the starting job number and the number to increment. His example is user IDs, but this type of script could be used for anything with an ID number. The script is then run from the command line with the parameters of your choosing.

0 comments voice your opinion now!
batch processing separation modulus


blog comments powered by Disqus

Similar Posts

DeveloperTutorials.com: AJAX and PHP Form Processing

WorkingSoftware Blog: Your templating engine sucks & everything you've written is spaghetti code

Stefan Mischook's Blog: Video Tutorial: Processing forms with PHP, Part 3

James Cohen's Blog: Poor Manís Parallelization for Batch Processing Jobs

Wez Furlong's Blog: Background/batch/workflow processing with PDO::PGSQL


Community Events

Don't see your event here?
Let us know!


interview library laravel5 api opinion introduction framework community language list voicesoftheelephpant unittest php7 extension version podcast wordpress laravel series release

All content copyright, 2015 PHPDeveloper.org :: info@phpdeveloper.org - Powered by the Solar PHP Framework