News Feed
Sections




News Archive
feed this:

Looking for more information on how to do PHP the right way? Check out PHP: The Right Way

Developer Tutorials Blog:
Parallel web scraping in PHP cURL multi functions
July 29, 2008 @ 07:57:00

The Developer Tutorials blog has posted a tutorial about scraping other website information in parallel (with their permission, of course) with the help of the cURL extension.

For anyone who's ever tried to fetch multiple resources over HTTP in PHP, the logic is trivial, but one key challenge is ever-present: latency delays. While web servers have perfectly good downstream links, latencies can increase script execution time tenfold just by downloading a few external URLs. But there's a simple solution: parallel cURL operations. In this tutorial, I'll show you how to use the "multi" functions in PHP's cURL library to get around this quickly and easily.

He starts with a basic cURL example, grabbing the content from example.com and putting it into a variable. He modifies this to make it a bit more complex and to run multiple fetches in parallel - creating more than one cURL object and using the culr_multi_* methods to manage them.

0 comments voice your opinion now!
webscraping curl function multi parallel tutorial


Michael Kimsal's Blog:
New antipattern? "Multi Master Data"
June 29, 2006 @ 07:00:35

If you've been programming for any length of time, you know the "joy" of working with someone else's code. When taking over a project, the first inclination I've seen with most programmers is to go in and format everything to what they'd like (or duplicate functionality). This is where the problem comes in, the trend that Michael Kimsal talks about in his new blog post - something he wonders about being an "antipattern".

I was discussing things with my brother the other day and I came up with a problem which he helped name. I'm currently maintaining some code, and it's quite a jumble. One of the things I can tell is that one of my predecessors began adding new sections of code to clean up the logic in other areas of the code. However, what never happened was the clean up of the old code, so now there's two places where the same set of data is retrieved in different ways.

He proposes the name "Multi Master Data" for the situation - two different sources, living in the same code, doing the same thing. Of course, he also mentions a situation where this type of problem can cause real issues, especially when trying to track down a bug (a bang your head on the desk moment).

0 comments voice your opinion now!
multi master datr antipattern clean multi master datr antipattern clean



Community Events





Don't see your event here?
Let us know!


interview community laravel list series podcast language testing developer release opinion install symfony2 refactor threedevsandamaybe unittest code introduction api framework

All content copyright, 2014 PHPDeveloper.org :: info@phpdeveloper.org - Powered by the Solar PHP Framework