On the Working Software blog there's a new post showing a solution to a issue with escaping quotes in XPath queries that's not just an issue in PHP.

I've been working with the Basecamp API to plugin our IRC bot that we use for time tracking and I'm astounded to learn that escaping single and/or double quotes for XPath queries in PHP does not have a well documented, best practices solution. In fact, it seems as though this is not peculiar to PHP. I took a look around and found this excellent article by "Kushal": http://kushalm.com/the-perils-of-xpath-expressions-specifically-escaping-quotes.

He's put together his own (PHP) solution to the problem - running the entire XPath query through a filtering method that splits it up, replaces the quote characters and combines it back down to a single string.

On the Script-Tutorials.com site there's a new article showing you how to use XPath to parse web pages, complete with screenshots of the page and the code to make it happen.

Today I will tell you how you can make parsers of remote HTML pages (in PHP). In this article I will show you how to perform xpath queries to Web pages. XPath - a query language to elements of xml or xhtml document. To obtain the necessary data, we just need to create the necessary query. For the work, we also need: browser Mozilla Firefox, firebug and firepath plugins. For our experiment, I suggest this webpage Google Sci/Tech News. Of course you can choose any other web page too.

They provide two demos and a downloadable package with everything you need. The script pulls in the page as a DOM document (which works as long as it's correctly formatted XML) and spits back out the matches from a few different XPath expressions. There's all sorts of sites out there that can help you with examples of other XPath expressions and syntax.

On the Qafoo blog today there's a new post from Tobias Schlitt about a method you can use to unit test methods that generate XML without messing with a lot of extra overhead just to test the results.

Testing classes which generate XML can be a cumbersome work. At least, if you don't know the right tricks to make your life easier. In this article, I will throw some light upon different approaches and show you, how XML generation can be tested quite easily using XPath.

He includes a sample class, qaPersonVisitor, that has methods inside it to create a simple XML documents based on the first and last name data into a DOM element. He sets up the basic test case that creates a simple person - including gender and date of birth - and offer a few different suggestions on handling the check (in PHPUnit tests):

• the naive way of rebuilding the DOM object and assert that they are equal
• testing the resulting XML from the DOM object against a pre-generated XML document
• matching the contents via CSS selectors
• using the tag matching assertions
• using XPath in a custom assertion (with short and long uses of it included)

Thomas Weinert has a recent post to his blog showing how to use one of the more powerful XML-handling features that PHP's DOM extension includes - XPath.

Often I hear people say "We use SimpleXML, because DOM is so noisy and complex". Well, I don't think so. This article explains how you can parse a XML (an Atom feed) using the PHP DOM extension. No other libraries are involved.

In his example he loads an external feed (his own) into a DOM object, blocks any errors with a few handy functions and creates a DOMXPath object on the DOM object to get ready for his queries. He shows how to make searches for titles, subtitles, looping over attributes and an element list returned from one of the first queries. A full code listing is also provided to show how it all fits together.

On PHPBuilder.com today there's a new tutorial posted about building a link scraping script with the combination of PHP and cURL (the script pulls in a page, grabs all of the links off of it and follows them, etc).

I actually built this a few years ago because I had grandiose visions of becoming the next Google. Clearly, that did not happen, mostly because my localhost, database, and bandwidth are not infinite. Yet this little robot has quite interesting applications and uses if you really have the time to play with and fine-tune it.

You'll need to have cURL support built into your PHP installation to get the scripts working, but the actual code itself is pretty simple. Curl and XPath do most of the heavy lifting of finding and following the links and its easy enough to drop them into a MySQL table from there. You can download the source here.

In this new post to his blog Juozas Kaziukenas takes a look at one method for getting the information out of a remote page - parsing it with PHP and XPath (assuming the page is correctly formatted).

When I was writing about how I use web scraping, I was still hadn't tried using Xpath (shame on me). [...] It turned out, that using Xpath is extremely easy, really. When you master it, you can do everything in seconds. Yes, you need to know how XML works and how to write correct Xpath queries (brief explanation of Xpath syntax is available at W3Schools), but hey - these topics are in 1st year of university.

He includes both some sample code (to fetch a titles and prices for cameras from bhphotovideo.com) and a link to a XPath checker you can use to ensure that your query is correctly formatted. It's good that he also includes a quick reminder about the ethical issue with web scraping - it could be considered stealing depending on where the information comes from and who is providing it.

Tobias Schlitt and Jacob Westhoff have written up and article (and posted it over on Tobias' blog) that wants to help you understand XPath better - whether you're a novice or have been using it for a while.

This paper will give an overview on XPath an addressing language for XML documents. XPath is a W3C recommendation currently in version 1.0. XPath was created in relation to the XSL recommendation and is intended to be used with XSLT and XPointer. Beside that, XPath can be used in a variety of programming languages, commonly in combination with a DOM API.

The article starts with an introduction to the concept of XPath, moves on to addressing, talks about XPath axes, mentions functions/operators/conditions and looks at styling XML with XPath and XSLT.

In response to this tutorial over on the PHPro.org website a new post has been made to the PHP in Action blog with an "even cooler" way to do the same sort of DOM fetching - XPath.

I'm a little bit surprised at the claim [the tutorial makes] that it's the "correct" (only) way, since there's at least one more that I find even cooler: XPath. Admittedly, it's slower, yet it's a more powerful language.

A quick example is included, building up from a simple search for anchor tags up to a custom query looking for just the anchor tags with a class of "bookmark".

On the Debuggable blog there's an interesting post where Felix talks a bit about something I've seen requested quite a bit - a method for locating information in an array. His answer is an XPath-style query system to root out your custom information.

One of the requirements [of the original Set::extract method] was that the new method would need to be faster or at least as fast as the old implementation. My first attempts were big failures. Not only did the solutions I came up with contain tons of bugs. No, they were are also a lot slower the old extract function. A few benchmarks later and I discovered the biggest bottleneck in my implementation: Recursiveness.

He notes that no doing things recursively (not just in this situation, but ever) can help with a speed boost. In his example, a small change made all the differences and the XPath implementation in the CakePHP core makes grabbing information from any array simple.

While the implementation does not support full XPath (and probably won't in future), feel free to make suggestions on additional selectors or the idea in general.

Chris Hartjes has answered some more questions his readers have asked in a new post to his blog today. This time the focus is on XML handling.

Welcome to the 3rd installment of me answering reader feedback questions. Today we deal with a topic that I deal with every day at my day job - working with XML in PHP.

He talks about SimpleXML (and how well it does its job) and their (his work's) current method of handling the storage of XML in a database. He describes both their current process and his ideal one, how he'd want to interface with their eXist backend.