I had a problem last week with converting UTF-16 encoded strings to UTF-8 using PHP's iconv library on a Linux server. my code worked fine on my machine but the same code resulted in a rubbish unreadable characters on our production server.

In his example (a basic "Hello World" in Arabic) he notes that there's no byte order mark on the string and, because of this, the iconv feature tries to guess if it's big-endian or little-endian. This guessing varies from machine to machine resulting in the inconsistencies he saw. The solution is to define the "to" and "from" for the conversion manually rather than letting it just guess.

One such thing is the occurrence of the UTF byte order mark, or BOM. [...] For UTF-8, especially on Windows, it has become more and more common to use it to indicate that the file is indeed UTF. Most text editors handle this well and you won't ever see these bytes. As it should be.

He points out what could cause an issue - using strcmp or substr but it can be prevented by looking at and removing those first three bytes if needed. He includes a snippet of code that does just that.

We ran into a really hard to find bug in the cookie handling of Zend_Http_Client, which has been filed as a bug back in August 2007 against version 1.0.1 (today we are at 1.5.2). More over this is a bug that other similar packages have gotten over in 2004.

He had to use wireshark to finally track down the culprit - a call to urlencode on the contents of the cookie before sending it. He also includes some code to overcome a problem he had with UTF-16 in one of his feeds (a custom function that takes in and returns a string translated correctly).