I would rather not have to convert these weird characters to the HTML character entities, if possible. I'd rather be able to use these characters directly on the web page. If this is for some reason a bad idea, let me know. This might be more of a general web design question (i already posted it there), but I figured it is still appropriate to post here as well since PHP is used to pull an entry from the database, and I figured a lot of you here would know the answer to the question.

The general consensus is to use UTF8 in this case, but there's a few reminders for the poster too:

• Don't forget to make the database UTF8 too
• Be sure you're sending the right Content-Type for the UTF8 data
• an link to an article about what "developers must know about unicode/charactersets"

Character sets can be confusing at the best of times. This post aims to explain the potential problems and suggest solutions. Although this is applied to PHP and a typical LAMP stack you can apply the same principles to any multi-tier stack.

He includes a "boring history" session (and recommends skipping if you just want the good stuff) that talks a bit about character sets and their history in computer system handling. All that said, he recommends using UTF-8 to ease your character encoding woes. He talks about configuring your editor to support it, making sure your browsers understand it and setting up your MySQL database connection to use it.

The process of making your applications/websites usable in many different locales is called internationalization, While customizing your code for different locales is called localization. Localization is the process of making your applications or websites local to where it is being viewed. For example, you can make a website more local to a particular place by converting its text to the predominate language of that location and by displaying the local time (e.g. German for people living in Germany or French for people living in France).

They show how to define constants that can be used in your application for the character set and language encoding. They use two major encodings - UTF-8 and ISO-8859-1 - in their examples of showing a sample "welcome" message in different languages. There's also a simple page to show you how to switch between languages if you'd like to give your visitors the option.

I realized that while this 3.5-year PHP consultant knew Unicode, UTF-8, character encodings such as ISO-8859-1 or ISO-8859-7, I didn't understand them as well as I thought I had. With that I threw this chapter in the book. Knowing about character encoding is what many developers have. Not as many truly understand it. In this chapter I try to de-mystify character encoding as a whole.

The excerpt introduces character encoding and what it really is - a translation for the computer to be able to handle the human language. The problem comes in when multiple tools try to define the same sort of letters/chatacters in different ways. He gives an example of a "hello world" string in a normal ASCII format versus one from the EBCDIC format and how it would be rendered by an ASCII-understanding browser.

Several people made a valid point in response to that post [about fundamentals in software development], in that it is often difficult to know what the fundamentals actually are (be they macro or micro level). So, I thought it would be a good idea to do an ongoing series of posts on some of the things that I consider to be fundamental - this post is the first installment.

He defines the structure of a URL (the scheme, host, port, etc), reserved characters and absolute versus relative URLs and how they're usually used.

Suffice to say, [WP-Syntax] does the job fine except for one very annoying issue. Whenever you have any kind of special characters in your code (which you inevitably do e.g. <, >, & etc.), these always render as their escaped representations.

It only happens with the WYSIWYG editor for writing content (which the tool's FAQ recommends turning off) but there's a better solution that Alan found - change a line of code in the plugin to use the htmlspecialchars_decode function to rid your install of this pesky escaping bug.

Basically, data is coming into our PHP application through a Javascript's XMLHttpRequest (ajax). [...] Normally everything in the $_POST and $_GET arrays is already decoded, so when you're dealing with these arrays you don't really have to think about this. This time however, I was dealing with some non-latin unicode characters and for some reason they were never decoded and ended up in the database as raw url-encoded strings.

The issue was caused by the fact that the non-latin characters were handled differently on the byte level and that, because of the choice PHP makes on which of the unicode characters to use for spaces, he found that there were two options for handling strings - escape() and encodeURI().

I recently worked with PHP's DOM XML extension while working on Zend Framework's Zend_Search_Lucene HTML highlighting capabilities, and uncovered some undocumented features and issues with the extension in regards to character encoding. The information contained in this article should also apply to other libxml-based DOM implementations, as PHP's DOM extension simply wraps that library.

There's five different tips he shares:

• Internal document encoding is always UTF-8
• Input data is always treated as UTF-8
• Text nodes and CDATA are stored as UTF-8 without transformations
• Document encoding does not affect loading behavior
• Save/dumping operations and encoding

He describes each of the points and includes some sample code and XML to parse to help illustrate each.

In the last part of this series we looked at what a variable was in general. Today we'll be covering strings and text. We'll look at the contents those variables would typically hold.

Topics include the differences between single and double quotes, appending to a string and special characters (like line feeds and tabs). Code snippets are provided for each to help you visualize what they mean.

Everyone else probably already knows this stuff, but I hit an issue today to that took a while to sort out. Fortunately, some kind folks on IRC helped me, but as it's embarrassing to ask for help on the same issue twice, I'm writing down what I've learned! The problem: Get a £ character stored to MySQL, retrieved and then displayed without any weird characters in front of it using UTF8.

His solution? Make sure you're using UTF-8 everywhere, not just when trying to insert into the database - in the broser's headers (both going in and coming out) and in the MySQL database insert. He gives code examples for each including database examples for PDO and the Zend_Db component of the Zend Framework.