Alex Mills has a interesting post to his blog answering a question he and several of the other WordPress developers at Automatic get about their blogging engine - why it doesn't including a default caching layer.

WordPress does actually have a built-in cache called the object cache. It was introduced way back in 2005 1 and it basically caches database query results. [...] However as soon as the page is done being generated, that object cache is discarded. Initially the object cache cached these little chunks of data to the filesystem so that they could be reused on subsequent pageviews. While great in theory, the concept turned out to be terrible in practice.

They opted out of the persistent object caching because, well, it was slower than some of the preexisting alternatives out there like database caching and caching plugins (with WP Super Cache).