Facebook was kind enough to open source the XHProf extension last year, but it flew under my radar until I saw a presentation including it earlier this year when they showed off HipHop. XHProf provides profiling information about your application, while being lightweight enough to run on a production server (against a percentage of requests). Once we got it installed we ran into a few limitations with the existing GUI.
His deadlines were tight, so he admits the code isn't as robust as it could be, but he does offer a tool that uses MySQL for data storage, keeps stats on things like peak memory usage and wall time and uses the Google Data Visualization API to run graphs of the data over time. You can see a few screenshots of the tool in action in the post including this sample graph and this comparison of two different runs against the same resources.
The project is available for download/contribution on github.