PHP MVC Framework Performance – Part 1
As part of my assessment of PHP MVC Frameworks which I blogged about in Choosing the best PHP MVC Framework I decided to do a compare performance of the various PHP MVC Frameworks I was considering. I did this using Apache Bench with ab -t 30 -c 10 http://testserver/framework/index.php as this is common and did the basic Hello World test where each framework just put out Hello World in an empty HTML document. Here are the results…
The test server was a Windows XP machine with T7200 2.00GHz CPU and 2GB RAM running WAMP with PHP 5.2.6, Apache 2.2.8. No opcache was used in this test altough that’s something I will probably use for another test.
The latest version of every framework was used and all features like DB access were disabled for the test to give a true comparison.
All tests were run 5 times and any results there weren’t around the average of other results were removed and the test run was repeated, to avoid skewing results.
It’s pretty pointless to provide figures in requests per second as every machine is different and everyone will hence get different results, but it makes a lot of sense to compare all frameworks against a common base. For that base I chose to use performance of raw PHP, which on my test machine did an average of 740 req/s. A test with a HTML test on the same machine did 850 req/s (so not a huge difference). I should also note that the exact file being requested was always used (e.g. index.php or index.html), as just referencing the directory further decreased performance because Apache had to work out which file to show for the directory index.
Here are the results:
- Raw PHP - 740 req/sec – 100%
- PHP Pro MVC 0.0.4 - 200 req/sec – 27%
- DooPHP 1.2 – 170 req/sec – 23%
- Yii Framework 1.1.1 – 130 req/sec – 18%
- Kohana PHP 2.3.4 – 55 req/sec – 7.5%
- CodeIgniter 1.7.2 – 38 req/sec – 5%
- Zend Framework 1.10 – 24 req/sec – 3%
So what does this all mean, let’s analyse the results…
Firstly PHP Pro MVC is a minimalistic MVC framwork written as an example of how to write an MVC framework and is used by OpenCart (with some modifications). I used it in the comparison to get an understanding how how any sort of even a basic PHP framework would impact performance.
As claimed on their website DooPHP is indeed the fastest of the frameworks, but it’s nowhere near as fast as they claim to be as compared to other frameworks. I also wouldn’t recommend using it because it’s not very logically structured compared to some others. Speed isn’t everything.
Yii is also indeed quite fast and to my surprise was significantly faster than Kohana or Code Igniter! However I tested v1.1.1 and Yii Framework Performance Bencharks were done on version 1.0.2. As I’ve said before I quite like Yii Framework because it just feels natural and logical.
Kohana was a bit faster than Code Igniter but in the same ballpark. As I’ve also pointed out I think that Yii is better than Kohana because code completion in PHP IDEs doesn’t work with Kohana and that’s a major issue.
Code Igniter was a bit slower than I would have hoped it would be, but that’s understandable because of all the interoperability code that doesn’t really need to be there if you’re using PHP5 only. This is consistent with Kohana vs Code Igniter Perofmance results obtained by other people.
Zend Framework is the slowest of the lot, which is understandable given how heavy the framework is and how many files get loaded just to render a template from a controller. This is also something that’s been pointed out in tests by other people.
I didn’t bother testing Cake PHP or Symfony as it’s well know that they are even slower than Zend Framework.
Now, here’s something VERY IMPORTANT!
You can compare performance of frameworks on Hello World tests all you want, but in a real life situation the performance of your framework will be dwarfed into insignificance by the following 2 factors:
- Database Access
- File System Access
Database Access and File System Access are the two slowest operations there are so how quick the actual code runs matters very little. In fact, the reason why Zend is so much slower than others is because it splits the code between so many files and it’s not even the parsing that’s the biggest problem, but file access, as I was running without opcode caching. With opcode caching you can expect the performance to improve to a point where the performance difference between frameworks will be insignificant in comparison to database access operations.
So, based on all of this my recommendation are as follows:
- Choose the framework that best suits your needs and don’t worry so much about raw performance (unless the framework is split between many many files and you’re not using opcode caching, in which case file system access becomes a limiting factor). You should consider other factors when about the PHP Framework as I’ve outlined in Choosing the best PHP MVC Framework.
- Try to use opcode caching wherever possible to avoid file system access becoming a limiting factor.
- Make sure you optimise your database performance (read queries and indexes) and try to minimise the number of database accesses (especially in loops) by using joins and retrieving multiple rows with a single query. This one thing will dwarf any difference you may see in PHP MVC Framework performance!
I prefer to use Code Igniter from all the choices available… as its the easiest and well documented…
Yes, CI is quite nice as it’s simple to use, has a large community and is well documented. The downside I found with CI is that it is a little out of date and for some reason slower than I thought! After looking at CI and Kohana and paying around with them I found that BOTH have issues with code completion in an IDE, so I do still think that Yii is better as a framework as it feels natural, performs better than CI and can be used easier in an IDE, which boosts productivity enormously. This is difficult to explain to someone who’s been using a hyped up Notepad or Dreamweaver for programming, but anyone who’s used to an IDE like Visual Studio, Eclipse or NetBeans will know exactly what I mean. It is also written using the latest PHP design concepts and is purely OO (not that I think that this is particularly important). What I don’t like about Yii is that the entire frameworks is based around Active Record and ORM, which is both bad for performance and un-necessary in my view. There is nothing wrong with just doing SQL queries, where you have control over exact SQL that gets run and don’t need to learn another language as such to write SQL.
I thing CI does not have some things that cakephp or Yii for example have.
For example the command line tool that create controllers etc and a crud application that is useful,there is better active reconrd implementetion and so on.
Ci is maybe simple but finally other frameworks make your life easier even maybe more difficult for someone.
I used CI for 6 months before I turn to cakephp and Yii so I ahve an opinion.
CI isn’t bad but I think there are better things out there like Yii. I don’t really like Cake because it’s slow as a dog and you spend more time finding workarounds than just coding stuff you need.
Why bother with benchmarks if you admit in your own article that they are pointless?
Really there is only one php framework benchmark article you ever need to read:
http://blog.astrumfutura.com/archives/421-PHP-Framework-Benchmarks-Entertaining-But-Ultimately-Useless.html
Anotyher vote for CI/kohana heere
“What I don’t like about Yii is that the entire frameworks is based around Active Record and ORM”
The use of Active Record is entirely optional and they also provide DAO. In fact, from their guide:
“Although Yii DAO and AR can handle nearly all database-related tasks, you can still use your own database libraries in your Yii application. As a matter of fact, Yii framework is carefully designed to be used together with other third-party libraries.”
In fact, when it came down to Framework choices recently it came down Kohana and Yii and I plumped for Yii due to:
* Better IDE support.
* Superior documentation.
* Better AR layer.
* Clever thinking, i.e. using a function to return the table name rather than protected methods, reducing memory footprint.
Yes, they are pretty much the same reasons I chose Yii over Kohana. As for ActiveRecord, yes, it is possible to use DAO with patterns like Table Module and Table Data Gateway and that’s pretty much what I’ve done. However the point I was trying to make is that the framework has very much been built around the ActiveRecord and ORM concepts, so if you don’t use them you end up loosing features and some of the nice things about the framework. I did some testing which I believe I commented on in another post and found that there isn’t an absolutely huge difference in performance between ActiveRecord and DAO as long as you’ve got caching configured so that the database schema doesn’t need to be queried every time (not even sure why they need to do that). But yes, Yii is definitely the better framework, unless you feel like writing your own super-lean one which does things only one way and the way you want them done. If you want a good example of implementing your own PHP MVC Framework I suggest looking at OpenCart, Daniel’s done a pretty good job of it.
tests are great .. do you consider to publish another test with opcode caching included ?
I could, but to be honest there really isn’t much point to the tests. When considering a PHP Framework you really need to look at a few other things like the size of the community and so on.
Basically having spent a long time on this I’d say the following… If you’re looking for a smaller personal or business project and don’t want to invest in a dedicated server but still want a good framework I’d recommend these in order of preference: Yii Framework, CodeIgniter, Kohana, CakePHP. If you want Enterprise Level, I’d recommend ZendFramework, Yii Framework (Close Contender for first) and Symphony.
Don’t look at performance as be all and end all, you really need to consider other factors… In the end performance is only one aspect and of course it’s fastest to write your own to do things just one way the way you want it done. Ultimately you’re quite unlikely to really hit the performance barriers with most frameworks on most projects, but you’re gonna struggle running something like Zend or Symphony in a shared hosting or even VPS environment with good performance.
kohana user here, found kohana after using CI, and kohana feels more natural than CI. I also love the template system. and now it’s been two years since the last time i code php, so i’am looking at today framework.
maybe i should try Yii now.
what’s currently best php IDE ? (speed is very important factor for me, so eclipse / netbeans is not a choice)
Personally, I would recommend Yii, but I think that Kohana and CI (2.0) which is available from BitBucket are not a bad choice as well. I’ve done a fair bit with Yii and I love it simply because whatever it is you’re thinking of doing which is a common web development task it has most likely already been solved in Yii (e.g. Clientside OK/Cancel). It’s pretty clear that the developers of Yii have had quite a lot of experience actually writing applications. I think that Yii is also a bit more mature and has a bigger community than Kohana 3 or CI 2.0 and that’s good reason to choose it as well. As for Kohana I’m just not a fan of misusing inheritance to enable code completion. Yii just feels like a true OO based designed that doesn’t implement workarounds. I do have to say though I’ve been working quite a bit with ASP.NET before and hence I’ve come to expect a certain level of stuff.
As for IDE, I would certainly go for Eclipse PDT or NetBeans as they’re the only two TRUE IDEs in my view when it comes to PHP development. Personally I’ve used both a fair bit and I think that Eclipse is the superior choice, especially when you use it with Aptana 2 and Subclipse. However, Netbeans is a much smaller install. However the problem with Netbeans for me is that it doesn’t have same thing as Aptana FTP and doesn’t have as good a support for SVN as Subclipse. In fact it was really annoying the way NetBeans shows SVN changes. In saying this I have not experienced any performance problems with either Eclipse or NetBeans, and I use both HEAVILY on a DAILY basis. So, I’m not sure where you got the idea that they have bad performance, unless you’re trying to develop on like a P3, but then again your iPhone has more grunt these days.
Have you ever tried Vim? (as an IDE of course)
haha… Vim is NOT what I (or anyone for that matter I would say) consider an IDE. An IDE has such things as intellisense, debugging and so on. There are only two true PHP IDEs that are heads above the rest and that seem to be more or less widely accepted, being Eclipse PDT and NetBeans or their derivatives (e.g. Zend Studio). There a few others but they seem to be falling behind a fair bit most of the big boys are putting their weight behind one of these two. This is kind of the same as Java IDEs, Eclipse and NetBeans are pretty much the defacto standard these days and I dare say very few people use anything else.
I’ve been over in Ruby/Rails land awhile, but I’m tired of configuring shared hosts and using dirty hacks just to get a site running. So, after evaluating some php frameworks, I eventually decided on Yii or Symfony2. The Symfony slowness issue seems to have been solved (http://symfony-reloaded.org/fast), and I’ve run a similar benchmark against Yii, and it appears to be on equal or slightly better terms in some cases. Plus, the whole ground-up redesign of v2 looks very promising if you’re into OOP.
But, I’m an IDE guy at heart (VS.NET dev during the day), so I think I will choose Yii since Symfony2 will also be in its infancy. CI seemed very heavy-handed, as did Zend. Kohana seemed lacking in the community, and DooPHP has an awful name (seriously, why would I want to ever tell a client I’m using the “DooPHP” framework?)
Yes, finding/configuring hosts is the problem with Ruby on Rails as nice as it may be. The problem with Symfony2 is that it’s not released as a production release and it generally takes some time for a production release to become stable, so it’s not something you can use in a production environment. So being fast if of little use if you can’t use it. Yii is also very similar to ASP.NET and ASP.NET MVC in it’s design (e.g. ADO.NET like DB interface). If you want to use an IDE than Yii is definitely a good choice when combined with Eclipse PDT or NetBeans. I don’t know what is meant by CI being heavyhanded, as CI is actually a very light framework, but instead of standard object concepts like you would be used to in .NET it uses the legacy concepts of working with PHP to get things working and IDE support is non-existent. Kohana has very much the same problem… instead of using pure OO to achieve what you need Kohana uses directory level hacks and under the hood the framework is quite messy. The problem with DooPHP is that it lacks far too many features you expect of a decent framework and lacks the community. Zend is quite good, but quite involved and slow as a result. If you’re a VS.NET guy, then perhaps you may actually consider ASP.NET MVC as you will still get far better tooling in VS.NET than you will with any framework and a PHP IDE.
For completeness sake it would have been nice to include CakePHP and Symfony as those are two of the most popular, and they were also the two I have considering given my preferences, but I’m thinking twice now. PHP having to load all those classes on every execution is so bogus compared to e.g. Rails.
Yes, CakePHP and Symfony are quite popular in comparison to some like Solar and DooPHP, but they have terrible performance. Symfony 2 promises better performance but its not yet in production so not worth considering at this stage. Perhaps Lithium sould also be considered, but it’s PHP 5.3+ only so can’t be run on most of the hosts as yet.
This was most informational. Thanks for sharing.
Well, I know it’s almost a year after the original post, but I gotta say this:
first, Kohana’s 3 documentation is a LOT better now – it seems a fairly new change on their website
second, I am using Eclipse PDT mostly, but also played with NetBeans and I have NO PROBLEMS with Kohana’s code completion in either program running on Ubuntu v10. I don’t know if it’s a recent fix or maybe you weren’t doing something right, because I just started using it, but it most definitely does work!