PHP MVC Framework Performance – Part 1
As part of my assessment of PHP MVC Frameworks which I blogged about in Choosing the best PHP MVC Framework I decided to do a compare performance of the various PHP MVC Frameworks I was considering. I did this using Apache Bench with ab -t 30 -c 10 http://testserver/framework/index.php as this is common and did the basic Hello World test where each framework just put out Hello World in an empty HTML document. Here are the results…
The test server was a Windows XP machine with T7200 2.00GHz CPU and 2GB RAM running WAMP with PHP 5.2.6, Apache 2.2.8. No opcache was used in this test altough that’s something I will probably use for another test.
The latest version of every framework was used and all features like DB access were disabled for the test to give a true comparison.
All tests were run 5 times and any results there weren’t around the average of other results were removed and the test run was repeated, to avoid skewing results.
It’s pretty pointless to provide figures in requests per second as every machine is different and everyone will hence get different results, but it makes a lot of sense to compare all frameworks against a common base. For that base I chose to use performance of raw PHP, which on my test machine did an average of 740 req/s. A test with a HTML test on the same machine did 850 req/s (so not a huge difference). I should also note that the exact file being requested was always used (e.g. index.php or index.html), as just referencing the directory further decreased performance because Apache had to work out which file to show for the directory index.
Here are the results:
- Raw PHP - 740 req/sec – 100%
- PHP Pro MVC 0.0.4 - 200 req/sec – 27%
- DooPHP 1.2 – 170 req/sec – 23%
- Yii Framework 1.1.1 – 130 req/sec – 18%
- Kohana PHP 2.3.4 – 55 req/sec – 7.5%
- CodeIgniter 1.7.2 – 38 req/sec – 5%
- Zend Framework 1.10 – 24 req/sec – 3%
So what does this all mean, let’s analyse the results…
Firstly PHP Pro MVC is a minimalistic MVC framwork written as an example of how to write an MVC framework and is used by OpenCart (with some modifications). I used it in the comparison to get an understanding how how any sort of even a basic PHP framework would impact performance.
As claimed on their website DooPHP is indeed the fastest of the frameworks, but it’s nowhere near as fast as they claim to be as compared to other frameworks. I also wouldn’t recommend using it because it’s not very logically structured compared to some others. Speed isn’t everything.
Yii is also indeed quite fast and to my surprise was significantly faster than Kohana or Code Igniter! However I tested v1.1.1 and Yii Framework Performance Bencharks were done on version 1.0.2. As I’ve said before I quite like Yii Framework because it just feels natural and logical.
Kohana was a bit faster than Code Igniter but in the same ballpark. As I’ve also pointed out I think that Yii is better than Kohana because code completion in PHP IDEs doesn’t work with Kohana and that’s a major issue.
Code Igniter was a bit slower than I would have hoped it would be, but that’s understandable because of all the interoperability code that doesn’t really need to be there if you’re using PHP5 only. This is consistent with Kohana vs Code Igniter Perofmance results obtained by other people.
Zend Framework is the slowest of the lot, which is understandable given how heavy the framework is and how many files get loaded just to render a template from a controller. This is also something that’s been pointed out in tests by other people.
I didn’t bother testing Cake PHP or Symfony as it’s well know that they are even slower than Zend Framework.
Now, here’s something VERY IMPORTANT!
You can compare performance of frameworks on Hello World tests all you want, but in a real life situation the performance of your framework will be dwarfed into insignificance by the following 2 factors:
- Database Access
- File System Access
Database Access and File System Access are the two slowest operations there are so how quick the actual code runs matters very little. In fact, the reason why Zend is so much slower than others is because it splits the code between so many files and it’s not even the parsing that’s the biggest problem, but file access, as I was running without opcode caching. With opcode caching you can expect the performance to improve to a point where the performance difference between frameworks will be insignificant in comparison to database access operations.
So, based on all of this my recommendation are as follows:
- Choose the framework that best suits your needs and don’t worry so much about raw performance (unless the framework is split between many many files and you’re not using opcode caching, in which case file system access becomes a limiting factor). You should consider other factors when about the PHP Framework as I’ve outlined in Choosing the best PHP MVC Framework.
- Try to use opcode caching wherever possible to avoid file system access becoming a limiting factor.
- Make sure you optimise your database performance (read queries and indexes) and try to minimise the number of database accesses (especially in loops) by using joins and retrieving multiple rows with a single query. This one thing will dwarf any difference you may see in PHP MVC Framework performance!
I prefer to use Code Igniter from all the choices available… as its the easiest and well documented…
Yes, CI is quite nice as it’s simple to use, has a large community and is well documented. The downside I found with CI is that it is a little out of date and for some reason slower than I thought! After looking at CI and Kohana and paying around with them I found that BOTH have issues with code completion in an IDE, so I do still think that Yii is better as a framework as it feels natural, performs better than CI and can be used easier in an IDE, which boosts productivity enormously. This is difficult to explain to someone who’s been using a hyped up Notepad or Dreamweaver for programming, but anyone who’s used to an IDE like Visual Studio, Eclipse or NetBeans will know exactly what I mean. It is also written using the latest PHP design concepts and is purely OO (not that I think that this is particularly important). What I don’t like about Yii is that the entire frameworks is based around Active Record and ORM, which is both bad for performance and un-necessary in my view. There is nothing wrong with just doing SQL queries, where you have control over exact SQL that gets run and don’t need to learn another language as such to write SQL.
I thing CI does not have some things that cakephp or Yii for example have.
For example the command line tool that create controllers etc and a crud application that is useful,there is better active reconrd implementetion and so on.
Ci is maybe simple but finally other frameworks make your life easier even maybe more difficult for someone.
I used CI for 6 months before I turn to cakephp and Yii so I ahve an opinion.
CI isn’t bad but I think there are better things out there like Yii. I don’t really like Cake because it’s slow as a dog and you spend more time finding workarounds than just coding stuff you need.
Why bother with benchmarks if you admit in your own article that they are pointless?
Really there is only one php framework benchmark article you ever need to read:
http://blog.astrumfutura.com/archives/421-PHP-Framework-Benchmarks-Entertaining-But-Ultimately-Useless.html
Anotyher vote for CI/kohana heere
“What I don’t like about Yii is that the entire frameworks is based around Active Record and ORM”
The use of Active Record is entirely optional and they also provide DAO. In fact, from their guide:
“Although Yii DAO and AR can handle nearly all database-related tasks, you can still use your own database libraries in your Yii application. As a matter of fact, Yii framework is carefully designed to be used together with other third-party libraries.”
In fact, when it came down to Framework choices recently it came down Kohana and Yii and I plumped for Yii due to:
* Better IDE support.
* Superior documentation.
* Better AR layer.
* Clever thinking, i.e. using a function to return the table name rather than protected methods, reducing memory footprint.
Yes, they are pretty much the same reasons I chose Yii over Kohana. As for ActiveRecord, yes, it is possible to use DAO with patterns like Table Module and Table Data Gateway and that’s pretty much what I’ve done. However the point I was trying to make is that the framework has very much been built around the ActiveRecord and ORM concepts, so if you don’t use them you end up loosing features and some of the nice things about the framework. I did some testing which I believe I commented on in another post and found that there isn’t an absolutely huge difference in performance between ActiveRecord and DAO as long as you’ve got caching configured so that the database schema doesn’t need to be queried every time (not even sure why they need to do that). But yes, Yii is definitely the better framework, unless you feel like writing your own super-lean one which does things only one way and the way you want them done. If you want a good example of implementing your own PHP MVC Framework I suggest looking at OpenCart, Daniel’s done a pretty good job of it.
tests are great .. do you consider to publish another test with opcode caching included ?
I could, but to be honest there really isn’t much point to the tests. When considering a PHP Framework you really need to look at a few other things like the size of the community and so on.
Basically having spent a long time on this I’d say the following… If you’re looking for a smaller personal or business project and don’t want to invest in a dedicated server but still want a good framework I’d recommend these in order of preference: Yii Framework, CodeIgniter, Kohana, CakePHP. If you want Enterprise Level, I’d recommend ZendFramework, Yii Framework (Close Contender for first) and Symphony.
Don’t look at performance as be all and end all, you really need to consider other factors… In the end performance is only one aspect and of course it’s fastest to write your own to do things just one way the way you want it done. Ultimately you’re quite unlikely to really hit the performance barriers with most frameworks on most projects, but you’re gonna struggle running something like Zend or Symphony in a shared hosting or even VPS environment with good performance.