Benchmark 3 - The Alternative PHP Cache (APC)
Published on 2010-12-16.
This is a benchmark test of the performance of PHP with and without APC (Alternative PHP Cache). The test was performed on an Intel P4 1.7 Ghz with 1036028 kB of memory over a private LAN. The server runs PHP 5.3.3 on a standard Debian installation. Between each change of settings all related processes was restarted.
Update 2019: Please note that the PHP Cache - APC is now considered unmaintained and dead.
The APC application is an Open Source cache for PHP written by Daniel Cowgill and George Schlossnagle. APC optimizes PHP intermediate code and caches data and compiled code from the PHP bytecode compiler in shared memory. APC is quickly becoming the de-facto standard PHP caching mechanism as it will be included (built-in) to the core of PHP starting with PHP 6.
On Debian Lenny and above APC can be installed using the command:
# apt-get install php-apc
APC will then be enabled by default.
Because APC is a compiler cache it is only really effective (which the benchmark will prove) when it is used on scripts that have a large code base.
Test 1
The first benchmark shows no increase in performance because it only deals with a simple class and method.
The small script I used to benchmark looks like this:
class c {
function hw()
{
echo "Hello World!\n";
}
function do_it()
{
$i = 0;
while ($i < 1000000) {
$i++; $this->hw();
}
}
}
$inst = new c();
$inst->do_it();
I am using the Apache HTTP server benchmarking tool for the benchmarking with the following command: $ ab -c5 -n10 This will create 5 concurrent connections that each will perform 10 requests.
Test 1 Without APC
Server Software: Apache/2.2.9
Server Hostname: webserver
Server Port: 83
Document Path: /hello-oo.php
Document Length: 13000000 bytes
Concurrency Level: 5
Time taken for tests: 15.081 seconds
Complete requests: 10
Failed requests: 0
Write errors: 0
Total transferred: 130002330 bytes
HTML transferred: 130000000 bytes
Requests per second: 0.66 [#/sec] (mean)
Time per request: 7540.598 [ms] (mean)
Time per request: 1508.120 [ms] (mean, across all concurrent requests)
Transfer rate: 8418.13 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 0
Processing: 6335 7410 651.2 7615 8746
Waiting: 3 14 22.2 3 74
Total: 6335 7410 651.2 7615 8746
Percentage of the requests served within a certain time (ms)
50% 7615
66% 7674
75% 7729
80% 7791
90% 8746
95% 8746
98% 8746
99% 8746
100% 8746 (longest request)`
APC is then enabled.
Note: apc.enabled can be set to 0 to disable APC. This is primarily useful when APC is statically compiled into PHP since there is no other way to disable it (when compiled as a DSO, the extension line in php.ini can just be commented-out).
Test 1 With APC
Server Software: Apache/2.2.9
Server Hostname: webserver
Server Port: 83
Document Path: /hello-oo.php
Document Length: 13000000 bytes
Concurrency Level: 5
Time taken for tests: 14.988 seconds
Complete requests: 10
Failed requests: 0
Write errors: 0
Total transferred: 130002330 bytes
HTML transferred: 130000000 bytes
Requests per second: 0.67 [#/sec] (mean)
Time per request: 7493.833 [ms] (mean)
Time per request: 1498.767 [ms] (mean, across all concurrent requests)
Transfer rate: 8470.66 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.3 0 1
Processing: 7317 7475 107.9 7476 7669
Waiting: 5 20 26.2 7 86
Total: 7318 7475 107.7 7476 7669
Percentage of the requests served within a certain time (ms)
50% 7476
66% 7489
75% 7550
80% 7600
90% 7669
95% 7669
98% 7669
99% 7669
100% 7669 (longest request)`
From the above there is no difference at all and I have checked that APC is enabled and caching files.
Test 2
In this test I decided to try using the PHP Benchmark Script with Firefox running on the client.
Test 1 Without APC - Avg Result
Strings Encryption Dates Images Arrays Filesystem Objects Overall 220 ms 145 ms 36 ms 361 ms 52 ms 200 ms 80 ms 1094 ms
Test 2 Without APC - Avg Result
Strings Encryption Dates Images Arrays Filesystem Objects Overall 219 ms 145 ms 36 ms 359 ms 53 ms 199 ms 82 ms 1092 ms
APC is then enabled.
Test 1 With APC - Avg Result
Strings Encryption Dates Images Arrays Filesystem Objects Overall 218 ms 144 ms 36 ms 367 ms 53 ms 198 ms 78 ms 1095 ms
Test 2 With APC - Avg Result
Strings Encryption Dates Images Arrays Filesystem Objects Overall 221 ms 144 ms 36 ms 359 ms 53 ms 197 ms 79 ms 1089 ms
Again, as exspected, there is no noticable performance boost.
Test 3
As mentioned previously, APC has to work on a lot of code in order to reveal any performance improvements, so I am now going to throw some "heavy" stuff on the webserver to see if that will reveal some better results. I have installed Wordpress, Drupal, and Phorum and populated them with some data.
I run each test with: ab -c5 -n50 which creates 5 concurrent connections with 50 requests on each.
Wordpress Without APC.
Server Software: Apache/2.2.9
Server Hostname: webserver
Server Port: 90
Document Path: /
Document Length: 9548 bytes
Concurrency Level: 5
Time taken for tests: 28.276 seconds
Complete requests: 50
Failed requests: 0
Write errors: 0
Total transferred: 501000 bytes
HTML transferred: 477400 bytes
Requests per second: 1.77 [#/sec] (mean)
Time per request: 2827.558 [ms] (mean)
Time per request: 565.512 [ms] (mean, across all concurrent requests)
Transfer rate: 17.30 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 2256 2816 100.4 2823 3072
Waiting: 2203 2517 77.9 2514 2800
Total: 2257 2816 100.4 2823 3072
Percentage of the requests served within a certain time (ms)
50% 2823
66% 2832
75% 2844
80% 2850
90% 2880
95% 2922
98% 3072
99% 3072
100% 3072 (longest request)`
Wordpress With APC.
Server Software: Apache/2.2.9
Server Hostname: webserver
Server Port: 90
Document Path: /
Document Length: 9548 bytes
Concurrency Level: 5
Time taken for tests: 12.734 seconds
Complete requests: 50
Failed requests: 0
Write errors: 0
Total transferred: 449878 bytes
HTML transferred: 426278 bytes
Requests per second: 3.93 [#/sec] (mean)
Time per request: 1273.351 [ms] (mean)
Time per request: 254.670 [ms] (mean, across all concurrent requests)
Transfer rate: 34.50 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 783 1268 651.8 999 3147
Waiting: 758 1155 587.0 921 2882
Total: 783 1268 651.8 999 3148
Percentage of the requests served within a certain time (ms)
50% 999
66% 1037
75% 1202
80% 1482
90% 3064
95% 3135
98% 3148
99% 3148
100% 3148 (longest request)
`
Wordpress is no longer the simple and performance friendly CMS it used to be and this time there is quite a performance boost. Without APC Wordpress could deliver 1.77 requests per second whereas it could deliver 3.93 requests per second when APC was running. That's about twice as many requests. The entire process was performed at less than half the time. With APC off it took 28.276 seconds to complete the 50 requests. With APC on it took only 12.734 seconds to complete the same amount of requests.
Now, what does this tell us besides the fact that Wordpress has become bloated, slow, and heavy to dance with?
APC can have especially beneficial effects in systems with a large number of includes. If you are working with scripts that has many recursive includes (say 30 or more) APC can provide over 100% in application performance, but if you are working on simple scripts with few includes you are not going to see any performance boosts using APC.
Now, I am not going into details about the specific setup of Wordpress or what version is actually running on the server because the test is not about optimizing Wordpress. The test is about how much improvement of performance we can achieve on a given installation of the same product with APC enabled versus not having APC enabled.
Let's run the same test on a Drupal installation.
Again I run each test with: ab -c5 -n50 which creates 5 concurrent connections with 50 requests on each.
Drupal Without APC.
Server Software: Apache/2.2.9
Server Hostname: webserver
Server Port: 91
Document Path: /
Document Length: 4957 bytes
Concurrency Level: 5
Time taken for tests: 18.988 seconds
Complete requests: 50
Failed requests: 0
Write errors: 0
Total transferred: 275650 bytes
HTML transferred: 247850 bytes
Requests per second: 2.63 [#/sec] (mean)
Time per request: 1898.817 [ms] (mean)
Time per request: 379.763 [ms] (mean, across all concurrent requests)
Transfer rate: 14.18 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 1588 1893 146.3 1843 2202
Waiting: 1588 1892 145.5 1843 2201
Total: 1588 1893 146.3 1844 2202
Percentage of the requests served within a certain time (ms)
50% 1844
66% 1854
75% 1870
80% 2098
90% 2157
95% 2160
98% 2202
99% 2202
100% 2202 (longest request)`
Drupal With APC
Server Software: Apache/2.2.9
Server Hostname: webserver
Server Port: 91
Document Path: /
Document Length: 4957 bytes
Concurrency Level: 5
Time taken for tests: 9.215 seconds
Complete requests: 50
Failed requests: 0
Write errors: 0
Total transferred: 275650 bytes
HTML transferred: 247850 bytes
Requests per second: 5.43 [#/sec] (mean)
Time per request: 921.476 [ms] (mean)
Time per request: 184.295 [ms] (mean, across all concurrent requests)
Transfer rate: 29.21 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 542 919 428.4 754 2266
Waiting: 541 918 424.8 754 2227
Total: 542 920 428.5 754 2266
Percentage of the requests served within a certain time (ms)
50% 754
66% 759
75% 830
80% 974
90% 2065
95% 2127
98% 2266
99% 2266
100% 2266 (longest request)`
Again APC provides a great performance boost. With APC off Drupal could deliver 50 requests in 18.988 seconds with an average of 2.63 requests per second. With APC on Drupal could deliver 50 requests in only 9.215 seconds with an average of 5.43 requests per second.
Now, it's time to run the same test on Phorum. Again I'm running with ab -c5 -n50 which creates 5 concurrent connections with 50 requests on each.
Phorum Without APC.
Server Software: Apache/2.2.9
Server Hostname: webserver
Server Port: 98
Document Path: /forum/index.php
Document Length: 26322 bytes
Concurrency Level: 5
Time taken for tests: 9.712 seconds
Complete requests: 50
Failed requests: 0
Write errors: 0
Total transferred: 1336350 bytes
HTML transferred: 1316100 bytes
Requests per second: 5.15 [#/sec] (mean)
Time per request: 971.164 [ms] (mean)
Time per request: 194.233 [ms] (mean, across all concurrent requests)
Transfer rate: 134.38 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 1
Processing: 728 966 90.0 943 1148
Waiting: 691 836 82.3 808 997
Total: 728 966 90.0 943 1148
Percentage of the requests served within a certain time (ms)
50% 943
66% 966
75% 1012
80% 1087
90% 1111
95% 1117
98% 1148
99% 1148
100% 1148 (longest request)`
Phorum With APC.
Server Software: Apache/2.2.9
Server Hostname: webserver
Server Port: 98
Document Path: /forum/index.php
Document Length: 26322 bytes
Concurrency Level: 5
Time taken for tests: 5.090 seconds
Complete requests: 50
Failed requests: 0
Write errors: 0
Total transferred: 1336350 bytes
HTML transferred: 1316100 bytes
Requests per second: 9.82 [#/sec] (mean)
Time per request: 508.971 [ms] (mean)
Time per request: 101.794 [ms] (mean, across all concurrent requests)
Transfer rate: 256.41 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 190 504 211.7 425 1095
Waiting: 181 414 200.7 330 977
Total: 191 504 211.7 425 1095
Percentage of the requests served within a certain time (ms)
50% 425
66% 458
75% 517
80% 595
90% 1057
95% 1083
98% 1095
99% 1095
100% 1095 (longest request)`
With APC off Phorum delivered the 50 requests at 9.712 seconds with an average of 5.15 requests per second. With APC on Phorum delivered the 50 requests at 5.090 seconds with an average of 9.82 requests per second.
Conclusion
In all three cases APC provides a significant performance boost of about twice the performance. The only situation where APC has little or no noticable effect is when the application is rather small and this makes sense because APC is a compiler cache.