PHP 7.4 OPCache Preloading benchmark results + note on database extension crashes

PHP 7.4 version brings in a slew of new features to the language, including a notable feature for performance: OPCache Preloading. It promises significant gains for complex PHP apps where the bootstrap is sizeable. I wanted to find out how realistic these claims were and if the feature is ready for prime time in real world deployments.

I decided to do a round of benchmarks with eZ Platform, a complex Symfony application on Symfony 4.4, which supports OPCache Preloading. If you're not familiar with eZ Platform, it is our Digital Experience Platform (DXP) for creating websites and web applications. Our foundation is the Symfony framework and we just upgraded eZ Platform to Symfony 4.

Unfortunately at the time of writing we did not have a release that is fully certified to work with Symfony 4.4 and PHP 7.4, so I ended up stitching together a version just for these benchmarks: github.com/janit/ezplatform-php74. This version represents a development version of our upcoming eZ Platform 3.0.0 release scheduled to be available in 2020.

The codebase is not optimized and not all features are guaranteed to work, but all the major features are in place. I felt it would be able to give some insight into relative performance improvements between PHP 7.3, 7.4 and 7.4 with OPCache Preloading.

The benchmark target is a simple landing page which loads and renders some content from our content repository (data model), whose SQL calls is effiently cached. TTL was set to zero so I would not be get numbers for caching, but pure processing throughput. Benchmarks were ran three times and the average was used in the report below.

I used dedicated VPSes for the target application and the load generator. The target server had 2 vCPU and 4 GB of RAM assigned. Both were Ubuntu 18.04 LTS with PHP packages (7.3 and 7.4) from Sury repositories with nginx 1.14.0 for the web server with PHP-FPM. Default settings for PHP was used, except for opcache.validate_timestamps which was set to off.

Running into trouble with OPCache Preloading

Since Symfony 4.4 and greater versions support OPCache Preloading out of the box, enabling in theory requires defining two configuration values in your php.ini file:

opcache.preload_user=www-data
opcache.preload=/var/www/ezplatform/var/cache/prod/srcApp_KernelProdContainer.preload.php

Initially I ran into some trouble of not having the generated preload file in place since I was clearing caches using the dev environment, instead of prod. Once this was in place I ran into another issue, with the system reporting a segfault in the FPM log (/var/log/php7.4-fpm.log):

[05-Dec-2019 12:57:53] WARNING: [pool www] child 2194 exited on signal 11 (SIGSEGV - core dumped) after 0.952515 seconds from start

The preloading caused the app to crash immediately, so I decided to dig underneath the hood a little bit and poke around with the generated preload file. I found that by commenting the includes for the Symfony container the constant segfaults were gone:

// require __DIR__.'/ContainerNacE9A2/srcApp_KernelProdContainer.php';

After removing this line PHP-FPM would not crash immediately, but start throwing seemingly random errors regarding missing methods of classes in dependencies:

2019/12/05 13:56:42 [error] 6644#6644: *131 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Uncaught Error: Call to undefined method Hoa\Consistency\Consistency::flexEntity() in /var/www/ezplatform/vendor/hoa/consistency/Consistency.php:361

So obviously the kernel was included for a reason and the system was not finding resources is was expecting. I noticed the compiled Symfony kernel was quite large at 1.3 MB, so I so I tried upping memory limits for OPCache and PHP itself. Generous resources did not help.

After reading through a few bug reports like Preloading segfaults at preload time and at runtime, I decided not to go further down the rabbit hole at this point in time. I already had benchmark results for PHP 7.3 vs. PHP 7.4 and that was already worth reporting.

Benchmark results and conclusion

Here is the chart comparing the throughput of the demo installation with PHP 7.3 and 7.4:

Again, it's not worth mulling over the exact throughput of the demo application, but focus on relative numbers. We see some consistent improvements when enabling OPCache Preloading on 7.4, but not alone warranting the 30-50% increase. However the cumulative improvements over PHP 7.3 are realistically well within the reach of the made claims.

Also, please note that the benchmark is not what you would call a "real world application". It's just a hello world, where the preloaded bootstrap is minimal (for a Symfony app). I expect OPCache Preloading to yield bigger gains the larger your application grows.

PHP extensions can break OPCache Preloading

Finally I wanted to try something a bit more realistic, and tried to benchmark an older proof-of-concept app I built way back in 2017 to demonstrate sharing application state between PHP and a JavaScript application. Since that I upgraded the application from Symfony 3 to 4. For this benchmark I upgraded it further to Symfony 4.4 and PHP 7.4.

Unfortunately I ran into some trouble again and could not get any benchmarks executed with OPCache Preloading enabled. This time the trouble was caused by the php7.4-sqlite3 package that the app required because of it's use of an SQLite database. I tweaked some settings like the opcache.interned_strings_buffer, that had caused similar errors before.

I could not get it to work and eventually had to give up. In my use case, when enabling the SQLite extension and restarting PHP-FPM, the system persistently logged fatal errors like:

[08-Dec-2019 11:18:54] WARNING: [pool www] child 1097 exited on signal 6 (SIGABRT - core dumped)

and

2019/12/08 11:16:41 [error] 3538#3538: *153 recv() failed (104: Connection reset by peer) while reading response header from upstream,

In my experience adopting OPCache Preloading in PHP 7.4 is not straightforward today. In addition to userland dependencies causing havoc, it seems that extensions like SQLite can also cause unexpected failures. I'm sure this'll change over time as issues get ironed out.

So if your app has trouble working with OPCache Preloading, I suggest you start dropping out dependencies from your composer.json as well as disabling PHP extensions one by one. Once you've found the cause of the it'll be easier to fix (if you've got the skills) or at least to create a minimal test case and report the bug to PHP, extension or PHP library project.

Update 1: I got a response on this post from Nikita Popov from the PHP dev team. His initial reaction was that SQLite or PHP extensions in general are unlikely the cause here. Instead it is probably because of require instead of opcache_compile_file(). A fix is being worked on in Symfony, but it the require behavour could be fixed in PHP 7.4.1.

Update 2: I tried again after PHP 7.4.1 was released problem persisted and I discovered that using MySQL extension also crashes the system. But again I got a response from Nikita, stating that this is expected. Some fixes did not make it to that version, and more OPCache Preloading fixes are coming in PHP 7.4.2 and the reason they for the crashes are not database extensions, but the incompatibilities with Doctrine - a database library (and ORM):

Update 3: Segfaults remain persistent when running PHP 7.4.2, the next promise of progress is with PHP 7.4.3, since there is a known issue with the opcache.preload_user setting:

@violuke There is a known issue with preloading and opcache.preload_user. If you use that ini setting, you'll have to wait for 7.4.3.
- [DependencyInjection] Use opcache_compile_file for preloader #34910