Benchmarking Laravel with Swoole, FrankenPHP, RoadRunner, php-fpm, and ngx-php

[@kdunglas]

Raw PHP performance is only part of the story. In real-life apps, enabling HTTP/3, compressing responses with modern formats such as Zstandard or Brotli, and using Early Hints will usually have a bigger impact on performance. FrankenPHP supports all of this by default!

[@pronskiy]

Happy to see you here, Kevin! And kudos for FrankenPHP and all of your work!

[@MaulikParmar210]

Forgot to mention that they all run as single application server binary ( thanks custom caddy build) unlike other solutions loading modules on runtime and require proper runtime setup. This is much more compact and can be tuned to individual needs for easy app deployment with single binary in image / deployment and few data mount points. Roadrunner is slight faster in raw performance because of goridge comapred to cgo sapi so it bypasses alot of runtime sapi code, this is expected to improve over time when SAPI itself will improve and php will depart from fpm era and become more general purpose language. Also in franken one can enjoy entire caddy ecosystem as an added bonus. The raw performance doesn't necessarily mean better throughput, most real world apps will be blocked by IO rather than available CPU where DB is the slowest to respond even if responses come under sub 5 milisecond range. The app will indefinitely wait in such cases. Would love to see sub 1ms range timings but it will take alot of efforts to systematically clean core runtime and add missing features like OS backed IPC, native threading, sync mechanisms to advance the development. Always good to see you in the community, thank you for giving us real frankenstien of php world, the name suits the solution 👍🏻

[@techrevolution1408]

@@MaulikParmar210 I really love Frankenphp. Selfcontained executable is game changer for me.

[@MrWancer]

Thanks for the video It's really interesting what modern ways of running php app exists nowdays and how fast & different they are If you would have some extra time, it would be interesting to see if build-in optimizations would affect performance somehow. - php-fpm pm.* - jit - opcache - preload - disabling circular reference collector (gc) - that's interesting one. It should be fine to have it disabled for php-fpm, but for other ways of running - not sure.

[@pronskiy]

Oooo thanks for this list! Now I have the next challenge: squeeze the best out of php-fpm with all the optimizations.

[@romanpronskiy]

I tried to add some optimisations in the repository. OPcache helps a lot indeed, but it seems like more can be done to improve it.

[@abdellahb1398]

is ngx-phpworker similar to workerman/adapterman ?

[@maisei88]

Рома спасибо большое за видео 🤗

[@pronskiy]

🫶💜

[@RahmaTVIndo]

Nice video! love it

[@midgardresident]

is opcache enabled when running php-fpm?

[@theshinyplayer2373]

I'm sorry, but there are som extreme issues with your testing. I did a quick test on a 5 year old machine, and I'm getting 12K rps on the default Laravel landing page with Swoole. That's around 50x your results. Here are some of the issues: 1. You're running on macos. That's not at all what people actually run their sites on. You might even be running on a different arch if your PC is ARM. 2. You're using docker. Docker does not have much overhead, normally. However, since you're on macos you're running docker in a VM, and the biggest issue is that you're using bind mounts. This isn't a issue on Linux, but on Windows and macos it's insanely slow. Move all your files into the container as part of the image, that helps. 3. You're probably just benchmarking sqlite as Laravel uses it for session storage by default. If you look at CPU usage during benchmarking you'll most likely see that your computer is mostly idle. Use something more realistic, like MariaDB or Redis, to reduce the storage bottleneck. CPU usage should be 100 % if you're benchmarking PHP execution. 4. You might not be running artisan optimize or optimizing the composer autoloader. Maybe you're also running Laravel in debug mode? All these things are things that are normal to do for an actual website, so the number you're getting are very unrealistic. Getting realistic numbers needs to be done on a realistic setup. Benchmarking and profiling software on a different OS(or arch) to the one that will be used in prod can give very wrong.

[@pronskiy]

@@theshinyplayer2373 thanks for bringing this up. Indeed, I believe all of your concerns are true. The code of benchmarks is available at GH/pronskiy/ngx-php_laravel so I’d be happy if you help improve it. I plan to rerun it to get more accurate numbers. I wouldn’t like to remove Docker though, cause it makes things much easier. Also techempower also run their benchmarks with Docker so, as you said, maybe running on Linux would help.

[@lemeshenko]

Why not to run it on vps. This will be closest to the real life scenario

[@duongphuhiep]

agree that the numbers are unrealistic because of the infrastructure (macOs, docker...) However the point is to compare them under the same infrastructure condition to see the relative differences. This video gives us a general idea about the gaps between them

[@theshinyplayer2373]

​@@duongphuhiep That's the problem, it doesn't work at all for a comparison that is relevant for a production env. When disk access is this slow it quickly becomes the bottleneck, and if some software uses the disk less it might get way ahead in these benchmarks while still being slower in a production env where the disk access is orders of magnitude faster and something else becomes the bottleneck.

[@husnixs]

@@pronskiy maybe you can test with CMS like wordpress. let's see who is the fast

[@shkabo]

I'd like to see this vid with symfony now :)

[@user-uk9er5vw4c]

this was very nice

[@CutCodeRu]

Excellent video, php-fpm top) we are waiting for more content from you

[@pronskiy]

thank you, folks 🫶

[@medilies]

Where does Apache fit in the benchmark?

[@pronskiy]

I haven't benchmarked it yet. My gut feeling is that it will be a bit slower than php-fpm. If you have some time, feel free to send a PR with a dockerfile to GH/pronskiy/ngx-php_laravel

[@ROX2]

What about ecosystem ? swoole has upd,tcp,websockets integrated and on my opinion its a killer feature combined with prety nice performance with easy setup and run. Swoole is my choice. But your benchmark interesting too. Feed for thought.

[@pronskiy]

Agree, I had a bit of experience with Swoole components and Hyperf and it was a pleasure to work with. What do you think is preventing more people from trying/using it?

[@ROX2]

​@@pronskiy i think first of all it all about php curse - old and absolutely wrong tutorials and learning materials that propaganda bad practices, bad code style, and of course - ignoring security issues. And i tell about overengineering examples too :) We also does not have enough public example of successful projects that use something else instead of php-fpm (but i know some). I personaly have expirience of development PHP Swoole UDP socket based backend server for multiplayer game used Unity game engine and it work great with a huge performance capacity! FPM desined to die in end, like php self. Using roadruner, Swoole etc, force you to worry about memory leaking and made something to restart process manualy or automatic if something goes wrong. This add some complexicity. BTW i see some good libraries in your monthly digest wich use CLI and i think this side of PHP underrated too like and other php application servers. PHP-FPM also has good feature - instant apply changes on save (if opcache disabled of course :D). Another approaches require restarting processes or using some third-party automation solutions. To summarize, I agree that FPM is easier for smaller projects that don't need to break stars from the sky, and solutions like Swoole are great for comparing with GO in many cases

[@kruhlyk_ua]

As for me FrankenPHP is cool but it's not ready to use in production/ Swoole looks like more stable solution.

[@pronskiy]

@@kruhlyk_ua agree. FrankenPHP is promising but not as mature as Swoole

[@kdunglas]

If you encounter issues with FrankenPHP, please open an issue. We fixed most of the known issues but a few (usually caused by buggy third-party extensions) remain.

[@pronskiy]

@@kdunglas thanks, Kevin. Is there a list of the buggy extensions somewhere?

[@budipriyatno5853]

I'd prefer frankenphp worker mode

[@lemeshenko]

You have memory leaks because of Laravel. It is not properly implemented for long running tasks. You need to use octane to properly clear the state.

[@pronskiy]

@@lemeshenko I do use Octane. You can check the code on GitHub pronskiy/ngx-php_laravel

[@seqgame7410]

`Hello world` test is a test that will never be applicable to a real world app. Swoole has coroutines, async db queries, workers for tasks. What result will ngx-php show when querying the database, while swoole continues to handle other connections while the database query is running? (Unfortunately, this is not implemented in Laravel Octane)

[@pronskiy]

@@seqgame7410 Those are valid concerns indeed. By default, ngx-php worker would wait, but in theory it is possible to do asynchronous db queries. It requires changing the architecture tho just like with swoole. So here I was thinking about “can we through something on it real quick and get performance boost?”

[@francomputos]

toooooooooooooooooooooooooooooo FAST

[@tarilonte]

If someone chooses Laravel it is because they have given up on performance.

[@Denakino]

Sure, Laravel is not the fastest, but it is more than fast enough for about 99% of cases.

[@pronskiy]

There’s some truth to that - I’d agree that Laravel's strength is in development speed rather than extreme performance. But that doesn’t mean you can’t challenge yourself to make it as fast as possible.

[@helioao]

“I don’t understand something that’s why I hate it”

[@lemeshenko]

Pure laravel executes in 40-50 ms, which is comparable with 20 ms of, for example, Net core

[@raiyansarker]

the best you can get is 300 rps? PHP is great, just not the performance!

[@winfle]

No, on my machine I have around 12k RPS on rr with 12 workers

[@raiyansarker]

@@winfle I believe you won't deploy on a server somewhere near that specs!