Last week I took worked on a benchmarking project I was considering for some time. It is about a deep dive into the two most popular PHP routing libraries, Symfony Routing and FastRoute. The twist is that I wanted to use a real world example and not the odd benchmark scripts I've seen so far.

Recent history of PHP routing

I've been researching routing in PHP for a while now. It's been doing this on and off since late 2015, and for that time a lot of things changed and evolved.

Back then, Nikita Popov's FastRoute emerged as one of the popular routing libraries. The excellent results he was able to achieve were based on his research on using regular expressions in an innovative way to match several routes at the same time. You should read his blog post "Fast request routing using regular expressions" about that, as he manages to explain very well where the performance benefits come from. It ends with a slight jab towards Symfony:

If you tried to put the Symfony router behind such a server, it would totally cripple your performance.

Nikota Popov February 18, 2014

Several years later, in early 2017, Frank de Jonge did some great research on Symfony's Routing Component. He submitted changes that improve static routes matching by using common prefixes, as well as few other performance optimizations. You can read more about the whole journey of the research, the profiling, and the proposed solutions in his blog post, "Battle Log: Symfony Routing performance considerations.".

Almost one year after that, in yearly 2018, Nicolas Grekas submitted improvements to Symfony Routing that further improved its performance and was able to surpass FastRoute. He did some great research, and used the lessons learned so far, such as the combined prefixes and the grouping of static routes. Everything in this process is described very well in a couple of blog posts:

• Making Symfony’s Router 77.7x faster - 1/2 February 16, 2018
• Making Symfony router lightning fast - 2/2 February 21, 2018

Remember that slight jab Nikita made? Nicolas shares that it was part of the motivation for the improvements:

Just for reference, this is not true anymore since Symfony 4.1, which is now faster than FastRoute. Thanks for the inspiration!

Nicolas Grekas March 13, 2018 5:32 PM

Currently (as of May 31, 2021) so far these are still the most popular routing PHP libraries:

• Symfony Routing component is used not only by the Symfony Framework but also by Laravel, as well as a lot more other projects

• FastRoute continues to be as popular as before, where it is used on its own, or by other popular projects such as the Slim framework and League\Route.

Real World Benchmark

So far so good. I've read everything I could find on this topic. That includes blog posts and articles, discussions on the pull requests for these libraries, and the occasional tweets. The one thing I had issues with were the benchmarks. When I looked at the code for the benchmarks used, they were very narrow and skewed towards some specific scenario that needs to be tested. I could not find anything that has the size and complexity of a real-world big PHP project.

I've read in the pull requests comments about people testing the changes submitted on their own projects. Big, real projects, which unfortunately are not open-source. Surely there is a set of routes out there that is both big enough to resemble a real world PHP application, and is also open-source and available for me to use.

I decided that I will created such benchmark, and the first thing to solve was:

What to use for the benchmark?

First, I started by searching and scouting through GitHub to find routes definitions from Laravel (routes.php, web.php, api.php) or from Symfony (routes.yml mostly). That got me nowhere as I was only able to find really small projects and nothing that I could use.

Second, I decided to reach out to and ask people if they know of such open-source projects. My focus was on Laravel as that seems to be the biggest community out there:

I am doing research on Laravel routing #performance and I need sample data. I am looking for real-life, real-world examples of big routes/web.php and routes/api.php files (or older app/Http/routes.php files).

If you want to help me, you can use the form below to either send me links (to Github projects for example) or just paste the contents of the files. Thanks in advance!

Kaloyan K. Tsvetkov March 27, 2021

That did not help either. Finally, I started thinking that I do not need a PHP project to benchmark this: I just need a big enough list of paths to route. Where do you think one can find such lists? Some of you probably have the answer in their heads, at least one of the answers - Swagger API Docs.

There are so many out there, and while looking for what to use, I started thinking that it would be nice if the API is somewhat popular and familiar and not anything obscure. Following this, I decided to use Bitbucket API. It has 182 routes, which are more than enough. Some of the routes are static (like /addon/linkers), but most are dynamic (e.g. /addon/linkers/{linker_key}/values).

You can see the full list here: https://developer.atlassian.com/bitbucket/api/2/reference/resource/

Only the paths are used, and the HTTP verbs/methods are ignored. This is a sample of what the input for the benchmark cases looks like

/addon
/addon/linkers
/addon/linkers/{linker_key}
/addon/linkers/{linker_key}/values
/addon/linkers/{linker_key}/values/{value_id}
/hook_events
/hook_events/{subject_type}
/pullrequests/{selected_user}
/repositories
/repositories/{workspace}
/repositories/{workspace}/{repo_slug}
...

Benchmark Cases

I did not wanted to do any of the adhoc benchmark scripts you see a lot. There is a very mature and good PHP benchmarking package in PHPBench, and I decided to use that.

Both libraries have different "strategies" when it comes to how they match the request path against the routes they have collected. I wanted to be able to benchmark each of them:

• Symfony Routing has UrlMatcher and CompiledUrlMatcher
• FastRoute has GroupCountBased, GroupPosBased, CharCountBased and MarkBased

There are three different scenarios which I want to benchmark:

• benchLast - match the last route in the route collection, as this is considered the worst case
• benchLongest -- match the longest route to test the complexity of parsing bigger paths
• benchAll -- match all of the routes in the route collection to average the overall performance

Finally, I wanted to be able to measure the number of route matches per second. I created a script for that that is using the phpbench cases and just runs thems and calculates the "per-second" stats.

Results

The project I created is at https://github.com/kktsvetkov/benchmark-php-routing

I worked on this on my home machine, which is an old beaten up 2011 Mac Mini. It is with PHP 7.1.3 and it showed symfony_compiled as the fastest case. There are no surprises here, as CompiledUrlMatcher has provided the fastest routing for several years now.

I've set up the Github Actions for that project to execute the benchmark scripts for PHP 7.1, PHP 7.2, PHP 7.3 and PHP 7.4 and compare the difference between the PHP versions.

Here are the results from quick-benchmark.php which calculates the paths matches per second:

PHP 7.1

+------------------+--------------+--------+------------------+-----------------+
| Case             | Scenario     | Routes | Time             | Per Second      |
+------------------+--------------+--------+------------------+-----------------+
| symfony_compiled | benchAll     | 364    | 0.409297 seconds | 889.329776154   |
| symfony_compiled | benchLast    | 300    | 0.481942 seconds | 622.48162298513 |
| symfony_compiled | benchLongest | 300    | 1.230177 seconds | 243.86731339213 |
| fast_group_count | benchAll     | 364    | 2.854642 seconds | 127.51160408835 |
| fast_group_pos   | benchAll     | 364    | 2.876335 seconds | 126.54992612869 |
| fast_mark        | benchAll     | 364    | 2.888358 seconds | 126.02315411124 |
| fast_char_count  | benchAll     | 364    | 2.902446 seconds | 125.4114619327  |
| fast_group_pos   | benchLast    | 300    | 2.475604 seconds | 121.1825566395  |
| fast_mark        | benchLast    | 300    | 2.496498 seconds | 120.16833808118 |
| fast_char_count  | benchLast    | 300    | 2.507835 seconds | 119.62509918124 |
| fast_group_count | benchLast    | 300    | 2.668138 seconds | 112.43796120841 |
| symfony          | benchAll     | 364    | 3.658297 seconds | 99.49985404727  |
| fast_mark        | benchLongest | 300    | 3.198704 seconds | 93.787984007856 |
| fast_char_count  | benchLongest | 300    | 3.223138 seconds | 93.076999323905 |
| fast_group_pos   | benchLongest | 300    | 3.230840 seconds | 92.855109815605 |
| fast_group_count | benchLongest | 300    | 3.304675 seconds | 90.780482413295 |
| symfony          | benchLongest | 300    | 3.723648 seconds | 80.566152938331 |
| symfony          | benchLast    | 300    | 4.979773 seconds | 60.243709364656 |
+------------------+--------------+--------+------------------+-----------------+

PHP 7.2

+------------------+--------------+--------+------------------+-----------------+
| Case             | Scenario     | Routes | Time             | Per Second      |
+------------------+--------------+--------+------------------+-----------------+
| symfony_compiled | benchAll     | 364    | 0.360886 seconds | 1008.6284926794 |
| symfony_compiled | benchLast    | 300    | 0.442319 seconds | 678.24365843404 |
| symfony_compiled | benchLongest | 300    | 0.992547 seconds | 302.25267856885 |
| fast_mark        | benchAll     | 364    | 2.034539 seconds | 178.91030980931 |
| fast_group_pos   | benchAll     | 364    | 2.083953 seconds | 174.66805480364 |
| fast_char_count  | benchAll     | 364    | 2.116721 seconds | 171.9640966591  |
| fast_group_count | benchAll     | 364    | 2.168229 seconds | 167.8789383841  |
| fast_mark        | benchLast    | 300    | 1.850838 seconds | 162.08874507808 |
| fast_char_count  | benchLast    | 300    | 1.862375 seconds | 161.08463473807 |
| fast_group_pos   | benchLast    | 300    | 1.904495 seconds | 157.52207271104 |
| fast_group_count | benchLast    | 300    | 1.929290 seconds | 155.49761376288 |
| symfony          | benchAll     | 364    | 2.519544 seconds | 144.47059326726 |
| fast_group_pos   | benchLongest | 300    | 2.380750 seconds | 126.01070142075 |
| fast_group_count | benchLongest | 300    | 2.410906 seconds | 124.43454472688 |
| fast_char_count  | benchLongest | 300    | 2.416596 seconds | 124.14155215417 |
| fast_mark        | benchLongest | 300    | 2.454727 seconds | 122.21317436736 |
| symfony          | benchLongest | 300    | 2.693696 seconds | 111.37114119265 |
| symfony          | benchLast    | 300    | 3.420336 seconds | 87.710680848901 |
+------------------+--------------+--------+------------------+-----------------+

PHP 7.3

+------------------+--------------+--------+------------------+-----------------+
| Case             | Scenario     | Routes | Time             | Per Second      |
+------------------+--------------+--------+------------------+-----------------+
| symfony_compiled | benchAll     | 364    | 0.333783 seconds | 1090.5283873862 |
| symfony_compiled | benchLast    | 300    | 0.383387 seconds | 782.49945119558 |
| symfony_compiled | benchLongest | 300    | 0.921058 seconds | 325.7124086605  |
| fast_mark        | benchAll     | 364    | 2.041492 seconds | 178.30096939752 |
| fast_group_pos   | benchAll     | 364    | 2.126128 seconds | 171.20324229753 |
| fast_group_count | benchAll     | 364    | 2.152566 seconds | 169.1005095411  |
| fast_char_count  | benchAll     | 364    | 2.172326 seconds | 167.56233880761 |
| fast_char_count  | benchLast    | 300    | 1.876143 seconds | 159.90252523982 |
| fast_group_count | benchLast    | 300    | 1.878880 seconds | 159.66958835844 |
| fast_group_pos   | benchLast    | 300    | 1.904677 seconds | 157.50702796943 |
| fast_mark        | benchLast    | 300    | 1.913834 seconds | 156.75339925419 |
| symfony          | benchAll     | 364    | 2.562547 seconds | 142.04617690139 |
| fast_char_count  | benchLongest | 300    | 2.394527 seconds | 125.28570577975 |
| fast_group_count | benchLongest | 300    | 2.419258 seconds | 124.00496969431 |
| fast_mark        | benchLongest | 300    | 2.433419 seconds | 123.28333153219 |
| fast_group_pos   | benchLongest | 300    | 2.499992 seconds | 120.00038910038 |
| symfony          | benchLongest | 300    | 2.713896 seconds | 110.54218584799 |
| symfony          | benchLast    | 300    | 3.544668 seconds | 84.634161352191 |
+------------------+--------------+--------+------------------+-----------------+

PHP 7.4

+------------------+--------------+--------+------------------+-----------------+
| Case             | Scenario     | Routes | Time             | Per Second      |
+------------------+--------------+--------+------------------+-----------------+
| symfony_compiled | benchAll     | 364    | 0.334722 seconds | 1087.4702395071 |
| symfony_compiled | benchLast    | 300    | 0.391615 seconds | 766.05866967986 |
| symfony_compiled | benchLongest | 300    | 0.918377 seconds | 326.66326060138 |
| fast_group_pos   | benchAll     | 364    | 1.836413 seconds | 198.2125036709  |
| fast_char_count  | benchAll     | 364    | 1.854658 seconds | 196.26261116629 |
| fast_group_count | benchAll     | 364    | 1.894953 seconds | 192.08919567152 |
| fast_mark        | benchAll     | 364    | 1.907493 seconds | 190.82637690195 |
| fast_mark        | benchLast    | 300    | 1.670489 seconds | 179.58812475032 |
| fast_char_count  | benchLast    | 300    | 1.686217 seconds | 177.91303706026 |
| fast_group_pos   | benchLast    | 300    | 1.698040 seconds | 176.67428240226 |
| fast_group_count | benchLast    | 300    | 1.705245 seconds | 175.9277973736  |
| symfony          | benchAll     | 364    | 2.359127 seconds | 154.29436105241 |
| fast_mark        | benchLongest | 300    | 2.150827 seconds | 139.48123657748 |
| fast_group_count | benchLongest | 300    | 2.189247 seconds | 137.03341011817 |
| fast_char_count  | benchLongest | 300    | 2.192200 seconds | 136.84883106702 |
| fast_group_pos   | benchLongest | 300    | 2.279765 seconds | 131.59250318029 |
| symfony          | benchLongest | 300    | 2.453912 seconds | 122.2537717685  |
| symfony          | benchLast    | 300    | 3.199316 seconds | 93.77004262047  |
+------------------+--------------+--------+------------------+-----------------+

Now, the takeaways:

• Indeed symfony_compiled is the fastest strategy. When comparing results for benchAll, it is 5.45x to 7x faster. This is the case where we match all of the available routes, so it contains both the good and the bad situations.

• The symfony_compiled strategy is not only the fastest when matching all routes, but also for benchLast and benchLongest. The gains however are more moderate as when matching the last route it is 4x to 5x faster, and when matching the longest route it is "only" 2.3x to 2.6x faster.

• All strategies struggle with both benchLast and benchLongest when compared to the average speed of benchAll. Matching the last route is 1.3x to 2.35x faster than matching the longest route. In the same time when comparing the average case of matching all routes against matching the last route, that operation is 1.2x to 1.45x slower.

Finally, why all the trouble? Well ...

Before improving something, you must be able to measure it.

Here's my Symfony Routing vs. FastRoute benchmark using a real-world scenario - that of Bitbucket API.

Kaloyan K. Tsvetkov May 30, 2021

That's right. Hopefully it will not take a lot more time for me to be able to share more details about my PHP routing project.

Edits

• June 1, 2021: Thanks to Jacob Dreesen for helping me fix some typos.

• June 3, 2021: I found a bug with quick-benchmark.php where both benchLast and benchLongest were executing the same case. I executed the benchmarks again, and I've updated the post with the new results, and - a fresh set of takeaways

• June 7, 2021: Nikita Popov and Saif Eddin Gmati pointed out in a tweet that I am not using the cachedDispatcher() in FastRoute. I did added that to the benchmark project and that drastically changed the results. You can read more about it here.