Introduction

Those of you that have been around the PHP community for any significant period of time have likely come across Xdebug, which is more or less the gold standard for debugging and development tooling for the language.

Whilst it has incredibly useful tools to help you get into the very bowels of your code when you're trying to solve a particuarly nasty edge case, you've likely found yourself wondering why your test suite now runs many orders of magnitude slower.

This post is not the place to scratch the surface of that issue, but suffice it to say that recording every function call, execution path, and variable assignment and profiling your code is no small feat.

Reporting on code coverage

The issue that I stumbled upon over the weekend was in a large factor of the Laravel Scheduled Task montioring SaaS Jake and I have been working on, thenping.me.

In the midst of refactoring the way we handle failure alert thresholds and customer notifications, we made significant changes - initially a handful of lines in a small number of files that wound up being 103 files and thousands of lines - that left me wondering if we had sufficient coverage of our application to verify everything was continuing to work as intended.

Code coverage tells you which lines of script (or set of scripts) have been executed during a request. With this information you can for example find out how good your unit tests are.

Fortunately, Xdebug provides code coverage analysis functionality out of the box, and using Pest (or PHPUnit natively), it's just a matter of passing an option to the binary in order to get the data I needed. Note that our test suite typically takes around 16 seconds to run.

1./vendor/bin/pest --coverage
2 
3Tests:  263 passed
4Time:   215.95s
5Cov:    82.46%

Pleasingly, we have a touch over 80% coverage. We're not looking for complete coverage, either, but good coverage over our critical code paths. There's lots of untested things - like default Laravel controllers, providers, etc. - that don't provide us much value.

What is less pleasing, however, is that our 16-second test suite is now taking 3.5 minutes to run, just so we can get an understanding of our code coverage.

This is on top of a couple Xdebug-specific test failures that were solved by increasing its xdebug.max_nesting_level value to 10,000. I suspect this is more to do with nested Blade templates and the way Laravel tends to pass the application container around.

But there's just needless layers of interruption on top of your workflow when all you want to do is get an understanding of your test coverage.

A new champion enters the arena

Enter James King.

Pcov for code coverage is a lot faster and seems to not get this maximum nesting level error

— James King (@Jamesking56) July 12, 2020

I was in the middle of hanging out with my family, but I was keen to check it out. I'd heard Joe Watkins talk about this tool on previous occasions but I'd not really looked into it too much previously. An encumbant's tool has that mindshare effect on you.

There's not really much to getting up and running with pcov.

1pecl install pcov

On my system (macOS 10.15) at the time of writing, though it stands to reason this will be the case for any system with PECL available, this will install the pcov extension, register it with PHP, and enable it.

That's it. You can check if you don't believe me.

1$ php -i | grep pcov
2pcov
3pcov.directory => auto
4pcov.exclude => none
5pcov.initial.memory => 65336 bytes
6pcov.initial.files => 64

The next time I ran my coverage report I was pleasantly surprised.

1./vendor/bin/pest --coverage
2 
3Tests:  263 passed
4Time:   19.93s
5Cov:    82.46%

It has negligible impact on my test suite, and gives me the same data in a fraction of the time as Xdebug.

When to choose pcov over Xdebug

You have access to the same output formats that are available to PHPUnit (formatted output, clover, JSON, HTML, etc.) with none of the overhead.

All this to say that pcov and Xdebug are very different tools that happen to offer overlap in functionality.

If you're after something that is going to give you just the coverage data - and something you can put into a CI process to track that your coverage doesn't go down over time, without blowing out your test suite, perhaps consider pcov over Xdebug.

You can even leave pcov enabled at all times, as it won't have the same impact on your development workflow as having Xdebug enabled.

If you need the enhanced debugging capabilities that Xdebug provides, and you're happy to wait for coverage reports to run periodically, check out Xdebug.

That said, there is absolutely no reason you can't have them both available side-by-side. Just be sure to toggle Xdebug off if you're running your test suite, or want to generate coverage reports.