PHP and Go, Together at Last!

PHP and Go, Together At Last!
PHP and Golang, together at last!

This blog post is about Spiral Framework and Roadrunner Server. I’ll briefly talk about what they are, then show how to compile a custom Roadrunner server, start developing with Spiral, using Docker.

Explain Like I’m 5 PHP Developers

Roadrunner works by creating a HTTP server with Golang’s excellent net/http package, and using Goridge as a bridge to pass PSR7 Request and Responses between PHP and Go. The PHP application is then a long-running, already bootstrapped PSR7-capable application that received the already parsed PSR7 request, dispatches it, and collects the response to give back into Go. [1]

Roadrunner offloads unnecessary operations from PHP to a more optimized server, and effectively swaps out the classic setup of Nginx+FPM with a PHP/Golang application that boosts flexibility and performance. [2]

Roadrunner can serve static files without the presence of Nginx, therefore, simplifying the creation of Docker containers. [3]

You can extend your PHP application by including Go libraries, [4], writing Go HTTP middleware, [5], or tweaking and extending the Roadrunner server. [6]

Spiral is a PHP Framework with a customized Roadrunner server. The main difference is that, when you use Spiral’s version of Roadrunner, it comes with more out-of-the-box solutions for PHP developers, Ie.

It’s possible to download the server pre-compiled, but that takes away our power of writing Go code. In this tutorial we start from scratch.

Let’s Go!

The instructions are for Mac, and assume you already have Docker Desktop installed, but the same concepts should work for Linux, and probably Windows.

Step 1

Create a directory for your project.
(If you want, replace hello-spiral with some other name.)

mkdir ~/hello-spiral

Step 2

Create a subdirectory called ./server/ and copy these files into it:

Example:

 
cd ~/hello-spiral
mkdir server
cd server
wget https://raw.githubusercontent.com/spiral/framework/master/main.go
wget https://raw.githubusercontent.com/spiral/framework/master/go.mod

Step 3

Download this Dockerfile into the root dir of ~/hello-spiral

The first stage compiles the app server, the second stage installs the Spiral skeleton app.

Step 4

Your file tree should look like this:

cd ~/hello-spiral; tree

Build a new Docker image:

cd ~/hello-spiral
docker build -t hello-spiral .

Step 5

Run the new Docker image:

docker run -it -p 8080:8080 -p 2112:2112 hello-spiral

Step 6

Go to http://localhost:8080 and verify that it works.

localhost:8080

Tada! It runs, but how do we develop?

Step 7

Let’s copy all the PHP files that were successfully installed in the container to our host, then mount them.

While the container is still running from Step 5, in another shell, do:

cd ~/hello-spiral
docker ps

This command will output your container ID:

docker ps

Use your ID in the next command (replace 1f057ae4e473 with your own id):

docker cp 1f057ae4e473:/var/www/app/. src

In the shell tab that is still running the server, stop the server (ctrl-c), then restart with a slight variation of the command from Step 5:

docker run -it -p 8080:8080 -p 2112:2112 -v "$(pwd)"/src:/var/www/app:cached hello-spiral

Keep Going!

Your local PHP files are now mounted in ~/hello-spiral/src, start developing! Change your Dockerfile:

# Setup Spiral
# RUN composer create-project spiral/app . --no-scripts
# Or comment above, uncomment below, and copy Spiral 
COPY ./src/ .

If you make changes in ~/hello-spiral/server, rebuild!

Machine Learning and AI using PHP

Montrealer’s who won’t let PHP go (pun intended), get in on that sweet venture capital? Here are two fantastic PHP options for doing machine learning (ML), artificial intelligence (AI), and unprecedented memorization (Singularity), in a dev stack we’re already pretty good at:

Rubix ML

Amazing tutorials like Iris Flower Classifier, Credit Risk Predictor, and Sentiment Analysis make all that buzzword soup turn into real code you can learn from and build on.

PHP-ML

Winner of the prestigious Eastern European guy handing out money to projects he likes award, this library is solid clean code.

Other camps may have their sensible conventions, consistent syntax, math fundamentals… But we have PHP! Let’s go!

Pcov Is Better than Phpdbg and Xdebug for Code Coverage

Since the last time, I changed from phpdbg to pcov.

Results? pcov is faster, uses less memory, is more accurate than phdbg.

Phpdbg

phpdbg
Time: 4.09 minutes, Memory: 3.32 GB

Pcov

pcov
Time: 3.48 minutes, Memory: 1.36 GB

Caveat, only works with PHPUnit 8+… Or does it?

Because my project is a WordPress plugin, and because WordPress has unresolved issues older than my 9-year-old, they don’t support PHPUnit 8, yet.

Lucky for me, unlucky for the maintainer who would rather not support this gross hack, pcov works with PHPUnit 7, and probably PHPUnit 6, thanks to pcov-clobber.

Previously on (show name)

In June 2018 I ran into a problem using Xdebug for code coverage.

PHPUnit without Xdebug took 5 minutes. PHPUnit with Xdebug (to generate code coverage reports) took ~50 minutes. This was too long for Travis CI. The job would crash, abort, and never finish code coverage.

Fixed by switching to phpdbg. It ran 10x faster. Maybe the code coverage metrics were a bit worse but at least it ran.

All was well until it wasn’t. Last week phpdbg started crashing Travis CI with [PHP Fatal error: Out of memory].

I needed another solution. Enter pcov.

How to switch to pcov if you are stuck with PHPUnit 7

It all started here?

Or maybe here: Running for Coverage

Basically, this:

So if you are doing this:

vendor/bin/phpunit --configuration phpunit.xml --coverage-clover coverage.xml

Or this:

phpdbg -qrr -d memory_limit=-1 vendor/bin/phpunit --configuration phpunit.xml --coverage-clover coverage.xml

Simply change to:

pecl install pcov; composer require pcov/clobber; vendor/bin/pcov clobber; vendor/bin/phpunit --configuration phpunit.xml --coverage-clover coverage.xml

And you’re done.

Here’s my diff:

Refactor Your Slow Form Using PHP Generators and Event Streams

Your form will still be slow but the user experience will be better. They will see a progress bar and see status updates in real time. The idea is to refactor something like this:

/**
 * A task that takes way too loooooooooooooooooooooooong...
 */
function task() {
    step1();
    step2();
    step3();
    //
    step100();
}

Into this:

/**
 * Yields a key/value pair
 * The key is between 1-100 and represents percentage completed
 * The value is a string of information for the user
 *
 * @return Generator
 */
function taskGenerator() {
    step1();
    yield 1 => 'Completed step 1';
    step2();
    yield 2 => 'Completed step 2';
    step3();
    yield 3 => 'Completed step 3';
    //
    step100();
    yield 100 => 'Completed step 100';
}
/**
 * A task that takes way too loooooooooooooooooooooooong...
 */
function task() {
    foreach (taskGenerator() as $percentage => $info) {
        // Do nothing, this is a compatibility wrapper 
        // that makes our generator work like a regular function
    }
}

And this:


let evtSource = new EventSource(url);

Before

Imagine you have this form somewhere on your corporate intranet:

Spinning beach ball of death. (Source code)

The user clicks submit. They wait, and wait, and wait. The task completes and they receive some feedback saying “everything seems fine”. It’s not particularly good but it does the job. Your company doesn’t have the resources, infrastructure, or competence to setup job queues and delegate these kind of tasks into the background. Everyone lives with it. The end.

Insert cliché “What if I told you” meme here.

After

It is possible to use PHP Generators and Event Streams to provide real-time feedback to the web browser.

The EventStream console in Chrome Browser.

With a bit of refactoring your old form could, instead, behave like this:

A status bar with real-time status updates. (Source code)

The heavy drinking in the new form is an EventEmitter class (Source code).

On the front end, the main changes were from this:

<input type="submit">

To:

<p><input type="submit"></p>
<div id="sse-progressbar"></div>
<p id="sse-info"></p>

And some JavaScript near the end of the form:

$('#tpsreport').on('submit', function (e) {
    e.preventDefault();
    let formSubmitButton = $('#tpsreport :submit');
    formSubmitButton.attr('disabled', true);

    let form = $('#tpsreport');
    let actionUrl = form.prop('action');
    let eventSourceUrl = actionUrl + (actionUrl.includes('?') ? '&' : '?') + $.param(form.find(':input'));

    let evtSource = new EventSource(eventSourceUrl);
    evtSource.onopen = function () {
        formSubmitButton.hide();
    };
    evtSource.onmessage = function (message) {
        let bar = $('#sse-progressbar');
        let info = $('#sse-info');
        let data = JSON.parse(message.data);
        switch (data.action) {
            case 'updateStatusBar':
                bar.progressbar({value: parseInt(data.percentage, 10)});
                info.html(data.info);
                break;
            case 'complete':
                evtSource.close();
                if (data.error) {
                    bar.progressbar({value: false});
                    info.html(data.error);
                } else {
                    window.location = actionUrl;
                }
                break;
        }
    };
    evtSource.onerror = function () {
        evtSource.close();
        $('#sse-progressbar').progressbar({value: false});
        $('#sse-info').html('EventStream Connection Error');
    };
});

The JavaScript (and jQuery) snippet:

  • Targets the form with id tpsreport
  • Stops the form from submitting and instead
  • Appends all the form data as $_GET parameters to the form’s action URL then
  • Passes that to a new EventSource
  • Updates sse-progressbar and sse-info when it receives an event stream message
  • Redirects the user back to the action URL when complete

On the back end, the time consuming function was refactored into a generator that yields a key/value pair. The key is between 1-100 and represents percentage completed. The value is a string of information meant for the user. Once you have a generator that follows this convention, pass it to the EventEmitter. The browser will start receiving an event stream.

/**
 * @return Generator
 */
function loooooooooooooooooooooooongGenerator() {
    yield 10 => "Hey Peter what's happening. I'm going to need those TPS reports... ASAP...";
    sleep(2);
    yield 30 => "Ah, ah, I almost forgot... I'm also going to need you to go ahead and come in on Sunday, too. We, uhhh, lost some people this week and we sorta need to play catch-up. Mmmmmkay? Thaaaaaanks.";
    sleep(2);
    yield 50 => '...So, if you could do that, that would be great...';
    sleep(2);
    yield 60 => 'Excuse me, I believe you have my stapler.';
    sleep(2);
    yield 90 => 'PC LOAD LETTER';
    sleep(2);
    yield 100 => 'Success!';
}
$emitter = new \KIZU514\EventEmitter();
$emitter->emit( loooooooooooooooooooooooongGenerator() );

Key ideas:

  • It’s not necessary to wait until the request finishes, PHP can emit event-stream responses (SSE) back to the web browser while it is working on something.
  • PHP Generators are a relatively simple refactoring hack to get those responses back to the browser.
  • sleep() is only meant as an example of a function call that takes a long time to finish, don’t put sleep in your production code, you already knew this, I hope?

TL;DRhttps://gist.github.com/connerbw/a2fa7712efe135d5854f4d32d67ca09f

Phpdbg Is Much Faster Than Xdebug For Code Coverage

I work on a project that uses Travis CI to test and build against three jobs. (PHP 7.0, 7.1, 7.2) I recently ran into a roadblock where Travis would fail with “No output has been received” on the job that did code coverage using Xdebug. A screenshot of the last successful build before the failure:

PHP 7.0.8 runs the tests without Xdebug and PHP 7.1 runs the same tests with Xdebug + Code coverage.

Only one of our three jobs runs code coverage because it’s ridiculously slow. We were disabling coverage on the other jobs so that, in a worse case scenario, we could at least check a hotfix in under 5 minutes.

This system was working fine until a few days ago. The test suite has kept growing and we got to the point where Travis just wouldn’t run code coverage anymore.

We tried every trick in the book and the best I could get it down to, minus the other stuff required to build (git clone, install dependencies, yarn build, phpcs, …) was 42 minutes.

Hooray the build didn’t crash and it only took 42 minutes to check!

Enter Phpdbg

PHP 7.0.8 runs the tests without Phpdbg and PHP 7.1 runs the same tests with Phpdbg + Code coverage.
WAT?!

Down from 42 minutes to 4!

Source:

https://docs.travis-ci.com/user/speeding-up-the-build/#php-optimizations

Caveats:

  • Travis CI docs say to do phpenv config-rm xdebug.ini but this crashes the build on environments where Xdebug is not installed. Fixed by conditionally checking if Xdebug is on:
before_script:
- if php -v | grep -q 'Xdebug'; then phpenv config-rm xdebug.ini; fi
  • phpdbg didn’t work in PHP 7.2: /home/travis/.travis/job_stages: line 57: 8492 Segmentation fault (core dumped). Fixed by running phpdbg only on PHP 7.1:
script:
- if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.1" ]]; then phpdbg -qrr -d memory_limit=-1 vendor/bin/phpunit --configuration phpunit.xml --coverage-clover coverage.xml; fi
- if [[ ${TRAVIS_PHP_VERSION:0:3} != "7.1" ]]; then vendor/bin/phpunit --configuration phpunit.xml; fi
  • A test that uses exec didn’t work: Unable to fork […] Fixed by only running that test when phpdbg is disabled. As we have three jobs (PHP 7.0, 7.1, and 7.2) and two of them don’t run code coverage (PHP 7.0 and 7.2) the test itself still runs, we just lose a bit of code coverage when running against PHP 7.1:

 $runtime = new \SebastianBergmann\Environment\Runtime();
 if ( ! $runtime->isPHPDBG() ) {
  // // TODO: exec(): Unable to fork error when running phpdbg()
 }
  • “The output is not the same…” Fixed by not giving a shit? Is there really 42 minutes of justifiable output difference between Xdebug and Phpdbg? Our codecov metrics remained the same. Our Travis build stopped failing.

I <? PHP

PHP7 Learning Resources

PHP made a lot of mistakes. Many are still in the language today. However PHP is 23 years old now. PHP7, released December 2015 is a huge improvement over older versions. PHP7 outperforms Ruby, Python, and many others. If you haven’t looked at it this year, do so. “PHP sucks” is mostly FUD now.

For PHP Developers:

For C Developers:

“You can do anything at zombocom with PHP. It’s shared nothing architecture makes it decent for web development.”

O’Doyle Rules!

Namespaces and Objects

Markdown editing for WordPress.

Parsedown Party is a new WordPress Plugin I wrote with PHP namespaces, Composer support, 90% code coverage, and Travis-CI for automatic testing and releases.

While writing this Plugin, I settled on some WordPress coding conventions that I’d like to talk about.

Namespaces

One of the weirder things I read as a WordPress Plugin developer are tips such as “prefix all your function with name_” or “put all your functions in a class and declare them as static.” This advice is to prevent code collisions with the other 53,123+ Plugins available for WordPress. Big number.

On the other hand, Packagist lists 164,796+ PHP Composer packages, does not follow the aforementioned recommendations and has no such code collisions. Why? Because they use namespaces.

PHP Namespaces have been available since 2009. Other programming languages such as Java or Perl have had them for longer. WordPress Core doesn’t encourage PHP Namespaces but the syntax is fully supported.

If my class is a bunch of static methods and nothing else then I am doing it wrong. I should instead write a library of functions. If I’m afraid of function name collisions then I should use Namespaces because they solve that exact problem and they work fine with WordPress.

add_action( 'login_head', '\Kizu514\Coolname\this_works_fine' );
add_filter( 'login_headerurl', '\Kizu514\Coolname\welcome_to_php' );

Objects

Another odd thing I see while browsing OPP (other people’s plugins) are objects with “mostly” static methods.  There is some object-oriented approach to the design but most of the methods are static because they hook into actions and filters.

I think that an object in a WordPress Plugin should have, at most, two static methods. Every other method in a class should be public, protected or private.

There are very few cases where static methods are necessary. Most of the time public methods can be used. At worst, namespaced functions can invoke objects as needed and these concerns should be separate.

Here’s an example of what I mean (the code must also have PHPDoc comments  but for brevity, I have omitted these):

namespace Kizu514\CoolClass;

class Plugin {
	private static $instance = null;
	private $obj1;
	private $obj2;
	private $obj3;

	static public function init() {
		if ( is_null( self::$instance ) ) {
			$obj1 = new \Obj1();
			$obj2 = new \Obj2();
			$obj3 = new \Obj4( new \Obj3() );
			self::$instance = new self( $obj1, $obj2, $obj3 );
			self::hooks( self::$instance );
		}
		return self::$instance;
	}

	static public function hooks( Plugin $obj ) {
		add_action( 'save_post', [ $obj, 'doSomething' ] );
		add_filter( 'the_content', [ $obj, 'doSomethingElse' ] );
	}

	public function __construct( $obj1, $obj2, $obj3 ) {
		$this->obj1 = $obj1;
		$this->obj2 = $obj2;
		$this->obj3 = $obj3;
	}

	public function doSomething( $post_id ) { /**/ }

	public function doSomethingElse( $content) { /**/ }
}

Some time passes….

add_action( 'after_setup_theme', [ '\Kizu514\CoolClass\Plugin', 'init' ] );

The first static method, init, is hooked into one of many available launch points like plugins_loadedafter_setup_theme, or init. Because these actions happen before everything else a second static method, hooks, can do the rest of the work from inside the object itself.

Admittedly, even if I am down to just two static methods, they are gross. Glaring problems include “Singletons are evil” and not using Inversion of Control.

I’m very happy with this code style and I will be using it moving forward.

Install Phalcon PHP On Windows 10

A common misconception is that you need a web server like IIS, Apache, or Nginx to get started with PHP7 development. In fact, PHP7 has its own built in web server that you can invoke at the command prompt. Many modern PHP frameworks support this, such as Phalcon PHP.

Prerequisite:
PHP7 and Composer on Windows 10

Installing Phalcon

Download the latest DLL file from GitHub and unzip to C:\PHP7\ext

Make sure the file you download matches your installed PHP version. For me, it was the non-thread safe version, so I picked a file that ended with _nts.zip

Add extension=php_phalcon.dll to your php.ini file

Drop to the command prompt and do:

php -v

If you get an error like “PHP Startup: Unable to load dynamic library…” that just means you installed the wrong version:

Whoops, I need x86 not x64.

Don’t panic, download another version, overwrite the DLL, and try again:

No news is good news.

(Source)

Installing Phalcon Dev Tools

In a new folder create a bare bones composer.json file with only this in it:

{
    "require-dev": {
        "phalcon/devtools": "~3.2"
    }
}

At the command prompt, cd to your folder, and do:

composer install

Create a simple project named my_project:

vendor\bin\phalcon.php.bat project my_project simple . 

Launch the built in web server:

cd my_project
..\vendor\bin\phalcon.php.bat serve

(Source)

Your Password Hashing Algorithm Is Bad And You Should Feel Bad

No

$pw = md5('password');
$pw = md5('salt' . 'password');
$pw = md5('complicated_salt' . 'password');
$pw = md5('complicated_salt' . strrev('password')); // Don't be clever.

Where md5() = sha1(), base64_encode(), etc.

This type of password hashing is still widespread and susceptible to rainbow table attacks.

Yes

$pw = password_hash('password', PASSWORD_DEFAULT);

(Source)

Uses bcrypt, this particular implementation auto-magically hardens itself over time.

How to use

You are responsible for new \Pdo(), $condition, maybe asking the user to make their 'password' not suck. Read the snippet and reason about it. Don’t just copy/paste, it won’t work.


// Save user password into database

$pw = password_hash($_REQUEST['pw'], PASSWORD_DEFAULT);
$stmt = $pdo->prepare('UPDATE users SET password=? WHERE condition=?');
$stmt->execute([$pw, $condition]);

// Verify user login

$stmt = $pdo->prepare('SELECT password FROM users WHERE condition=?'); 
$stmt->execute([$condition]); 
$row = $stmt->fetch();

if (password_verify($_REQUEST['pw'], $row['password'])) {
  // Check if PHP has improved password security for us
  if (password_needs_rehash($row['password'], PASSWORD_DEFAULT)) {
    // Fix password for next time
    $pw = password_hash($_REQUEST['pw'], PASSWORD_DEFAULT);
    $stmt = $pdo->prepare('UPDATE users SET password=? WHERE condition=?');
    $stmt->execute([$pw, $condition]);
  }
  // Log in
} else {
  // Invalid password
}

Keep on shaking that salt shaker.