Refactor Your Slow Form Using PHP Generators and Event Streams

Your form will still be slow but the user experience will be better. They will see a progress bar and see status updates in real time. The idea is to refactor something like this:

/**
 * A task that takes way too loooooooooooooooooooooooong...
 */
function task() {
    step1();
    step2();
    step3();
    //
    step100();
}

Into this:

/**
 * Yields a key/value pair
 * The key is between 1-100 and represents percentage completed
 * The value is a string of information for the user
 *
 * @return Generator
 */
function taskGenerator() {
    step1();
    yield 1 => 'Completed step 1';
    step2();
    yield 2 => 'Completed step 2';
    step3();
    yield 3 => 'Completed step 3';
    //
    step100();
    yield 100 => 'Completed step 100';
}
/**
 * A task that takes way too loooooooooooooooooooooooong...
 */
function task() {
    foreach (taskGenerator() as $percentage => $info) {
        // Do nothing, this is a compatibility wrapper 
        // that makes our generator work like a regular function
    }
}

And this:


let evtSource = new EventSource(url);

Before

Imagine you have this form somewhere on your corporate intranet:

Spinning beach ball of death. (Source code)

The user clicks submit. They wait, and wait, and wait. The task completes and they receive some feedback saying “everything seems fine”. It’s not particularly good but it does the job. Your company doesn’t have the resources, infrastructure, or competence to setup job queues and delegate these kind of tasks into the background. Everyone lives with it. The end.

Insert cliché “What if I told you” meme here.

After

It is possible to use PHP Generators and Event Streams to provide real-time feedback to the web browser.

The EventStream console in Chrome Browser.

With a bit of refactoring your old form could, instead, behave like this:

A status bar with real-time status updates. (Source code)

The heavy drinking in the new form is an EventEmitter class (Source code).

On the front end, the main changes were from this:

<input type="submit">

To:

<p><input type="submit"></p>
<div id="sse-progressbar"></div>
<p id="sse-info"></p>

And some JavaScript near the end of the form:

$('#tpsreport').on('submit', function (e) {
    e.preventDefault();
    let formSubmitButton = $('#tpsreport :submit');
    formSubmitButton.attr('disabled', true);

    let form = $('#tpsreport');
    let actionUrl = form.prop('action');
    let eventSourceUrl = actionUrl + (actionUrl.includes('?') ? '&' : '?') + $.param(form.find(':input'));

    let evtSource = new EventSource(eventSourceUrl);
    evtSource.onopen = function () {
        formSubmitButton.hide();
    };
    evtSource.onmessage = function (message) {
        let bar = $('#sse-progressbar');
        let info = $('#sse-info');
        let data = JSON.parse(message.data);
        switch (data.action) {
            case 'updateStatusBar':
                bar.progressbar({value: parseInt(data.percentage, 10)});
                info.html(data.info);
                break;
            case 'complete':
                evtSource.close();
                if (data.error) {
                    bar.progressbar({value: false});
                    info.html(data.error);
                } else {
                    window.location = actionUrl;
                }
                break;
        }
    };
    evtSource.onerror = function () {
        evtSource.close();
        $('#sse-progressbar').progressbar({value: false});
        $('#sse-info').html('EventStream Connection Error');
    };
});

The JavaScript (and jQuery) snippet:

  • Targets the form with id tpsreport
  • Stops the form from submitting and instead
  • Appends all the form data as $_GET parameters to the form’s action URL then
  • Passes that to a new EventSource
  • Updates sse-progressbar and sse-info when it receives an event stream message
  • Redirects the user back to the action URL when complete

On the back end, the time consuming function was refactored into a generator that yields a key/value pair. The key is between 1-100 and represents percentage completed. The value is a string of information meant for the user. Once you have a generator that follows this convention, pass it to the EventEmitter. The browser will start receiving an event stream.

/**
 * @return Generator
 */
function loooooooooooooooooooooooongGenerator() {
    yield 10 => "Hey Peter what's happening. I'm going to need those TPS reports... ASAP...";
    sleep(2);
    yield 30 => "Ah, ah, I almost forgot... I'm also going to need you to go ahead and come in on Sunday, too. We, uhhh, lost some people this week and we sorta need to play catch-up. Mmmmmkay? Thaaaaaanks.";
    sleep(2);
    yield 50 => '...So, if you could do that, that would be great...';
    sleep(2);
    yield 60 => 'Excuse me, I believe you have my stapler.';
    sleep(2);
    yield 90 => 'PC LOAD LETTER';
    sleep(2);
    yield 100 => 'Success!';
}
$emitter = new \KIZU514\EventEmitter();
$emitter->emit( loooooooooooooooooooooooongGenerator() );

Key ideas:

  • It’s not necessary to wait until the request finishes, PHP can emit event-stream responses (SSE) back to the web browser while it is working on something.
  • PHP Generators are a relatively simple refactoring hack to get those responses back to the browser.
  • sleep() is only meant as an example of a function call that takes a long time to finish, don’t put sleep in your production code, you already knew this, I hope?

TL;DRhttps://gist.github.com/connerbw/a2fa7712efe135d5854f4d32d67ca09f

Phpdbg Is Much Faster Than Xdebug For Code Coverage

I work on a project that uses Travis CI to test and build against three jobs. (PHP 7.0, 7.1, 7.2) I recently ran into a roadblock where Travis would fail with “No output has been received” on the job that did code coverage using Xdebug. A screenshot of the last successful build before the failure:

PHP 7.0.8 runs the tests without Xdebug and PHP 7.1 runs the same tests with Xdebug + Code coverage.

Only one of our three jobs runs code coverage because it’s ridiculously slow. We were disabling coverage on the other jobs so that, in a worse case scenario, we could at least check a hotfix in under 5 minutes.

This system was working fine until a few days ago. The test suite has kept growing and we got to the point where Travis just wouldn’t run code coverage anymore.

We tried every trick in the book and the best I could get it down to, minus the other stuff required to build (git clone, install dependencies, yarn build, phpcs, …) was 42 minutes.

Hooray the build didn’t crash and it only took 42 minutes to check!

Enter Phpdbg

PHP 7.0.8 runs the tests without Phpdbg and PHP 7.1 runs the same tests with Phpdbg + Code coverage.
WAT?!

Down from 42 minutes to 4!

Source:

https://docs.travis-ci.com/user/speeding-up-the-build/#PHP-optimizations

Caveats:

  • Travis CI docs say to do phpenv config-rm xdebug.ini but this crashes the build on environments where Xdebug is not installed. Fixed by conditionally checking if Xdebug is on:
before_script:
- if php -v | grep -q 'Xdebug'; then phpenv config-rm xdebug.ini; fi
  • phpdbg didn’t work in PHP 7.2: /home/travis/.travis/job_stages: line 57: 8492 Segmentation fault (core dumped). Fixed by running phpdbg only on PHP 7.1:
script:
- if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.1" ]]; then phpdbg -qrr -d memory_limit=-1 vendor/bin/phpunit --configuration phpunit.xml --coverage-clover coverage.xml; fi
- if [[ ${TRAVIS_PHP_VERSION:0:3} != "7.1" ]]; then vendor/bin/phpunit --configuration phpunit.xml; fi
  • A test that uses exec didn’t work: Unable to fork […] Fixed by only running that test when phpdbg is disabled. As we have three jobs (PHP 7.0, 7.1, and 7.2) and two of them don’t run code coverage (PHP 7.0 and 7.2) the test itself still runs, we just lose a bit of code coverage when running against PHP 7.1:

 $runtime = new \SebastianBergmann\Environment\Runtime();
 if ( ! $runtime->isPHPDBG() ) {
  // // TODO: exec(): Unable to fork error when running phpdbg()
 }
  • “The output is not the same…” Fixed by not giving a shit? Is there really 42 minutes of justifiable output difference between Xdebug and Phpdbg? Our codecov metrics remained the same. Our Travis build stopped failing.

I <? PHP

PHP7 Learning Resources

PHP made a lot of mistakes. Many are still in the language today. However PHP is 23 years old now. PHP7, released December 2015 is a huge improvement over older versions. PHP7 outperforms Ruby, Python, and many others. If you haven’t looked at it, do so. “PHP sucks” in 2018 is mostly FUD now.

For PHP Developers:

For C Developers:

O’Doyle Rules!

Namespaces and Objects

Markdown editing for WordPress.

Parsedown Party is a new WordPress Plugin I wrote with PHP namespaces, Composer support, 90% code coverage, and Travis-CI for automatic testing and releases.

While writing this Plugin, I settled on some WordPress coding conventions that I’d like to talk about.

Namespaces

One of the weirder things I read as a WordPress Plugin developer are tips such as “prefix all your function with name_” or “put all your functions in a class and declare them as static.” This advice is to prevent code collisions with the other 53,123+ Plugins available for WordPress. Big number.

On the other hand, Packagist lists 164,796+ PHP Composer packages, does not follow the aforementioned recommendations and has no such code collisions. Why? Because they use namespaces.

PHP Namespaces have been available since 2009. Other programming languages such as Java or Perl have had them for longer. WordPress Core doesn’t encourage PHP Namespaces but the syntax is fully supported.

If my class is a bunch of static methods and nothing else then I am doing it wrong. I should instead write a library of functions. If I’m afraid of function name collisions then I should use Namespaces because they solve that exact problem and they work fine with WordPress.

add_action( 'login_head', '\Kizu514\Coolname\this_works_fine' );
add_filter( 'login_headerurl', '\Kizu514\Coolname\welcome_to_php' );

Objects

Another odd thing I see while browsing OPP (other people’s plugins) are objects with “mostly” static methods.  There is some object-oriented approach to the design but most of the methods are static because they hook into actions and filters.

I think that an object in a WordPress Plugin should have, at most, two static methods. Every other method in a class should be public, protected or private.

There are very few cases where static methods are necessary. Most of the time public methods can be used. At worst, namespaced functions can invoke objects as needed and these concerns should be separate.

Here’s an example of what I mean (the code must also have PHPDoc comments  but for brevity, I have omitted these):

namespace Kizu514\CoolClass;

class Plugin {
	private static $instance = null;
	private $obj1;
	private $obj2;
	private $obj3;

	static public function init() {
		if ( is_null( self::$instance ) ) {
			$obj1 = new \Obj1();
			$obj2 = new \Obj2();
			$obj3 = new \Obj4( new \Obj3() );
			self::$instance = new self( $obj1, $obj2, $obj3 );
			self::hooks( self::$instance );
		}
		return self::$instance;
	}

	static public function hooks( Plugin $obj ) {
		add_action( 'save_post', [ $obj, 'doSomething' ] );
		add_filter( 'the_content', [ $obj, 'doSomethingElse' ] );
	}

	public function __construct( $obj1, $obj2, $obj3 ) {
		$this->obj1 = $obj1;
		$this->obj2 = $obj2;
		$this->obj3 = $obj3;
	}

	public function doSomething( $post_id ) { /**/ }

	public function doSomethingElse( $content) { /**/ }
}

Some time passes….

add_action( 'after_setup_theme', [ '\Kizu514\CoolClass\Plugin', 'init' ] );

The first static method, init, is hooked into one of many available launch points like plugins_loadedafter_setup_theme, or init. Because these actions happen before everything else a second static method, hooks, can do the rest of the work from inside the object itself.

Admittedly, even if I am down to just two static methods, they are gross. Glaring problems include “Singletons are evil” and not using Inversion of Control.

I’m very happy with this code style and I will be using it moving forward.

Install Phalcon PHP On Windows 10

A common misconception is that you need a web server like IIS, Apache, or Nginx to get started with PHP7 development. In fact, PHP7 has its own built in web server that you can invoke at the command prompt. Many modern PHP frameworks support this, such as Phalcon PHP.

Prerequisite:
PHP7 and Composer on Windows 10

Installing Phalcon

Download the latest DLL file from GitHub and unzip to C:\PHP7\ext

Make sure the file you download matches your installed PHP version. For me, it was the non-thread safe version, so I picked a file that ended with _nts.zip

Add extension=php_phalcon.dll to your php.ini file

Drop to the command prompt and do:

php -v

If you get an error like “PHP Startup: Unable to load dynamic library…” that just means you installed the wrong version:

Whoops, I need x86 not x64.

Don’t panic, download another version, overwrite the DLL, and try again:

No news is good news.

(Source)

Installing Phalcon Dev Tools

In a new folder create a bare bones composer.json file with only this in it:

{
    "require-dev": {
        "phalcon/devtools": "~3.2"
    }
}

At the command prompt, cd to your folder, and do:

composer install

Create a simple project named my_project:

vendor\bin\phalcon.php.bat project my_project simple . 

Launch the built in web server:

cd my_project
..\vendor\bin\phalcon.php.bat serve

(Source)

Switchers Guide To Windows 10 (For Web Developers)

I’m an OS X user from 2003 until 2011 and a Ubuntu user from 2012 until Windows 10.

The freedom to make irrational decisions.

Bash

Here are two (of many) options:

Install Git for Windows. Git for Windows provides a BASH emulator.

Install "Bash On Ubuntu on Windows"

Apt / Homebrew

Checkout Chocolatey or Scoop.

ALT + `

Install EasySwitch. Upvote this feature request in Feedback Hub.

Privacy

Install Shut Up 10. Check out BleachBit.

Minimum Viable Toolset

Start Menu Shortcuts go in:

C:ProgramDataMicrosoftWindowsStart MenuPrograms*

Where is the `/etc/hosts` file?

c:WINDOWSsystem32driversetchosts

Configure Notepad++ as Git editor

git config --global core.editor "'C:/Program Files (x86)/Notepad++/notepad++.exe' -multiInst -notabbar -nosession -noPlugin"

(source)

Are you a switcher too? Share your tips in the key party comments below.

Your Password Hashing Algorithm Is Bad And You Should Feel Bad

No

$pw = md5('password');
$pw = md5('salt' . 'password');
$pw = md5('complicated_salt' . 'password');
$pw = md5('complicated_salt' . strrev('password')); // Don't be clever.

Where md5() = sha1(), base64_encode(), etc.

This type of password hashing is still widespread and susceptible to rainbow table attacks.

Yes

$pw = password_hash('password', PASSWORD_DEFAULT);

(Source)

Uses bcrypt, this particular implementation auto-magically hardens itself over time.

How to use

You are responsible for new \Pdo(), $condition, maybe asking the user to make their 'password' not suck. Read the snippet and reason about it. Don’t just copy/paste, it won’t work.


// Save user password into database

$pw = password_hash($_REQUEST['pw'], PASSWORD_DEFAULT);
$stmt = $pdo->prepare('UPDATE users SET password=? WHERE condition=?');
$stmt->execute([$pw, $condition]);

// Verify user login

$stmt = $pdo->prepare('SELECT password FROM users WHERE condition=?'); 
$stmt->execute([$condition]); 
$row = $stmt->fetch();

if (password_verify($_REQUEST['pw'], $row['password'])) {
  // Check if PHP has improved password security for us
  if (password_needs_rehash($row['password'], PASSWORD_DEFAULT)) {
    // Fix password for next time
    $pw = password_hash($_REQUEST['pw'], PASSWORD_DEFAULT);
    $stmt = $pdo->prepare('UPDATE users SET password=? WHERE condition=?');
    $stmt->execute([$pw, $condition]);
  }
  // Log in
} else {
  // Invalid password
}

Keep on shaking that salt shaker.

WordPress REST API Quickstart

The WordPress REST API has been available since 4.7.  It’s robust, consistent, and nifty to work with. Why? Backend and mobile developers can use other frameworks while still keeping WordPress around for their customers. Frontend developers can build sites using JavaScript without having to touch PHP. Up is down, left is right, dogs and cats living together… Let’s get started!

Recommended Tools

Troubleshooting

  • JSON Formatter: CTRL/CMD+Click a triangle to collapse/expand nodes at the same level.
  • YARC: When testing with Basic Authentication, make sure you are logged out of WordPress first.

Getting Started

WP API supports all HTTP Methods: GET, POST, PUT, PATCH, DELETE, OPTIONS.

WP API respects permissions but the developer must setup authentication separately.

Schema

WP API is self-documenting. Send an OPTIONS request to any endpoint and get back JSON Schema compatible info on how to use it:

OPTIONS

To get the entire API schema in a single query, add context=help at the index. (Ie. http://site/wp-json?context=help )

Features

WP API items have a _links node based on HAL (Hypertext Application Language):

_links

To reduce the number of HTTP requests use the _embed parameter to tell the API that the response should include embeddable resources.

_embed

WP API exposes pagination info in the response header.

Pagination

PHP to JSON

WP API renders JSON in a generic way that does not match the DB columns. Keep calm and RTFM:

if ( ! empty( $schema['properties']['author'] ) ) {
    $data['author'] = (int) $post->post_author;
}
if ( ! empty( $schema['properties']['slug'] ) ) {
    $data['slug'] = $post->post_name;
}
if ( ! empty( $schema['properties']['content'] ) ) {
    $data['content'] = array(
        'rendered'  => post_password_required( $post ) ? '' : apply_filters( 'the_content', $post->post_content ),
        'protected' => (bool) $post->post_password,
    );
}
{
  "author": 1,
  "slug": "chapter-1",
  "content": {
    "rendered": "<p>Hello World!</p>",
    "protected": false
  }
}

Example

Setup the Basic Authentication Plugin on your development environment.

In YARC, add your credentials:

YARC CredentialsSend an OPTIONS request to a post endpoint. The response will contain, among other information:

 {
   "methods": [
     "POST",
     "PUT",
     "PATCH"
   ],
   "title": {
     "required": false,
     "description": "The title for the object.",
     "type": "object"
   },

Translation: The API client can send a PUT request to change the title.

In YARC, send a PUT request with the following JSON to the endpoint:

{ 
  "title": "My changed title!" 
}

Congratulations, you just changed the title. 

…cue the sound of a thousand keyboards furiously hacking.

Write Unit Tests For Your WordPress Plugin Using PhpStorm Code Completion

Git clone the WordPress develop repository somewhere on your hard drive:

git clone git@github.com:WordPress/wordpress-develop.git

Comment out the class_alias() functions in phpunit6-compat.php because these break PhpStorm code completion. (These files aren’t actually used by the testing framework, we only downloaded them so they could be included in the Project Configuration’s Include Path. Leave this line uncommented if you are using PhpUnit 6+:)

class_alias( 'PHPUnitFrameworkTestCase', 'PHPUnit_Framework_TestCase' );

PhpStorm -> Settings -> Languages & Frameworks -> PHP: Add wordpress-develop/tests/phpunit/includes to your Include Path.

Use WP-CLI to generate the tests scaffolding.

Write tests that extend WP_UnitTestCase. Look at the code in wordpress-develop/tests/phpunit/tests for examples.

Autocomplete!