How to Make Modern PHP More Modern? With Preprocessing!
Free JavaScript Book!
Write powerful, clean and maintainable JavaScript.
RRP $11.95
Let’s have a bit of fun. A while ago, I experimented with PHP macros, adding Python range syntax. Then, the talented SaraMG mentioned an RFC, and LordKabelo suggested instead adding C#-style getters and setters to PHP.
Aware of how painfully slow it can be for an outsider to suggest and implement a new language feature, I took to my editor…
The code for this tutorial can be found on Github. It’s been tested with PHP
^7.1, and the generated code should run on PHP
^5.6|^7.0.
How Do Macros Work Again?
It’s been a while (and perhaps you’ve never heard of them) since I’ve talked about macros. To refresh your memory, they take code that looks like this:
macro { →(···expression) } >> { ··stringify(···expression) } macro { T_VARIABLE·A[ ···range ] } >> { eval( '$list = ' . →(T_VARIABLE·A) . ';' . '$lower = ' . explode('..', →(···range))[0] . ';' . '$upper = ' . explode('..', →(···range))[1] . ';' . 'return array_slice($list, $lower, $upper - $lower);' ) }
…and turn custom PHP syntax, like this:
$few = many[1..3];
…into valid PHP syntax, like this:
$few = eval( '$list = ' . '$many' . ';'. '$lower = ' . explode('..', '1..3')[0] . ';' . '$upper = ' . explode('..', '1..3')[1] . ';' . 'return array_slice($list, $lower, $upper - $lower);' );
If you’d like to see how this works, head over to the the post I wrote about it.
The trick is to understand how a parser tokenizes a string of code, build a macro pattern, and then apply that pattern recursively to the new syntax.
The macro library isn’t well documented, though. It’s difficult to know exactly what the pattern needs to look like, or what valid syntax to generate in the end. Every new application begs for a tutorial like this to be written, before others can understand what’s really going on.
Building A Base
So, let’s look at the application at hand. We’d like to add getter and setter syntax, resembling that of C#, to PHP. Before we can do that, we need to have a good base of code to work from. Perhaps something in the form of a trait that we can add to classes needing this new functionality.
We need to implement code that will inspect a class definition and create these dynamic getter and setter methods for each special property or comment it sees.
Perhaps we can start by defining a special method name format, and magic
__get and
__set methods:
namespace App; trait AccessorTrait { /** * @inheritdoc * * @param string $property * @param mixed $value */ public function __get($property) { if (method_exists($this, "__get_{$property}")) { return $this->{"__get_{$property}"}(); } } /** * @inheritdoc * * @param string $property * @param mixed $value */ public function __set($property, $value) { if (method_exists($this, "__set_{$property}")) { return $this->{"__set_{$property}"}($value); } } }
Each method starting with the name
__get_ and
__set_ needs to be connected to an as-yet undefined property. We can imagine this syntax:
namespace App; class Sprocket { private $type { get { return $this->type; } set { $this->type = strtoupper($value); } }; }
…being converted to something very much like:
namespace App; class Sprocket { use AccessorTrait; private $type; private function __get_type() { return $this->type; } private function __set_type($value) { $this->type = strtoupper($value); } }
Defining Macros
Defining the required macros is the hardest part of any of this. Given the lack of documentation (and widespread use), and with only a handful of helpful exception messages, it’s mostly a lot of trial and error.
I spent a few hours coming up with the following patterns:
macro ·unsafe { ·ns()·class { ···body } } >> { ·class { use AccessorTrait; ···body } } macro ·unsafe { private T_VARIABLE·var { get { ···getter } set { ···setter } }; } >> { private T_VARIABLE·var; private function ··concat(__get_ ··unvar(T_VARIABLE·var))() { ···getter } private function ··concat(__set_ ··unvar(T_VARIABLE·var))($value) { ···setter } }
Ok, let’s look at what these two macros are doing:
- We begin by matching
class MyClass { ... }, and inserting the
AccessorTraitwe built previously. This provides the
__getand
__setimplementations, which links
__get_barto
print $class->baretc.
- We match the accessor block syntax, and replace it with an ordinary property definition, followed by a couple of individual method definitions. We can wrap the exact contents of the
get { ... }and
set { ... }blocks within these functions.
At first, when you run this code, you’ll get an error. That’s because the
··unvar function isn’t a standard part of the macro processor. It’s something I had to add, to convert from
$type to
type:
namespace Yay\DSL\Expanders; use Yay\Token; use Yay\TokenStream; function unvar(TokenStream $ts) : TokenStream { $str = str_replace('$', '', (string) $ts); return TokenStream::fromSequence( new Token( T_CONSTANT_ENCAPSED_STRING, $str ) ) ; }
I was able to copy (almost exactly) the
··stringify expander, which is included in the macro parser. You don’t need to understand much about the internals of Yay in order to see what this is doing. Casting a
TokenStream to a string (in this context) means you’re getting the string value of whatever token is currently referenced – in this case it’s
··unvar(T_VARIABLE·var) – and perform string manipulations on it.
(string) $tsbecomes
"$type", as opposed to
"T_VARIABLE·var".
Usually, these macros are applied when they are placed inside the script they are meant to apply to. In other words, we could create a script resembling:
<?php macro ·unsafe { ... } >> { ... } macro ·unsafe { ... } >> { ... } namespace App; trait AccessorTrait { ... } class Sprocket { private $type { get { return $this->type; } set { $this->type = strtoupper($value); } }; }
… then we could run it using a command like:
vendor/bin/yay src/Sprocket.pre >> src/Sprocket.php
Finally, we could use this code (with some Composer PSR-4 autoloading), using:
require __DIR__ . "/vendor/autoload.php"; $sprocket = new App\Sprocket(); $sprocket->type = "acme sprocket"; print $sprocket->type; // Acme Sprocket
Automating Conversion
As a manual process, this sucks. Who wants to run that bash command every time they change
src/Sprocket.pre? Fortunately, we can automate this!
The first step is to define a custom autoloader:
spl_autoload_register(function($class) { $definitions = require __DIR__ . "/vendor/composer/autoload_psr4.php"; foreach ($definitions as $prefix => $paths) { $prefixLength = strlen($prefix); if (strncmp($prefix, $class, $prefixLength) !== 0) { continue; } $relativeClass = substr($class, $prefixLength); foreach ($paths as $path) { $php = $path . "/" . str_replace("\\", "/", $relativeClass) . ".php"; $pre = $path . "/" . str_replace("\\", "/", $relativeClass) . ".pre"; $relative = ltrim(str_replace(__DIR__, "", $pre), DIRECTORY_SEPARATOR); $macros = __DIR__ . "/macros.pre"; if (file_exists($pre)) { // ... convert and load file } } } }, false, true);
You can save this file as
autoload.php, and use
filesautoloading to include it through Composer’s autoloader, as explained in the documentation.
The first part of this definition comes straight out of the example implementation of the PSR-4 specification. We fetch Composer’s PSR-4 definitions file, and for each prefix, we check whether it matches the class currently being loaded.
If it matches, we check each potential path, until we find a
file.pre, in which our custom syntax is defined. Then we get the contents of a
macros.pre file (in the project base directory), and create an interim file – using
macros.pre contents + the matched file’s contents. That means the macros are available to the file we pass to Yay. Once Yay has compiled
file.pre.interim →
file.php, we delete
file.pre.interim.
The code for that process is:
if (file_exists($php)) { unlink($php); } file_put_contents( "{$pre}.interim", str_replace( "<?php", file_get_contents($macros), file_get_contents($pre) ) ); exec("vendor/bin/yay {$pre}.interim >> {$php}"); $comment = " # This file is generated, changes you make will be lost. # Make your changes in {$relative} instead. "; file_put_contents( $php, str_replace( "<?php", "<?php\n{$comment}", file_get_contents($php) ) ); unlink("{$pre}.interim"); require_once $php;
Notice those two booleans at the end of the call to
spl_autoload_register. The first is whether or not this autoloader should throw exceptions for loading errors. The second is whether this autoloader should be prepended to the stack. This puts it before Composer’s autoloaders, which means we can convert
file.pre before Composer tries to load
file.php!
Creating A Plugin Framework
This automation is great, but it’s wasted if one has to repeat it for every project. What if we could just
composer require a dependency (for a new language feature), and it would just work? Let’s do that…
First up, we need to create a new repo, containing the following files:
composer.json→ autoload the following files
functions.php→ create macro path functions (to other libraries can add their own macro files dynamically)
expanders.php→ create expander functions, like
··unvar
autoload.php→ augment Composer’s autoloader, loading each other library’s macro files into each compiled
.prefile
{ "name": "pre/plugin", "require": { "php": "^7.0", "yay/yay": "dev-master" }, "autoload": { "files": [ "functions.php", "expanders.php", "autoload.php" ] }, "minimum-stability": "dev", "prefer-stable": true }
This is from
composer.json
<?php namespace Pre; define("GLOBAL_KEY", "PRE_MACRO_PATHS"); /** * Creates the list of macros, if it is undefined. */ function initMacroPaths() { if (!isset($GLOBALS[GLOBAL_KEY])) { $GLOBALS[GLOBAL_KEY] = []; } } /** * Adds a path to the list of macro files. * * @param string $path */ function addMacroPath($path) { initMacroPaths(); array_push($GLOBALS[GLOBAL_KEY], $path); } /** * Removes a path to the list of macro files. * * @param string $path */ function removeMacroPath($path) { initMacroPaths(); $GLOBALS[GLOBAL_KEY] = array_filter( $GLOBALS[GLOBAL_KEY], function($next) use ($path) { return $next !== $path; } ); } /** * Gets all macro file paths. * * @return array */ function getMacroPaths() { initMacroPaths(); return $GLOBALS[GLOBAL_KEY]; }
This is from
functions.php
You may be cringing at the thought of using
$GLOBALS as a store for the macro file paths. It’s unimportant, as we could store these paths in any number of other ways. This is just the simplest approach to demonstrate the pattern.
<?php namespace Yay\DSL\Expanders; use Yay\Token; use Yay\TokenStream; function unvar(TokenStream $ts) : TokenStream { $str = str_replace('$', '', (string) $ts); return TokenStream::fromSequence( new Token( T_CONSTANT_ENCAPSED_STRING, $str ) ) ; }
This is from
expanders.php
<?php namespace Pre; if (file_exists(__DIR__ . "/../../autoload.php")) { define("BASE_DIR", realpath(__DIR__ . "/../../../")); } spl_autoload_register(function($class) { $definitions = require BASE_DIR . "/vendor/composer/autoload_psr4.php"; foreach ($definitions as $prefix => $paths) { // ...check $prefixLength foreach ($paths as $path) { // ...create $php and $pre $relative = ltrim(str_replace(BASE_DIR, "", $pre), DIRECTORY_SEPARATOR); $macros = BASE_DIR . "/macros.pre"; if (file_exists($pre)) { // ...remove existing PHP file foreach (getMacroPaths() as $macroPath) { file_put_contents( "{$pre}.interim", str_replace( "<?php", file_get_contents($macroPath), file_get_contents($pre) ) ); } // ...write and include the PHP file } } } }, false, true);
This is from
autoload.php
Now, additional macro plugins can use these functions to hook their own code into the system…
Creating A New Language Feature
With the plugin code built, we can refactor our class accessors to be a stand-alone, automatically applied feature. We need to create a few more files to make this happen:
composer.json→ needs to require the base plugin repository and autoload the following files
macros.pre→ macro code for this plugin
functions.php→ place to hook the accessor macros into the base plugin system
src/AccessorsTrait.php→ largely unchanged from before
{ "name": "pre/class-accessors", "require": { "php": "^7.0", "pre/plugin": "dev-master" }, "autoload": { "files": [ "functions.php" ], "psr-4": { "Pre\\": "src" } }, "minimum-stability": "dev", "prefer-stable": true }
This is from
composer.json
namespace Pre; addMacroPath(__DIR__ . "/macros.pre");
This is from
functions.php
macro ·unsafe { ·ns()·class { ···body } } >> { ·class { use \Pre\AccessorsTrait; ···body } } macro ·unsafe { private T_VARIABLE·variable { get { ···getter } set { ···setter } }; } >> { // ... } macro ·unsafe { private T_VARIABLE·variable { set { ···setter } get { ···getter } }; } >> { // ... } macro ·unsafe { private T_VARIABLE·variable { set { ···setter } }; } >> { // ... } macro ·unsafe { private T_VARIABLE·variable { get { ···getter } }; } >> { // ... }
This is from
macros.pre
This macro file is a little more verbose compared to the previous version. There’s probably a more elegant way of handling all the arrangements the accessors could be defined in, but I haven’t found it yet.
Putting it all together
Now that everything is so nicely packaged, it’s rather straightforward to use the new language feature. Take a look at this quick demonstration!
You can find these plugin repositories on Github:
Conclusion
As with all things, this can be abused. Macros are no exception. This code is definitely not production-ready, though it is conceptually cool.
Please don’t be that person who comments about how bad you think the use of this code would be. I’m not actually recommending you use this code, in this form.
Having said that, perhaps you think it’s a cool idea. Can you think of other language features you’d like PHP to get? Maybe you can use the class accessors repository as an example to get you started. Maybe you want to use the plugin repository to automate things, to the point where you can see if your idea has any teeth.
Let us know how it goes in the comments.
Since writing this tutorial, I’ve been frantically working on the underlying libraries. So much that there’s now a site where this code is hosted and demonstrated:. It’s still in an alpha state, but it showcases all the code I’ve spoken about here and then some. There’s also a handy REPL, in case you’d like to try any of the macros.
Get practical advice to start your career in programming!
Master complex transitions, transformations and animations in CSS!