Converting WordPress to Web App: Adding a Build Process

Posted on by

Part three of my Converting WordPress to Web App series, as we convert this site from a standard WP website to a cache-enabled, offline-first, performance-optimized, installable Web App, that takes advantage of Build and Deployment processes, and sits safely revisioned in a Version Control repo.

The steps that I plan to take are listed below; each will become a link as I complete and publish that part of the series.

  1. Series Intro
  2. Adding Version Control
  3. Adding a Build Process (this post)
  4. Adding a Deployment Process
  5. Adding Caching and Offline Support
  6. Adding “Add to home screen” functionality

And now, on to…

Adding a Build Process

Be forewarned: This is a lengthy one, digging into a lot of stuff, so get your coffee and make sure you’re in a comfy chair…

While a Build Process can be really powerful, this is where the standard WP set-up starts to quiver… Any long-time WP developer is used to the easy 5-minute installation, manual CSS, JS, and PHP edits, and FTPing changes to the web. This is the “standard” WP process.

But when a Build Process is introduced (and soon a Deployment Process), those manual edits and uploads can cause repo conflicts, and will cause lost changes. For example, if one team member manually edits a CSS file and FTPs it to the server, and another team member triggers a Build Process, a CSS preprocessor might compile a new CSS file, overwriting the first team member’s changes.

The new approach will be to make edits only in the preprocessor file (referred to as the “source” file), then letting the Build Process create the CSS file (referred to as the “distribution” file). (The upcoming Deployment Process will then push that CSS file to the server.)

While this change may sound more cumbersome (and it is, but only initially!), there are way too many benefits from a Build Process to not use one. We just need to change our ways a wee bit. And change is supposed to be a good thing, right?? ;-)

Preparing for a Build Process

Before getting started, a little planning is needed. I mentioned “source” and “distribution” files above. The division is basically “before the Build Process” and “after the Build Process”, or more accurately, “used by the Build Process” and “created by the Build Process. A deeper way to think about it is “doesn’t need to be deployed” and “does need to be deployed”.

Source files consist of any files that will get some work done to them or will only be used to create other files; these are the files that the Build Process will “process”, and thus do not need to be deployed to a server. A common practice is to put such files into a directory named src. The Build Process will then save the resulting files into a distribution directory, commonly named dist. The dist directory is what will be deployed to the server during the upcoming Deployment Process.

This means, to get ready, a little reorganizing is needed: any Less or Sass files, unminified files, individual SVG icons, etc., should go into the src directory, and WP Core, Theme and Plugin files should go into the dist directory.

At this point the repo’s root directory should look something like this abbreviated example:

/dist
  /wp-admin
  /wp-content
  /wp-includes
  .htaccess
  favicon.ico
  ...
  wp-config.php
  ...
/src
  /icons
  /scripts
  /styles
.gitattributes
.gitignore
...

Finally, Building a Build Process!

Although I am already familiar with Grunt, and Grunt is still an awesome, powerful tool, the new kid on the block is Gulp, and there are a lot of reasons to go with Gulp, so that’s what I chose for this project.

There are a ton of things Gulp can do during a Build Process, and every project’s Build Process should be at least slightly unique. Here are the tasks I want to perform during this project’s Build:

  • Concatenate and minify SVG icons.

    As I am not much of a designer, I went hunting for “free SVG social icons” and was floored at how hard it was to find good quality sets that had the six icons I need! Finally, at the end of the second page of my Google Search (I know, who knew there was more than one page of Google Search results??) I came across iconmonstr. All I can say is AWESOME!

    Then I went straight to CSS Tricks to find out how to use SVG icons…

    I found three articles that I recommend reading if this is new to you:

    1. SVG symbol a Good Choice for Icons. Don’t pay too much attention, just skim to get the idea, then move on to…
    2. SVG use with External Reference, Take 2. Basically, you can use an external SVG sprite, and it’s totally awesome, except it doesn’t work in any IE, and is just now being fixed in Edge. There are a few options listed for implementing the actual icons, and I am going with using a PHP @include to add the svg block at the top of each page, because I only have a few icons, and my icons are simple, single colors. You may want another option if you have more icons or your icons are more complex.
    3. Icon System with SVG Sprites. Putting all that together into an “icon system” via a Build Process. (And since this article is focused on Grunt, I went hunting and found gulp-svgstore, which does the same thing, and even references this article.)

    In addition to concatenating the individual icons into a single sprite file, I also wanted to minify them to remove all the garbage typically found in exported SVG files. I chose to go with gulp-svgmin, mainly because it uses SVGO, which has impressed people such as , and who is going to argue with Jake?

    gulp-svgmin also adds an ID to each symbol in the sprite, so the icons can properly use them.

  • Automate prefixing and cleaning CSS. Using mixins within preprocessors are nice, but they require continued maintenance (such as when a prefix is no longer needed), so I chose to automate even that, by using Autoprefixer. While I don’t need a pre-processor, the benefits of this post-processor are way too huge to pass up… Not only does it add prefixes based on my custom browser-matrix, but it also removes the ones I no longer need. Such wow! The above article is all about Grunt, but it works with Gulp too; the “official” Gulp-version appears to be PostCSS/Autoprefixer.
  • Concatenate and/or minify CSS and JS files. I chose gulp-concat for file concatenation, then I chose gulp-minify-css for my CSS minification and gulp-uglify for my JS minification. No fancy reasons, these are just where I ended up.
  • Determine my site’s critical CSS. If you are not yet familiar with this concept, I first read about it from the Google Developers team, with regards to their PageSpeed Module, but you can find a ton of writings about it online. The following are some good inspiration, a couple specifically with regards to WordPress:

    There are two main critical CSS methods you will see used: ‘s Critical and The Filament Group‘s Critical CSS. As I compared the configuration and set-up of both, Addy’s simply looked much cleaner to me, so that’s what I went with.

    These three articles really got me set-up:

    1. To set-up the Gulp Task all I really needed was Critical’s GitHub page. The examples and in-page documentation got nearly everything working (with the caveat regarding the base that I mentioned above). The next step is to put this minified CSS into pages for first-time users.
    2. For this, I started with as he demonstrates how to get this code into WP templates (scroll to the bottom, just before the Wrapping up section).
    3. That said, Ryan’s version does mean that the critical CSS gets pushed into every page, for every download; this is a bit wasteful. To resolve that issue, I borrowed again from Jeremy’s article, where he takes advantage of a browser cookie to determine whether or not the user already has the full CSS file cached.

A couple quick code blocks, because these required some custom work…

Here is my critical CSS Task:

// generate critical CSS
gulp.task( 'styles-critical', function() {

    // prevent Node from balking at self-signed ssl cert
    process.env.NODE_TLS_REJECT_UNAUTHORIZED = 0;

    // run critical css
    critical.generate({
        /* note: cannot use 'base:' or will break remote 'src:' */

        // we want css, not html
        inline: false,

        // css source file
        css: 'src/styles/style.css',

        // css destination file
        dest: 'dist/wp-content/themes/atg/critical-min.css',

        // page to use for picking critical
        src: 'https://aarontgrogg.dreamhosters.com/',

        // make sure the output is minified
        minify: true,

        // pick multiple dimensions for top nav
        dimensions: [{
            height: 500,
            width: 300
        }, {
            height: 600,
            width: 480
        }, {
            height: 800,
            width: 600
        }, {
            height: 940,
            width: 1280
        }, {
            height: 1000,
            width: 1300
        }, {
            height: 1200,
            width: 1800
        }, {
            height: 1200,
            width: 2300
        }]
    });
});

Note the first line inside of the Task’s callback function, process.env.NODE_TLS_REJECT_UNAUTHORIZED = 0;. Node doesn’t like self-signed SSL certificates, which is what I use for my local and stage servers; this tells it to shut-up and do it’s job. Never use this setting on a production Node server, it disables a ton of security features!

And this is what I added to my functions.php:

// utility function to dynamically create cache-buster, based on file's last modified date
// adapted from: http://www.particletree.com/notebook/automatically-version-your-css-and-javascript-files/
if ( !function_exists( 'atg_create_cache_buster' ) ) {
  function atg_create_cache_buster( $url ){
    return filemtime( $url );
  }
}

// utility function to dynamically add cache-buster to file name
// adapted from: http://www.particletree.com/notebook/automatically-version-your-css-and-javascript-files/
if ( !function_exists( 'atg_add_cache_buster' ) ) {
  function atg_add_cache_buster( $url, $buster ){
    $path = pathinfo( $url );
    $ver = '.' . $buster . '.';
    return $path['dirname'] . '/' . str_replace( '.', $ver, $path['basename'] );
  }
}

// add the critical CSS in the <head>
if ( ! function_exists( 'atg_add_css' ) ) :
  function atg_add_css() {

    // name of css file
    $cssfile = '/style-min.css';

    // file path for the css file
    $csspath = get_stylesheet_directory() . $cssfile;

    // get cache-buster
    $cachebuster = (string) atg_create_cache_buster( $csspath );

    // url for the css file
    $cssurl = atg_add_cache_buster( get_stylesheet_directory_uri() . $cssfile, $cachebuster );

    // check if they need the critical CSS
    if ( $_COOKIE['atg-csscached'] == $cachebuster ) {

      // if they have the cookie, then they have the CSS file cached, so simply enqueue it
        wp_enqueue_style( 'atg-style', $cssurl );

    } else {

      // write the critical CSS into the page
      echo '<style>';
        include( get_stylesheet_directory() . '/critical-min.css' );
      echo '</style>'.PHP_EOL;

      // add loadCSS to the page; note the PHP variables mixed in for the cookie setting
      echo "<script>!function(e,t){'use strict';function s(s){function n(){var t,s;for(s=0;s-1&&(t=!0);t?r.media='all':e.setTimeout(n)}var r=t.createElement('link'),i=t.getElementsByTagName('script')[0],a=t.styleSheets;return r.rel='stylesheet',r.href=s,r.media='only x',i.parentNode.insertBefore(r,i),n(),r}s('".$cssurl."'),t.cookie='atg-csscached=".$cachebuster.";expires=\"".date("D, j M Y h:i:s e", strtotime("+1 week"))."\";path=/'}(this,this.document);</script>".PHP_EOL;

      // add the full CSS file inside of a noscript, just in case
      echo '<noscript><link rel="stylesheet" href="'.$cssurl.'"></noscript>'.PHP_EOL;
    }

  } // atg_add_css
endif; // function_exists

The first couple functions, credited in the comments, create a dynamic cache-buster based on the last-modified date of the file itself; sort of a self-defining cache-buster; very clever! Then I pretty much follow Jeremy’s example exactly, with minor modifications to make it all WordPressy. Note the need for get_stylesheet_directory() when dealing with the file’s path (for checking the last-modified date, and for the include), and get_stylesheet_directory_uri() to get the file’s URL (for wp_enqueue_style and loadCSS).

Also, a quick note regarding loadCSS’ cookie expiration date: it is really far in the future (Tue, 19 Jan 2038 03:14:07 GMT, to be exact). But just because that cookie exists does not necessarily mean that the CSS file is still in the browser’s cache; there are a lot of issues (sorry, “challenges”) with browser cache. Users can clear it or it can fill quickly with the MBs of assets that we push through the tubes. And when a browser’s cache is full, older files get kicked out. Though the worst-case scenario is that the browser gets a normal CSS link, this kind of defeats the purpose of this exercise…

So, all that just to say that I’ve shortened my cookie date considerably, to one week from the user’s visit. I feel like this is a safe estimate. In my worst-case scenario, a user might the inline CSS when they do have the CSS file in cache, but for now, I think that makes more sense.

Next, I needed to add a few new lines into my .htaccess file to strip the cache buster and deliver the actual file:

# remove cache-buster
<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteBase / 
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule ^(.+)\.(.+)\.(js|css)$ $1.$3 [L]
</IfModule>

And finally, in my header.php, in place of the link to my CSS file, I add the following function call:

<?php atg_add_css(); ?>

That wraps up the critical CSS stuff!

Additional plugins used

There were also a few additional plugins that I either needed or chose to use:

  • First and foremost was gulp-load-plugins, because it means only maintaining a single list of Gulp plugins (in the package.json file) and it makes referring to the plugins much easier because I know that they are all namespaced inside the plugins object I create, and the plugin names have all been normalized.
  • I didn’t want to be doing all the concatenating and minifying with each and every Build Process, because most files will not change between Builds. So I added gulp-changed (for my CSS and JS files) and gulp-changed-in-place (for the third-party CSS and JS files). Both create hashes of the files they have processed, and check for updates to those files before processing them again. If any of the files have changed since the last Build Process, they will be processed again, but otherwise they will be ignored.
  • gulp-svgmin requires the use of Node.js’s path module in order to create a unique ID for each individual symbol in the concatenated sprite. This one threw me for a few minutes, because after you spend a few hours in Gulpland, it’s easy to think that everything is Gulp, but this one is simply Node.js…
  • Live Reload. gulp-livereload is a developer’s wet dream, when developing locally! Monitoring your local environment files for updates, then automatically triggering things like another Build Process, then a browser refresh, sure saves a lot of tabbing or clicking around when you’re trying to iron something out.

If you’re looking for more info on getting started with Gulp, offers a nice Gulp starter file, with explanations for everything he uses (some of which you may not want, and can easily remove).

Repo Files versus Deployment Files

Another concept to discuss at this point is the difference between files that should, and should not, be included in the repo: Repos should not contain any files that are created by the Build Process.

This might seem odd, because that means a repo will not contain any concatenated and minified CSS and JS files, and it will not contain any optimized and concatenated SVG files…

So after using that fancy Build Process to create all those files, and you’re not going to commit them?

Correct. The reason for this is that repos have a really difficult time determining the difference between such files. Also, since they are going to be rebuilt every time we perform a Build Process, there is really no reason to include them. Our source files are the important ones to revision and share between team members; with them, our created files can easily be created any time.

Additionally, adding Gulp to this project added a slew of new files to the repo’s directory, sitting inside a new root-level node_modules directory. Not only do these constitute a tremendous number of files, but they are also something that can be installed on a computer after the repo is cloned. And, the files that are installed tend to be specific to an operating system. Keeping such files out of the repo means that Windows users can clone the same repo as Mac or Linux users, and npm install will install whatever files their operating system needs, and not any that they don’t.

Updating the .gitignore File

So to keep these unwanted files out of the repo, I added a few lines to the .gitignore file:

node_modules
dist/wp-content/themes/atg/critical-min.css
dist/wp-content/themes/atg/style-min.css
dist/wp-content/themes/atg/icons/icons.svg

Status Thus Far

You can see where I am thus far by checking out the repo on GitHub, though this will be a working repo, meaning it will change as this series progresses. So if you are reading this much later than it was written, it may not look the way that it seems like it should; just keep pushing through and eventually it will!

Next Up

This was by-far the hardest step in this process, because there is a lot to pull together, nearly everything had a bunch of options, and a lot of it was kind of specific to this project…

Next time when we will be Adding a Deployment Process, and there are a few items you may want to have ready if you want to “play along at home”:

  1. A Deployment Process is basically the act of moving files from your locahost or repo to a web server. There are many ways to move files, the most common is easily via FTP, but I will be using a service called DeployBot. It basically monitors your repo, and, when it sees changes (from a push), it can automatically push those changes to your Stage server for testing and review. DeployBot also offers a one-button deployment to your live server when you are ready, and provides a rollback feature, as well as their own Build and ignore options. So, have a read around, and if you’re interested in this option, create an account and browse around, but don’t start “attaching” anything yet…
  2. The deployment process will require access to your repo, so make sure you have those login credentials handy, and if you want your deployment process to be able to automatically deploy to your Stage server, have one setup and have those FTP credentials handy as well.

That should be it!

Until then, happy Web Apping,
Atg

Top⇪

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.