Incremental LESS Builds with Gulp

I spent the last few days trying to get an incremental build running for our LESS files using Gulp. I had a really hard time finding a single resource with all the information I needed, and spent a lot of time in trial-and-error, so I thought I’d share the fruits of my labors here.

Background

Our company leverages LESS heavily for building themed, bundled CSS files. We were previously using Web Essentials to have these files get built for us. This was an imperfect solution, but we put up with it because it worked… until it didn’t. Once I realized that Web Essentials wasn’t working, I looked into some other options and determined that GulpJs seemed to be the best option.

Benefits

GulpJs offers some awesome benefits, only a few of which were available with Web Essentials.

1. Having a command-line solution means we no longer have to commit the generated files into our repository, or include them in our Visual Studio project. This not only saves space, but helps us to avoid having to deal with as many merge conflicts. Minified files are particularly bad in version control systems, because most of the code changes you make will occur on a single line of code.
2. Thanks to source-mapping, we can use pre-minified files in both development and production environments, while improving our ability to debug in either environment!
3. Having an incremental build means that we don’t have to rebuild every LESS file every time any LESS file is changed. Instead, we’ll just update any CSS output that depends on that LESS file.
4. GulpJs will keep track of all file changes, even those made outside of Visual Studio. This means, for example, that we can map our LESS files to our working directory in Chrome, make edits directly in Chrome Dev Tools, and have our changes appear almost instantly in the browser.

Working Solution

Here’s the gulpfile.js I finally ended up with. Realizing that much of this is not going to be at all obvious to someone without experience in gulp, I went pretty heavy on the comments:

/*
* This file defines how our static resources get built.
* From the StaticCommon root folder, call "gulp" to compile all generated
* client-side resources, or call "gulp watch" to keep checking source 
* files, and rebuild them whenever they are changed. Call "gulp live" to 
* do both (build and watch).
*/

/* Dependency definitions: in order to avoid forcing everyone to have 
* node/npm installed on their systems, we are including all of the 
* necessary dependencies in the node_modules folder. To install new ones,
* you must install nodejs on your machine, and use the "npm install XXX" 
* command. */
var gulp = require('gulp');
var less = require('gulp-less');
var LessPluginCleanCss = require('less-plugin-clean-css'),
    cleanCss = new LessPluginCleanCss();
var sourcemaps = require('gulp-sourcemaps');
var rename = require('gulp-rename');
var cache = require('gulp-cached');
var progeny = require('gulp-progeny');
var filter = require('gulp-filter');
var plumber = require('gulp-plumber');
var debug = require('gulp-debug');

gulp.task('less', function() {
    return gulp
        // Even though some of our LESS files are just references, and 
        // aren't built, we need to start by looking at all of them because 
        // if any of them change, we may need to rebuild other less files.
        .src(
        ['Content/@(Theme|Areas|Css)/**/*.less'],
        { base: 'Content' })
        // This makes it so that errors are output to the console rather 
        // than silently crashing the app.
        .pipe(plumber({
            errorHandler: function (err) {
                console.log(err);
                // And this makes it so "watch" can continue after an error.
                this.emit('end');
            }
        }))         
        // When running in "watch" mode, the contents of these files will 
        // be kept in an in-memory cache, and after the initial hit, we'll
        // only rebuild when file contents change.
        .pipe(cache('less'))
        // This will build a dependency tree based on any @import 
        // statements found by the given REGEX. If you change one file,
        // we'll rebuild any other files that reference it.
        .pipe(progeny({
            regexp: /^\s*@import\s*(?:\(\w+\)\s*)?['"]([^'"]+)['"]/
        }))
        // Now that we've set up the dependency tree, we can filter out 
        // any files whose
        // file names start with an underscore (_)
        .pipe(filter(['**/*.less', '!**/_*.less']))
        // This will output the name of each LESS file that we're about 
        // to rebuild.
        .pipe(debug({ title: 'LESS' }))
        // This starts capturing the line-numbers as we transform these 
        // files, allowing us to output a source map for each LESS file 
        // in the final stages.
        // Browsers like Chrome can pick up those source maps and show you 
        // the actual LESS source line that a given rule came from, 
        // despite the source file's being transformed and minified.
        .pipe(sourcemaps.init())
        // Run the transformation from LESS to CSS
        .pipe(less({
            // Minify the CSS to get rid of extra space and most CSS
            // comments.
            plugins: [cleanCss]
        }))
        // We need a reliable way to indicate that the file was built
        // with gulp, so we can ignore it in Mercurial commits.
        // Lots of css libraries get distributed as .min.css files, so
        // we don't want to exclude that pattern. Let's try .opt.css 
        // instead.
        .pipe(rename(function(path) {
            path.extname = ".opt.css";
        }))
        // Now that we've captured all of our sourcemap mappings, add
        // the source map comment at the bottom of each minified CSS 
        // file, and output the *.css.map file to the same folder as 
        // the original file.
        .pipe(sourcemaps.write('.'))
        // Write all these generated files back to the Content folder.
        .pipe(gulp.dest('Content'));
});

// Keep an eye on any LESS files, and if they change then invoke the 
// 'less' task.
gulp.task('watch', function() {
    return gulp.watch('Content/@(Theme|Areas|Css)/**/*.less', ['less']);
});

// Build things first, then keep a watch on any changed files.
gulp.task('live', ['less', 'watch']);

// This is the task that's run when you run "gulp" without any arguments.
gulp.task('default', ['less']);

Lessons Learned

Apart from those things mentioned in the comments file above, here are some additional notes that might come in handy.

1. I had to install NodeJs on my local machine so that I could have access to npm, but I didn’t want to force all of our developers to do the same, so I ran npm install ... for all of the dependencies listed at the top of the gulpfile, checked in the node_modules folder to source control, and included a copy of node.exe in our project’s root folder so that the gulp command could be invoked by people without NodeJs installed on their computer. This uses several MB more disk space in our source control, but this is in a subrepository that’s set aside for hosting images, CSS, JavaScript libraries, etc., so it’s not such a bad thing.
2. Because I didn’t want to change things too much, we’re using the pattern established by Web Essentials, to not build any LESS files that start with an underscore. We have several LESS files that are only used as references for other LESS files.
3. Also to avoid changing things too much, I’m having the built files get put into a folder alongside their LESS counterparts. In the future, we may want to change this so that there’s only a single folder we have to “ignore” in Mercurial and Visual Studio.
4. The gulp-progeny plugin needs to have a special regex option to accommodate parenthetical arguments in “@import (reference) _file.less“-type imports.
5. I found it much better to leverage the Clean-CSS plugin for gulp-less rather than running the gulp-minify-css plugin separately. It involves fewer module files, runs more quickly, and (most importantly) preserves source-map information from the LESS transform.
6. Update 2/6/2015: Once this solution made it to our main branch, the build server ran into a problem–it couldn’t load all the npm dependencies because some of them had such deep dependency trees that the file names were too long for a windows system. (Our continuous integration server loads things into a slightly deeper directory than what most of our developers use.) After a bit of research, I found that I could flatten the dependencies out using the flatten-packages module, which not only solves this problem but also reduces the number of files we have to keep in the repository!
7. Update 2/9/2015: Gulp has some issues with its error management that make it so little errors in LESS files will end up silently crashing the gulp run. gulp-plumber was made to fix this problem, and I originally used it according to the directions on its npm readme. However, I later discovered that when you’re using gulp.watch() along with plumber, it needs a little extra configuration. This post almost got it right, but the name of the error handling function option needs to be errorHandler, not handleError. The code above now reflects this fix.

Sacrifices

I added gulp to our main build batch file, but our LESS files take a ridiculous amount of time to rebuild. In order to avoid having to do a full LESS rebuild with every recompile, I opted not to have gulp run when Visual Studio builds our projects. Developers are going to have to remember to run gulp (or have gulp live running) in order to see the correct version of all our styles.

Conclusion

I feel that the benefits of this move far outweigh the downsides. I’m really glad I was able to get the incremental builds working, and now that we’re already incurring the cost of having to run gulp live during development, I plan to incorporate a lot of our other bundling and concatenation efforts into our gulp file, which should simplify our workflows even more, moving forward.