I decided this was something that needs to be written. Not much has been written on this topic and I know we’re not the only ones struggling with 
carrying a legacy application around while drooling over the latest shiny toys that we feel like we’ll never get to use. I don’t know how this is going to turn out. We may fail horribly, we may give up, we may get transferred to another project, but maybe, just maybe, there’s a chance we’ll succeed.
A lot has been written about the greatness that is webpack. However, not a lot has been written about transitioning a 3 year-old, 400k line AngularJS beast of mixed JavaScript and TypeScript bundled by Grunt to webpack.
A few weeks ago we were facing a few problems with our development environment:
- Adding a new “feature” required tweaking no less than 8 config files to satisfy the Grunt monster that runs about 30 tasks to create a build
- Running one of our 8500 unit tests with karma has a 30 second turn-arond time between saving a single file and getting a single test to run (using “fit”. If you ever interview with me, please know this answer).
- Our future, an Angular (4) application, was built recently from scratch to be more modularized, etc, etc. The original plan was that new modules could be created there, and downgraded via ng-upgrade to work within the AngularJS (v1) application. This worked great for a PoC and even a few features here and there, but the behemoth lingered. We needed a better upgrade plan, otherwise we’d never be able to switch off of the legacy code base. More on this later, as it’s very related, but more of an Angular topic than webpack.
At this point I wish to make a few related points:
- This site gets millions of uniques per day. It’s a great testament to the extraordinary team that built the foundation (before my time)
- We have a fairly solid CI environment which runs linters and unit tests multiple times a day on more than 20 branches
- I believe we’re facing a problem that many large applications face… More and more features get added while the infrastructure that supports it, is neglected and starts to bust at the seams. Management/Product doesn’t want to dedicate hours for something that “has always worked”. Someone must constantly push for keeping the environment sane and humming along. It is true, the application reflects the organization that built it.
False start
One of the first problems we identified in our build process is that it is “copy heavy”. We have about 5000 [J|T]S/[LE|C]SS/HTML files in our application. Grunt does a lot of work by copying these files around. Thankfully we have SSD locally, but our poor Jenkins build machines don’t, and it’s noticeable. Even locally though, the amount of processing and I/O is unnecessary and just slow enough to make it frustrating. We were able to make some headway by specifying an outDir for TS compilation and in the new Angular application, and we were able to use a symlinking method that is much faster in the new application. But that didn’t help the here and now.
So, a few of us discussed the idea of moving from Grunt to gulp so that we could take advantage of the streaming capabilities and not write to disk so often. The thinking was that if we just do all the file transformations in memory, it had to be faster. Unfortunately, the migration was a little bumpy, partially due to our inexperience with gulp, and I believe partially because it felt like we were replacing a hammer with a screwdriver, when we really wanted a power drill. This was also done “in our spare time” which normally isn’t a problem, but somehow the momentum was just never there.
gulp, like webpack, is great when you start from scratch, but can be a little mind bending when you are trying to replicate the (often crazy) gymnastics we made Grunt do to create a build. As we transitioned various tasks over, it became apparent that this was just a temporary band-aid and wouldn’t help the future of the application. I also believe it never gained traction because, while we were learning something new, gulp is no longer the new hotness, the transition just felt like grunt work (yeah I did). There is genuine excitement for learning to use webpack, especially as it grows in usage in the industry. I hate to admit it, but level-of-hotness does impact developer enthusiasm.
To give you an idea of what we’re up against, here are the Grunt tasks we run to perform a build: Names have been tweaked to protect the innocent.
‘clean:build’, ‘build-login-app’, ‘build-ng-upgrade’, ‘build-mobile-app’, ‘copy:production’, ‘lang-copy’, ‘es6-bundler’, ‘concat:production’, ‘preprocess:html’, ‘preprocess:app-js’, ‘uglify:app-js’, ‘uglify:vendor’, ‘uglify:app-components’, ‘less:production’, ‘sass’, ‘concat:vendorCss’, ‘postcss’, ‘cssmin’, ‘cssvalidator’, ‘strip_code’, ‘stripJsonComments’, ‘json-merge’, ‘json-minify’, ‘ngtemplates:app.framework.app.templates’, ‘ngtemplates:app.framework.ext.templates’, ‘ngtemplates:app.feature1.templates’, ‘ngtemplates:app.feature2.templates’, ‘ngtemplates:app.admin.templates’, ‘ngtemplates:app.feature3.templates’, ‘ngtemplates:app.feature4.templates’, ‘ngtemplates:app.feature5.templates’, ‘ngtemplates:app.feature6.templates’, ‘ngtemplates:app.feature7.templates’, ‘ngtemplates:app.feature8.templates’, ‘ngtemplates:app.feature9.templates’, ‘ngtemplates:app.settings.templates’, ‘ngtemplates:app.tour.templates’, ‘ngtemplates:app.feature10.templates’, ‘ngtemplates:app.feature11.templates’, ‘ngtemplates:app.feature12.templates’, ‘uglify:templates’, ‘htmlmin’, ‘replace:modules’, ‘replace:bust-cache’, ‘replace:app-js’, ‘replace:ga’, ‘replace:login’, ‘replace:ngUpgradeProd’, ‘clean:css’
Yup. That’s a lot. Each one of these calls several others. It’s crazy town.
What’s the best way to get some webpack going? Come back for part 2 where I’ll show you how we started taming the beast by bringing webpack in to perform very specific tasks.
[INSERT IMAGE OF DEVELOPMENT TEAM SLAYING A DRAGON WITH A webpack SWORD & SHIELD]