If you missed the first part of this journey, pop over to Part 1.
OK, let’s get rolling here and jump right in…. let’s convert everything to use webpack and reap the benefits!
YES!
Me: “Let’s start easy…. OK, how do I concat then minify then…” Me: 10 minutes later: “Oh shit, there is no then“
OK, so webpack is a completely different mindset to the task-runner-approach I am familiar with like Make/Rake/Grunt/Gulp, heck even Ant. Different doesn’t mean bad (that’s what I tell my kids!), but this isn’t going to be a quick transition.
So where do we start, for real?
We’ve had Grunt tasks in place for years (literally) so we already create minified bundles. These are the main ones:
We decided that the first step should be to attack our 3rd party dependencies - namely anything in node_modules/ that we didn’t create. We still call it “vendor” because we used to use Bower for everything. Then we used a mix of Bower and npm, and finally now we are 100% npm (well some of us moved to yarn, but npm v5 looks sweet… I digress). But the name “vendor” has stuck, and I think that’s OK.
Aside: Bower is a good reminder that all software has a shelf life. Nothing lasts forever in this crazy fast changing world of software. I don’t know what will replace webpack, but we’re being diligent about our inputs and outputs, and we should have an easier time transitioning to the next thing because we are more educated on just what exactly is in our project. Also, in a weird way, the webpack config seems kinda self documenting, which is awesome.
We settled on vendor.min.js because:
- It’s all 3rd party stuff that has hopefully already been webpack’d by someone
- It’s all JavaScript (no TypeScript to worry about… yet)
- We could literally replace the “uglify:vendor” task with a new task named “shell:webpack”
By approaching it this way, we could immediately merge into staging (our production line) without needing to wait until we have the entire project converted. We were fearful this could take weeks/months to fully complete and wanted to show progress… and also feel good about ourselves. 
webpackify the vendors
If you’re like us, you have more 3rd party dependencies than you’d like to admit. It’s cool. We all do it. We’re trying to be more stringent now, but back when we started, we’d pull in anything and everything. Hence we have 37 distinct JavaScript files we rely on. Yup. 37. Stop me if this sounds familiar: modernizr, detectizr, jQuery, lodash, moment, d3, bootstrap, fastclick, angular, angular-cookie, angular-cookies (wha?), angular-animate, angular-ui-router, angular-moment, angulartics, isotope, phoneformat, bootstrap-colorpickers, highcharts, ng-sortable, ocLazyLoad, log4javascript… you get the idea. Honestly at this point it doesn’t matter if it’s 2 or 22, we have a bunch of files from other places that we need to concat & minify into a single file named vendor.min.js. This isn’t rocket science. Let’s see how we’d do it in webpack.
here we go
First, we specify where we want this file to go. That looks easy enough: there is a config option named “output” that can take a “filename”. We know where we want it to end up and what to call it so:
module.exports = {
output: {
filename: ‘build/static/app/vendor/js/vendor.min.js’
},
}
Now, how do we tell webpack to “go grab all these things in node_modules/ and do your thing”? webpack has this concept of an “entry point”. Sweet! I can just point it at the entry point of my application and … DOH! Wait a sec. I can’t do that. If I do that, it’s going to try to do everything, I want to do this systematically and only work on node_modules/ … Oh and did I mention that I don’t have an actual entry point? This is a Angular 1.5 application built with a mix of JavaScript and TypeScript. Shame on me. I thought our entry point might be index.html! Clearly pointing webpack at node_modules/ wasn’t going to work (yeah that was my first hope). Even if somehow it did work, we’d have to filter out anything that we’ve written ourselves (what I call 2nd party libraries), plus all the junk that isn’t used at runtime… yeah it’d be a mess.
create your own entry point
It’s fun working with smart people. This time we leaned on Tor, who has a little webpack experience. He’s getting credit for this whether he figured it out himself, read it somewhere or just took a wild guess (I’m betting on the first one). Say hello to the “entry point file”. Basically what we will do is create a new file that imports all the files that we want in the bundle. In other words, we’re faking it ‘til we’re making it…. and Tor approves. [caption id=”” align=”aligncenter” width=”240”]
Tor to the rescue![/caption] So we build up a file called vendor.ts that looks like this:
import ‘../node_modules/modernizr/modernizr’;
import ‘../node_modules/detectizr/dist/detectizr’;
import ‘../node_modules/jquery/dist/jquery’;
import ‘../node_modules/jquery.easing/jquery.easing’;
import ‘../node_modules/lodash/index’;
import ‘../node_modules/moment/min/moment-with-locales.min’;
import ‘../node_modules/moment-timezone/builds/moment-timezone-with-data’;
import ‘../node_modules/d3/d3’;
import ‘../node_modules/bootstrap/dist/js/bootstrap’;
import ‘../node_modules/fastclick/lib/fastclick’;
…
…
Then we update our webpack.config.js to use it as an entry point:
module.exports = {
output: {
filename: ‘build/static/app/vendor/js/vendor/min.js’
},
entry: {
vendor: ‘./webpack-config/vendor.ts’,
},
}
Let’s do this! At this point, we’ve created our webpack Grunt task, so we can just run “grunt shell:webpack”…
Running “shell:webpack” (shell) task
Hash: 4580ce86f289c11a996b
Version: webpack 2.6.1
Time: 4245ms
Asset Size Chunks Chunk Names
build/static/redbox/vendor/js/vendor.min.js 5.15 MB 0 [emitted] [big] vendor
build/static/redbox/vendor/js/vendor.min.js.map 6.25 MB 0 [emitted] vendor
[1] .//jquery/dist/jquery.js 248 kB {0} [built]/intl-tel-input/lib/libphonenumber/build/utils.js 217 kB {0} [built]
[112] ./
[113] .//isotope/dist/isotope.pkgd.js 103 kB {0} [built]/jquery-form/jquery.form.js 42.5 kB {0} [built]
[114] ./
[115] .//jquery.easing/jquery.easing.js 4.87 kB {0} [built]/lodash/index.js 411 kB {0} [built]
[116] ./
[117] .//log4javascript/log4javascript.js 128 kB {0} [built]/modernizr/modernizr.js 51.4 kB {0} [built]
[118] ./
[119] .//moment-timezone/builds/moment-timezone-with-data.js 193 kB {0} [built]/moment/min/moment-with-locales.min.js 150 kB {0} [built]
[120] ./
[121] .//ng-sortable/dist/ng-sortable.js 38.9 kB {0} [built]/oclazyload/dist/ocLazyLoad.js 57.7 kB {0} [built]
[122] ./
[123] .//phoneformat/phoneformat.min.js 433 kB {0} [built]/tslib/tslib.js 6.47 kB {0} [built]
[124] ./
[127] ./webpack-config/vendor.ts 2.08 kB {0} [built]
+ 113 hidden modules
SUCCESS! We’ve got a vendor.min.js that weighs in at a hefty 5MB (yup, this is before we uglify… patience my friend), it’s in the right place, and now we should be able to start up our application!
make it part of the build
Next we replace the Grunt task “uglify:vendor” with “shell:webpack” and cross our fingers while our dev server starts up. Fire up the browser, and … [caption id=”attachment_731” align=”aligncenter” width=”1062”]
vendor.min.js:108094 Uncaught TypeError: Cannot read property ‘documentElement’ of undefined
index.html:35 Uncaught ReferenceError: angular is not defined[/caption] That’s weird. After a quick double check, angular.js is in fact in our entry point file and, because we didn’t yet uglify, I can actually see it in vendor.min.js. Not sure what to do next, we decided - “OK if they don’t ALL work, let’s try paring them down”. This required putting back the “uglify:vendor” task and commenting out only the inclusion of modernizr & detectizr. Back in vendor.ts, we remove everything but modernizr & detectizr. We also had to modify our setup slightly. We’ll have Grunt continue to build vendor.min.js and we will have webpack build a new file called vendor-webpack.min.js. Finally, we’ll modify index.html to load vendor-webpack.min.js before vendor.min.js. PHEW!
Hash: a76a5e5b36fcdc8a8181
Version: webpack 2.6.1
Time: 187ms
Asset Size Chunks Chunk Names
build/static/redbox/vendor/js/vendor-webpack.min.js 73.7 kB 0 [emitted] vendor
build/static/redbox/vendor/js/vendor-webpack.min.js.map 87.4 kB 0 [emitted] vendor
[0] .//detectizr/dist/detectizr.js 16.8 kB {0} [built]/modernizr/modernizr.js 51.4 kB {0} [built]
[1] ./
[2] ./webpack-config/vendor.ts 2.08 kB {0} [built]
Fire up the browser, and … [caption id=”attachment_732” align=”alignnone” width=”914”]
Uncaught TypeError: Cannot read property ‘detect’ of undefined[/caption] What now? I think this is better, but WTF?
to the google
Having really no clue what’s going on, we took to Google and quickly learned that we need to let webpack know about any libraries that expose global variables. This is done with the expose-loader. OK, this isn’t so bad:
yarn add expose-loader
Since both modernizr and detectizr expose globals, we’ll add both to webpack.config.js:
module: {
rules: [
{
test: require.resolve(‘modernizr’),
use: [{
loader: ‘expose-loader’,
options: ‘Modernizr’
}]
},
{
test: require.resolve(‘detectizr’),
use: [{
loader: ‘expose-loader’,
options: ‘Detectizr’
}]
}
]
}
Fire up the browser, and … [caption id=”attachment_733” align=”aligncenter” width=”706”]
Uncaught TypeError: Cannot read property ‘documentElement’ of undefined[/caption] Hmmm. That looks familiar. It’s the same error we had before, but it seems that window.Modernizr is defined at least. And we don’t see the error about Detectizr.detect. In the devtools console we can see that both Modernizr and Detectizr are globally available. We’ll call this progress!?
fixing the ‘izrs
Again, not really having any clue about where to go next, we check Google and find some links specific to modernizr. As much as I hate the copy-paste-pray method, sometimes it’s all you got.
yarn add imports-loader exports-loader
So we blindly add to our configuration:
module: {
rules: [
{
test: require.resolve(‘modernizr’),
use: [{
loader: ‘expose-loader’,
options: ‘Modernizr’
}, {
loader: ‘imports-loader?this=>window!exports-loader?window.Modernizr’
}]
},
{
test: require.resolve(‘detectizr’),
use: [{
loader: ‘expose-loader’,
options: ‘Detectizr’
}, {
loader: ‘imports-loader?this=>window!exports-loader?window.Detectizr’
}]
}
]
}
Fire up the browser, and … No errors! It works! Commit! Push! Call it a day! Well not so quick. What the heck is this thing doing?
the import/export business
Here we learn that many NodeJS modules are written to expect “this” to be the global space, and browser’s call the global space “window”. So what the imports-loader does is transform “this” to “window” with the “this=>window” syntax. Read more about shimming.
Let’s break this down a bit:
loader: ‘imports-loader?this=>window!exports-loader?window.Modernizr’
|————|||———-|||————|||————–|
| | | | | | |
| here | loader | here |
| come some | separator | come some |
| parameters | | parameters |
| | | |
loader the loader the
name params name params
to the loader to the loader
Aside: This is probably the last time I try to do an ascii diagram by hand. If you are on a phone, I am sorry, it’s kind of a mess. Admittedly it was fun to make. By hand. With vim. Yup. This little shorthand is intimidating at first, but becomes more clear when you use it the second time. Now, if you’re paying attention, it seems we are both “exporting” and “exposing” window.Modernizr. What’s up with that? Do we need both? Well, it certainly works with both, let’s try without expose-loader - Running webpack results in this error:
ERROR in ./~/modernizr/modernizr.js
Module parse failed: /Users/oloreb/dev/redbox-pay/redbox_angular/node_modules/imports-loader/index.js?this=>window!exports-loader?window.Modernizr!/Users/oloreb/dev/redbox-pay/redbox_angular/node_modules/modernizr/modernizr.js Unexpected token (1411:13)
You may need an appropriate loader to handle this file type.
| })(this, this.document);
|
| }.call(window!exports-loader?window.Modernizr));
@ ./webpack-config/vendor.ts 1:0-45
ERROR in ./~/detectizr/dist/detectizr.js
Module parse failed: /Users/oloreb/dev/redbox-pay/redbox_angular/node_modules/imports-loader/index.js?this=>window!exports-loader?window.Detectizr!/Users/oloreb/dev/redbox-pay/redbox_angular/node_modules/detectizr/dist/detectizr.js Unexpected token (507:13)
You may need an appropriate loader to handle this file type.
| }(this, this.navigator, this.document));
|
| }.call(window!exports-loader?window.Detectizr));
@ ./webpack-config/vendor.ts 2:0-50
OK, so it’s needed to even run webpack, but it’s not entirely clear why. Well, let’s try to consolidate these loaders. The shorthand syntax is pretty sweet, so let’s append the expose-loader and just make this a one-liner:
loader: ‘imports-loader?this=>window!exports-loader?window.Modernizr!expose-loader?Modernizr’
Ugh. That didn’t work. We get the same build error as above. According to the documentation, webpack v2 doesn’t support chaining this way, so maybe we shouldn’t head down this road. Let’s back up a bit and clearly define the 3 things we know need to happen:
{
test: require.resolve('modernizr'),
use: \[{
loader: 'expose-loader',
options: 'Modernizr'
}, {
loader: 'imports-loader',
options: 'this=>window'
}, {
loader: 'exports-loader',
options: 'window.Modernizr'
}\]
},Ordering here is important. webpack processes this array bottom->top. We need the exports-loader to run before imports-loader. I still haven’t figured out why we need both the expose-loader and the exports-loader, but we’re back to working again. It seems it would only be necessary if you were using an import or require. Maybe it’s because we’re importing it in vendor.ts? I figured if we exposed it, that would be enough, because it seems all that is needed is for the variable to be available globally. (TODO: Update when I figure out why) Now let’s see if we can pass the “options” as parameters to the loader like in webpack v1
{
test: require.resolve('modernizr'),
use: \[{
loader: 'expose-loader?Modernizr'
}, {
loader: 'imports-loader?this=>window'
}, {
loader: 'exports-loader?window.Modernizr'
}\]
},As expected, this also works. And by the way, I am loving the short-hand!
clean up in aisle 2
If we go back & look at the documentation, webpack v2 will also accept an array of loaders in the “use” option. So we can make this even more pleasing to the eye with:
{
test: require.resolve(‘modernizr’),
use: [
‘expose-loader?Modernizr’,
‘imports-loader?this=>window’,
‘exports-loader?window.Modernizr’
]
},
{
test: require.resolve(‘detectizr’),
use: [
‘expose-loader?Detectizr’,
‘imports-loader?this=>window’,
‘exports-loader?window.Detectizr’
]
}
Now we’re talking!
git commit -am “webpack: vendor-webpack.min.js contains modernizr and detectizr” && git push origin HEAD
one giant leap
We’ll end here for today. We need to latch onto these successes, no matter how seemingly minor. We can’t yet make a pull request to staging until we uglify (that’s coming up), and we don’t want to make another web request to get this 2nd vendor javascript file. So we will hold this off in a separate branch until we complete those 2 items. We learned a bit about loaders and how they can be used to massage libraries to work with webpack. We also learned that we need to be careful with v1 vs v2 configuration options. Lastly, and most importantly, even when you think you are taking a small chunk (vendor), there’s always a smaller chunk you can work on (i.e. just Modernizr & Detectizr). Next time we’ll take a look at the rest of the vendors, and hopefully fully replace the vendor.min.js. If you made it this far, thank you for your time. You are part of a teeny tiny group, we meet on Tuesdays. Follow this journey from the beginning.