Bundling and Distributing Complex ES6 Libraries in an ES5 World09 Jul 2015
We previously announced pileup.js,
a genome viewer designed to be embedded in web applications.
Since then, we have restructured the code base and created a cleaner distribution.
These both make
pileup.js easier to integrate into applications.
We faced several challenges along the way
that are likely to be encountered by anyone building and releasing modern web libraries,
so this post will detail the workflow we’ve come up with to solve them.
Bundled File Distribution
pileup.js to your application is as simple as including our bundled distribution file,
which can be downloaded from NPM, Github, or Browserify’s CDN.
That file defines a global function
require, which you can use to create new
in particular, you don’t have to worry about managing/including its dependencies.
To make this possible, our code goes through two transformations:
pileup.js code is written in
EcmaScript 6 (ES6) and JSX,
and annotated with Flow types.
These syntaxes are great and tend to take a lot of burden off the developers,
but they also come with a necessary code transformation step,
commonly referred to as transpiling.
What this means in practice is that people can’t use the original code directly,
since we still live in an EcmaScript 5-dominated world.
For example, if you were to check out our code from GitHub,
and try to use it “as is”, this is what you will get:
SyntaxError is expected,
import type syntax is part of Flow and not ES5.
To make our code ES5-compatible,
we have to run it through JSTransform first:
This transpiles all the code from the
src folder into another one.
Once the transpiling is completed,
you can now start using the transformed code under
As such, it does not make sense to distribute the untransformed code via NPM,
nor does it make sense to keep track of the transformed code via GitHub;
therefore, we ignore the dist folder via our
and list only the files under
dist for release in our
Browserify: bundling everything into a single file
Even if we distribute the plain ES5 code,
we still need to make things easy for people who would like to use
pileup.js in their web applications.
And by default, it is not possible to use a library structured as a node module
as they don’t understand
and can’t handle a module’s multi-file structure.
Browserify solves this by converting a node module into a form that can be used for web applications.
It can bundle a node library and all of its dependencies into a single file by wrapping it up in an anonymous closure
and optionally exporting a function (
require) to access its contents.
When run on a project folder,
browserify takes hints from your package description (
and starts the process by reading the module specified in the
It then traverses all the code that is used by this file and its dependencies.
These files get bundled in a closure
and structured in a way to make dependencies work seamlessly within that single bundled file.
This means that if you open the bundled pileup.js and examine its contents,
you will also see the code for our dependencies, such as
After transpiling our code and before publishing it on NPM,
browserify as a task and bundle the code up:
- calls the command line
browserifyutility on the current folder (
- allows access to the main browser module via
pileupalias, so that you can
- creates a map from the bundled file into individual source files for easier debugging;
- saves the bundled file as
And since this new browserified file is under
it also is distributed via NPM so that people don’t have to bother with all these steps
and can use this file directly from the repository:
UglifyJS: optional transformation to compress the code for production
Minification dramatically reduces the size of our bundled file. Here is the proof:
You can see that the bundled file,
pileup.js, is gigantic by web standards
while the minified version,
pileup.min.js, is much more reasonable.
To accomplish this,
we need to transform our code once more, this time through UglifyJS:
which turns the code into an ugly yet functionally-equivalent form by renaming variables and reducing whitespace in the file. This minified file is better for distribution, but it is terrible for development purposes since it is almost impossible to debug or read. Therefore, our tests and playground examples all make use of the bundled version instead of the minified one; but we encourage the use of the minified version for production purposes.
Although the current structure of
pileup.js is now relatively simple,
we had to try different configurations with different setups before settling on this particular solution.
Before this, we were running browserify
and transforming the code at the same time using
Although the final output was the same,
we hit multiple issues during tests/flow-checks
and reducing the size of other web applications that make use of
Specifically, the former was due to the difference between the way watchify and flow were parsing the code, which are both required for our test procedures.
When transforming the code simultaneously with
we were asking it to alias the main module as
pileup to be able to
watchify was trying to bundle the main code together with the tests,
the relative path the tests files were using to
require the main module was causing trouble,
therefore breaking many of our tests.
If we started using the
pileup alias instead of the relative paths to require the module,
flow was complaining about missing modules.
We tried to work around this problem,
but it was a hack.
The latter was mainly because we were only distributing the bundled file.
This meant that any other web application that was depending on our library had to deal with our massive distribution file.
To better understand this problem,
imagine that you are working on an application that depends on
d3 is also a dependency for
it makes sense to not bundle two
d3s into a single file when transforming the code through
pileup.js is already bundled together with its own dependencies,
d3 has to be bundled into the application once again,
causing redundancy in the production code and considerably inflating it.
Due to these problems,
we decided to uncouple the
and start distributing the intermediate ES5 code to allow developers interact with our CommonJS-compatible module when desired.
Overall, after a few iterations of trial-and-error (and some level of frustration in between),
we are quite happy with the end result of our refactoring to improve the way we distribute
Thanks to these recent changes and the new structure of our module,
our library now plays nicely with online editors, such as JSFiddle:
In case you missed it in the previous blog post,
you can take a look at the public demo of
If you are looking for some example code to get inspiration for your web application,
our examples folder folder is a good place to start
If you bump into any issues using
pileup.js in your application,
please let us know.