Modern Frontend Workflow in a .NET World Part 2: NPM

Originally posted on the DCS Innovation Labs Blog.

All code for this post is available on github.

In part 1, I gave a brief overview of some of the modern frontend practices that I've found useful in my day to day work. In part 2, we're going to take a look at a package manager, NPM. NPM will serve as the basis for all of the tooling we will be going over in the next few posts so it's important to have a basic understanding before we move on.

NPM is the only package manager that I'm going to focus on in this series as it is the go to place for frontend dependencies at the time of writing. Bower, a package manager focused solely on frontend, is still relevant, but seems to be losing popularity to NPM fairly rapidly. I can say pretty confidently that you will be able to find any package you are looking for on NPM.

The setup process for NPM is pretty straightforward so let's jump right in!

1. Download Node

NPM is bundled with Node, so even if you aren't developing applications using Node, you will need it for NPM and pretty much any frontend tool that we will talk about in future blog posts.

The simplest way to get Node is from nodejs.org. I would recommend grabbing the version labeled as "Current" (6.3.1 at the time of writing) as this should provide you with the highest level of compatibility. Once you have downloaded it, just run the executable, go through the install wizard, and you'll be off to the races!

Screenshot of nodejs.org

The slightly more complicated, but slightly better (IMO) way to install Node is with a version manager. This makes it trivial to change versions of node on the fly. If you are simply going to use Node for frontend tooling, you're probably safe just using the official installer. If you intend to go further with Node though, having a version manager can be very helpful. If you are running on a *nix machine, you can grab NVM here. If you are running on Windows, you can grab NVM-Windows here. I'm not going to go through the installation process for these in this post for the sake of brevity, but both projects have excellent documentation that should get you up and running no sweat.

2. Initialize NPM in your project

For the purposes of this tutorial, I'm going to set up a new project, but the same steps apply if you are adding NPM to an existing project. I'm going to be using PowerShell on Windows but the same steps apply on Bash, ZSH, and pretty much any other Bash-like shell.

The first thing I'm going to do is create a new directory for my project, and hop into it.

C:\Projects> mkdir npm_sample_app


    Directory: C:\Projects


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/22/2016   4:24 PM                npm_sample_app


C:\Projects> cd npm_sample_app  
C:\Projects\npm_sample_app>  

Next, I'm going to run npm init. This is a nice wizard that will ask you a few questions to get your project set up.

C:\Projects\npm_sample_app> npm init  
This utility will walk you through creating a package.json file.  
It only covers the most common items, and tries to guess sensible defaults.

See `npm help json` for definitive documentation on these fields  
and exactly what they do.

Use `npm install <pkg> --save` afterwards to install a package and  
save it as a dependency in the package.json file.

Press ^C at any time to quit.  
name: (npm_sample_app)  
version: (1.0.0)  
description:  
entry point: (index.js)  
test command:  
git repository:  
keywords:  
author:  
license: (ISC)  
About to write to C:\Projects\npm_sample_app\package.json:

{
  "name": "npm_sample_app",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}


Is this ok? (yes) yes  
C:\Projects\npm_sample_app>  

As you can see, npm init is going to ask you a bunch of questions, and provide you with nice defaults for them. Feel free to leave all of these as default for the purposes of this tutorial.

You may be thinking that you don't care about most of this stuff. You just want to install your dependencies. Don't worry, we'll get there, but, it's helpful to understand why it's asking all of this. When you use NPM for a project, you are essentially creating a module. Of course you don't have to publish your project to NPM, but NPM is heavily reliant upon modules for its dependency resolution. That is why it is asking you for a description, author, etc. These are sensible fields that most published NPM modules will use.

Back to getting things set up, if you run ls after running npm init, you will see that a file package.json was created.

C:\Projects\npm_sample_app> ls


    Directory: C:\Projects\npm_sample_app


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
-a----        7/22/2016   4:58 PM            210 package.json


C:\Projects\npm_sample_app>  

If we take a look at that file, we will see the fields that npm init asked us about are populated.

C:\Projects\npm_sample_app> cat package.json  
{
  "name": "npm_sample_app",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}
C:\Projects\npm_sample_app>  

We're not going to be publishing this project to NPM, so we can clear out all of the fields from package.json except for name and version (both which are required) just so it's a bit cleaner.

C:\Projects\npm_sample_app> cat package.json  
{
  "name": "npm_sample_app",
  "version": "1.0.0"
}
C:\Projects\npm_sample_app>  

That's all of the setup! It's important to note that all npm-init does is create package.json. There's no hidden voodoo going on. You can very well just create a package.json file by hand if you find that easier.

3. Install your dependencies

Finally we're ready to get to the good part - installing our dependencies.

For the purposes of this post, we're going to be creating a really small single page web site. To keep it simple, the only dependencies we will worry about are Bootstrap, jQuery, and Respond. This will give us everything we need to create a good looking web page that will work on IE8.

Most of the time, you can expect the NPM package name to be exactly what you think. For example, the package name for Bootstrap is bootstrap. This isn't always the case though. A library of all NPM packages is available on npmjs.com so that's where you want to check if you have any questions about package names.

Screenshot of npmjs.com

Let's start with Bootstrap. The simplest way to install a NPM dependency is to run npm install --save [packageName]. This will install the latest version of the package and save it as a dependency in package.json. The --save parameter is very important. If we don't include it, NPM will download the package, but won't save it as a dependency. Let's go ahead and install Bootstrap.

C:\Projects\npm_sample_app> npm install --save bootstrap  
npm WARN package.json [email protected] No description  
npm WARN package.json [email protected] No repository field.  
npm WARN package.json [email protected] No README data  
npm WARN package.json [email protected] No license field.  
[email protected] node_modules\bootstrap  
C:\Projects\npm_sample_app>  

If you remember back to earlier in the post, we removed some fields from package.json that we wouldn't be needing due to us not publishing this package to NPM. NPM warns us that we don't have those fields, but it still happily installs our package, so we don't need to worry about it. If you find that these warnings bug you more than having those unneeded fields in package.json, feel free to add them back in.

So a couple of things happened when we ran npm install --save bootstrap. First, let's take a look at package.json.

C:\Projects\npm_sample_app> cat package.json  
{
  "name": "npm_sample_app",
  "version": "1.0.0",
  "dependencies": {
    "bootstrap": "^3.3.6"
  }
}
C:\Projects\npm_sample_app>  

As you can see, we now have a dependencies object and we have our Bootstrap dependency specified as "bootstrap": "^3.3.6" (3.3.6 is the current version of Bootstrap at the time of writing). The default npm install --save [packageName] behavior is to install the current version of the package, and specify that if NPM install is run again, it should install the most recent version of the package up to and including changes of the minor version. This is great for development purposes, but for production, it is generally safer to specify an exact version of the package you want. So once I am ready to go to production, I will remove the ^ from the package version and end up with "bootstrap": "3.3.6". This specifies that the next time npm install is run, it will install exactly version 3.3.6.

If you are familiar with semver, this probably all makes sense to you, but if you're a bit confused on the versioning, I'd highly recommend taking a look at the official semver documentation as well as the NPM semver documentation.

The next thing that we need to take note of is that there is a new node_modules directory in our project directory.

C:\Projects\npm_sample_app> ls


    Directory: C:\Projects\npm_sample_app


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/25/2016  10:48 AM                node_modules  
-a----        7/25/2016  10:48 AM            104 package.json


C:\Projects\npm_sample_app>  

node_modules is where NPM will save your dependencies. If we take a look inside, we can see that there is a directory for Bootstrap.

C:\Projects\npm_sample_app> ls .\node_modules\


    Directory: C:\Projects\npm_sample_app\node_modules


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/25/2016  10:48 AM                bootstrap


C:\Projects\npm_sample_app>  

And inside there we see all of the files that make up the bootstrap module.

C:\Projects\npm_sample_app> ls .\node_modules\bootstrap\


    Directory: C:\Projects\npm_sample_app\node_modules\bootstrap


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/25/2016  10:48 AM                dist  
d-----        7/25/2016  10:48 AM                fonts  
d-----        7/25/2016  10:48 AM                grunt  
d-----        7/25/2016  10:48 AM                js  
d-----        7/25/2016  10:48 AM                less  
-a----        7/25/2016  10:48 AM            425 CHANGELOG.md
-a----        7/25/2016  10:48 AM          15108 Gruntfile.js
-a----        7/25/2016  10:48 AM           1084 LICENSE
-a----        7/25/2016  10:48 AM           2757 package.json
-a----        7/25/2016  10:48 AM           7489 README.md


C:\Projects\npm_sample_app>  

Let's go ahead and install our other two dependencies. Next up is jQuery, and we need to treat it a bit differently. As the goal of this page is for it to work on IE8, we need to grab an older version of jQuery as they dropped support for IE8 with version 2. Because of this, we're going to get the latest version of jQuery 1, which at the time of writing is 1.12.4. To do this, we're going to run npm install --save [email protected].

C:\Projects\npm_sample_app> npm install --save [email protected]  
npm WARN package.json [email protected] No description  
npm WARN package.json [email protected] No repository field.  
npm WARN package.json [email protected] No README data  
npm WARN package.json [email protected] No license field.  
[email protected] node_modules\jquery  
C:\Projects\npm_sample_app>  

If we look at package.json, we will see "jquery": "^1.12.4" was installed and that we will let the minor version update using the ^ prefix until we are ready for production.

C:\Projects\npm_sample_app> cat package.json  
{
  "name": "npm_sample_app",
  "version": "1.0.0",
  "dependencies": {
    "bootstrap": "^3.3.6",
    "jquery": "^1.12.4"
  }
}
C:\Projects\npm_sample_app>  

We can also now see the jquery directory under node_modules right next to bootstrap.

C:\Projects\npm_sample_app> ls .\node_modules\


    Directory: C:\Projects\npm_sample_app\node_modules


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/25/2016  10:48 AM                bootstrap  
d-----        7/25/2016  11:26 AM                jquery


C:\Projects\npm_sample_app>  

And that the jquery directory has all of jQuery's files.

C:\Projects\npm_sample_app> ls .\node_modules\jquery\


    Directory: C:\Projects\npm_sample_app\node_modules\jquery


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/25/2016  11:26 AM                dist  
d-----        7/25/2016  11:26 AM                external  
d-----        7/25/2016  11:26 AM                src  
-a----        7/25/2016  11:26 AM           9939 AUTHORS.txt
-a----        7/25/2016  11:26 AM            190 bower.json
-a----        7/25/2016  11:26 AM           1606 LICENSE.txt
-a----        7/25/2016  11:26 AM           2952 package.json
-a----        7/25/2016  11:26 AM           1794 README.md


C:\Projects\npm_sample_app>  

Lastly, let's install Respond using npm install --save respond.js.

C:\Projects\npm_sample_app> npm install --save respond.js  
npm WARN package.json [email protected] No description  
npm WARN package.json [email protected] No repository field.  
npm WARN package.json [email protected] No README data  
npm WARN package.json [email protected] No license field.  
[email protected] node_modules\respond.js  
C:\Projects\npm_sample_app>  

This will give us our final package.json.

C:\Projects\npm_sample_app> cat package.json  
{
  "name": "npm_sample_app",
  "version": "1.0.0",
  "dependencies": {
    "bootstrap": "^3.3.6",
    "jquery": "^1.12.4",
    "respond.js": "^1.4.2"
  }
}
C:\Projects\npm_sample_app>  

And our final node_modules directory.

C:\Projects\npm_sample_app> ls node_modules


    Directory: C:\Projects\npm_sample_app\node_modules


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/25/2016  10:48 AM                bootstrap  
d-----        7/25/2016  11:26 AM                jquery  
d-----        7/25/2016  11:36 AM                respond.js


C:\Projects\npm_sample_app>  

And once again we can see all of the dependency's files under the respond.js directory.

C:\Projects\npm_sample_app> ls .\node_modules\respond.js\


    Directory: C:\Projects\npm_sample_app\node_modules\respond.js


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/25/2016  11:36 AM                cross-domain  
d-----        7/25/2016  11:36 AM                dest  
d-----        7/25/2016  11:36 AM                src  
d-----        7/25/2016  11:36 AM                test  
-a----        7/25/2016  11:36 AM             34 .npmignore
-a----        7/25/2016  11:36 AM            242 bower.json
-a----        7/25/2016  11:36 AM           2232 Gruntfile.js
-a----        7/25/2016  11:36 AM           1076 LICENSE-MIT
-a----        7/25/2016  11:36 AM           1398 package.json
-a----        7/25/2016  11:36 AM           8258 README.md


C:\Projects\npm_sample_app>  

4. Use your dependencies

Now that we have all of our dependencies specified and installed, it's time to make use of them. To demonstrate this, I'm going to build out a super basic HTML page.

If you are having a hard time following the code in this cramped space, you can check it out on Github.

<!doctype html>  
<html>

<head>  
  <title>NPM Sample App</title>

  <!-- Style dependencies. -->
  <link rel="stylesheet" href="node_modules/bootstrap/dist/css/bootstrap.min.css" />

  <!-- Performance critical script dependencies. -->
  <script type="text/javascript" src="node_modules/respond.js/dest/respond.matchmedia.addListener.min.js"></script>
  <script type="text/javscript" src="node_modules/respond.js/dest/respond.min.js"></script>
</head>

<body>  
  <div class="container">
    <h1>NPM Sample App</h1>

    <div class="row">
      <div class="col-md-6">
        <div class="panel panel-default">
          <div class="panel-heading">
            <h2>Left Panel</h2>
          </div>

          <div class="panel-body">
            Nam interdum purus nec ante viverra eleifend. Proin feugiat ante non leo lacinia, blandit porta nunc venenatis.
          </div>
        </div>
      </div>

      <div class="col-md-6">
        <div class="panel panel-default">
          <div class="panel-heading">
            <h2>Right Panel</h2>
          </div>

          <div class="panel-body">
            Etiam ultricies pretium lorem at aliquet. Nam dapibus justo tellus, at bibendum ex dignissim eget. Phasellus facilisis sed turpis sed varius.
          </div>

          <div class="panel-footer">
            <div class="btn-group">
              <button type="button" class="btn btn-default dropdown-toggle" data-toggle="dropdown">
                Dropdown <span class="caret"></span>
              </button>
              <ul class="dropdown-menu">
                <li><a href="#">Something</a></li>
                <li><a href="#">Something else</a></li>
              </ul>
            </div>
          </div>
        </div>
      </div>
    </div>
  </div>

  <!-- Script dependencies. -->
  <script type="text/javascript" src="node_modules/jquery/dist/jquery.min.js"></script>
  <script type="text/javascript" src="node_modules/bootstrap/dist/js/bootstrap.min.js"></script>
</body>

</html>  

As you can see, the scripts and styles are included in the exact same way you would include them if you downloaded them from their respective sources. Most package maintainers strive to keep their directory structures the same between versions so that you can update without fear of your references breaking. While it can differ between packages, you can normally find the compiled source files in a dist or dest directory as the above dependencies do.

So, with our dependencies all installed and referenced, here is the final result, working in IE8 in all of it's glory!

Screenshot of page built above This page is hosted on github pages for your viewing pleasure.

5. Source control and re-installing dependencies

Since we now have this awesome dependency management mechanism in place, we no longer have to include our bulky dependencies in source control. I'm using git for source control, but the process to exclude files tends to be pretty similar for other source control systems. I'm going to create a .gitignore file to tell git what files to ignore.

C:\Projects\npm_sample_app [master]> cat .gitignore  
# NPM Dependencies.
node_modules  
C:\Projects\npm_sample_app [master]>  

So that's great that we won't have our dependencies in source control, but, how do we get them the next time we need to clone our repo or deploy the app? We just have to run npm install.

Looking at a freshly cloned repo, we don't see the node_modules directory as we would expect.

C:\Projects\npm_sample_app [master]> ls


    Directory: C:\Projects\npm_sample_app


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
-a----        7/25/2016   3:46 PM             35 .gitignore
-a----        7/25/2016   2:54 PM           2066 index.html
-a----        7/25/2016  11:36 AM            157 package.json


C:\Projects\npm_sample_app [master]>  

Let's run npm install.

C:\Projects\npm_sample_app [master]> npm install  
npm WARN package.json [email protected] No description  
npm WARN package.json [email protected] No repository field.  
npm WARN package.json [email protected] No README data  
npm WARN package.json [email protected] No license field.  
[email protected] node_modules\respond.js

[email protected] node_modules\jquery

[email protected] node_modules\bootstrap  
C:\Projects\npm_sample_app [master]>  

We can see that we now have our node_modules directory back.

C:\Projects\npm_sample_app [master]> ls


    Directory: C:\Projects\npm_sample_app


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/25/2016   3:49 PM                node_modules  
-a----        7/25/2016   3:46 PM             35 .gitignore
-a----        7/25/2016   2:54 PM           2066 index.html
-a----        7/25/2016  11:36 AM            157 package.json


C:\Projects\npm_sample_app [master]>  

And we can see that it has our dependencies in it.

C:\Projects\npm_sample_app [master]> ls .\node_modules\


    Directory: C:\Projects\npm_sample_app\node_modules


Mode                LastWriteTime         Length Name  
----                -------------         ------ ----
d-----        7/25/2016   3:49 PM                bootstrap  
d-----        7/25/2016   3:49 PM                jquery  
d-----        7/25/2016   3:49 PM                respond.js


C:\Projects\npm_sample_app [master]>  

Most CI tools have the ability to run npm install so you can grab your dependencies right when you're deploying without having to keep track of them.

6. Visual Studio Integration

We are going to go over this in more detail in a future post where I'm going to show you how to integrate your whole frontend workflow into Visual Studio, but I wanted to mention this one awesome feature that is baked into VS2015. It's not terribly well documented, but when you open a project in Visual Studio with a package.json, it will automatically run npm install for you so your dependencies are always intalled and up to date!

This may not seem like a huge deal as it's not hard to just run it on the command line, but the big implications here are for team work. If another developer clones a project that you are using NPM in, the dependencies won't be there since they aren't in source control. Since Visual Studio runs npm install automatically though, they don't have to worry about running it themselves. As far as they are concerned your scripts are just included in your project like they have always been. The best part is, you don't have do do a single thing for this to work.

You're done!

That's all there is to it! Just doing basic dependency handling with a package manager has the potential to make your life much easier with a minimum implementation cost. As we will see in future posts, NPM also enables you to do much more and lies at the core of most frontend processes and tools.