Node 101 - Modules and Packages

Modules vs Packages

A package is

A module is

npm

npm is the official package manager for Node. Basically, developers can publish their packages onto npm so others can use it. There are currently 201,790 total packages on npm.

npm also provides a command line interface (CLI) that allows you to interact with the registry, so you can perform actions usch as search, install, login, publish packages, all from the command line.

npm is like rubygems for Ruby, composer for PHP.

Structure of a Package

Since most of today's major platforms functions on modules, it makes sense that there are some form of specifications to determine what is a module. Today, there exists three major specifications for modules - CommonJS and AMD (Asynchronous Module Definition), and the ECMAScript 6 Modules.

A Node module on npm follows the CommonJS Specifications.

There are XX different parts to a module - definition, logic.

package.json

The module is defined in a file named package.json, which includes information of the module's name, version, its functions, dependencies etc.

Let's take a look at an example package.json file.

{
  "name": "paydollar",
  "version": "0.0.1",
  "description": "A library for integrating with PayDollar.",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/brewhk/paydollar-node.git"
  },
  "keywords": [
    "paydollar",
    "payments",
    "payment",
    "gateway",
    "asiapay",
    "ecommerce"
  ],
  "author": "Brew <dev@brew.com.hk> (http://brew.com.hk)",
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/brewhk/paydollar-node/issues"
  },
  "homepage": "https://github.com/brewhk/paydollar-node#readme"
}
  • The name is the name of the package, and must be unique.
  • The version is the current version of the package. Node uses Semantic Versioning, which in a nutshell means the first number represents the major version, the second the the minor version, and the third is the patch version.
  • main defines which file should be ran first when running this package.
  • The author field uses the format Your Name <email@example.com> (http://example.com)
index.js

In the package.js above, the main field has the value index.js, and so this is where the entry point to the package code is.

The package author could just as well named it paydollar.js or starthere.js, but following convention means other developers can follow and contribute to your code much easily.

require()

Chances are a package requires other packages to work. To use objects and functions exported (see below) by those packages you must

  1. Run npm install <pkg> --save to install a package and save it as a dependency in the package.json file.
  2. require it in your code

require() loads the package synchronously and returns the exported object. You'd require packages at the top of the file so it's available in the rest of the file.

var _ = require('lodash');

Now we can use lodash functions such as _.forEach() in the rest of the index.js file.

require() is also used if you have functions and objects defined in another file.

require('./convertToCurrencyCode.js');

This will load the convertToCurrencyCode.js file which resides in the same directory as the index.js file.

Whenever you first call require, the returned module is stored in a cache. If and when the same module is required again, the module is returned from this cache instead of being ran again.

module.exports

module.exports is the most important part of that file - whatever is assigned to module.exports will become available to applications or other packages that depend on this package.

It also means that whatever is not assigned to module.exports will not be available.

To summarize - you require other modules to use in your module, and you export anything that you wan to make available publicly.

Creating your own Package

Now you know all the theory, let's get our hands dirty. Here we will be creating a module named paydollar that integrates with the PayDollar Payment Gateway.

Here's a step-by-step guide on how to create your own module:

  1. Search. Check that there is not an existing module that does what you want already. If there is, you may want to consider using it, or better still, add new functions to it and contribute!

    You can search on npmjs.com, or you can do so straight from your command line using npm search <term>. Below, a search for braintree yields 14 results.

    danyll@ ~ $ npm search braintree
    NAME                   DESCRIPTION                                       
    42-cent-braintree      42-cent adaptor for braintree payment gateway     
    braintree              A library for integrating with Braintree.         
    braintree-angular      Use Braintree in your Angular app                 
    braintree-datahero                                                       
    braintree-react        > _Easier than sticking cash into a dirty envelope
    braintree-web          A suite of tools for integrating Braintree in the 
    braintree_encryption   An unofficial npm package for the Braintree Javasc
    card-validator         A library for validating credit card fields       
    credit-card-type       A library for determining credit card type        
    ember-braintree        Braintree's Drop-In Payment UI as an Ember compone
    form-napper            Hijack and submit forms                           
    framebus               Framebus allows you to easily send messages across
    mallorca               Man-in-the-middle proxy for HTTPS with SSL verific
    react-native-braintree A react native interface for integrating payments
    

    You can search for multiple terms using npm search, e.g. npm search braintree payments

  2. Check.Think of a name check it's availability. npm view is a command which "shows data about a package", and throws an error if there are no package with that name.

    danyll@ ~ $ npm view paydollar
    ...
    npm ERR! 404 'paydollar' is not in the npm registry.
    ...
    

    So we're good to go!

  3. Create Repository. Your code should be tracked in a repository, and when defining your package later, you'd need to fill in the repository URL. So let's first create a remote repository (e.g. on GitHub or BitBucket)

    Our repository URL is https://github.com/brewhk/paydollar-node.git

  4. Initialize. Clone your repository and move into the directory. Run npm init which will run a prompt that helps you fill out the package.json file. Three fields are required - Name, version and entry point - but you should fill out as much information as possible.

    danyll@ ~/Sandbox $ git clone git@github.com:brewhk/paydollar-node.git
    Cloning into 'paydollar-node'...
    remote: Counting objects: 5, done.
    remote: Compressing objects: 100% (5/5), done.
    remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0
    Receiving objects: 100% (5/5), done.
    Checking connectivity... done.
    danyll@ ~/Sandbox $ cd paydollar-node
    danyll@ ~/Sandbox $ npm init
    This utility will walk you through creating a package.json file.
    It only covers the most common items, and tries to guess sensible defaults.
    
    
    See `npm help json` for definitive documentation on these fields
    and exactly what they do.
    
    
    Use `npm install <pkg> --save` afterwards to install a package and
    save it as a dependency in the package.json file.
    
    
    Press ^C at any time to quit.
    name: (node-module-dump) paydollar
    version: (1.0.0) 0.0.1
    description: A library for integrating with PayDollar.
    entry point: (index.js) 
    test command: 
    git repository: https://github.com/brewhk/paydollar-node.git
    keywords: paydollar, payments, payment, gateway, asiapay, ecommerce
    author: Brew <dev@brew.com.hk> (http://brew.com.hk)
    license: (ISC) MIT
    About to write to /home/daniel/Sandbox/node-module-dump/package.json:
    
    
    {
      "name": "paydollar",
      "version": "0.0.1",
      "description": "A library for integrating with PayDollar.",
      "main": "index.js",
      "scripts": {
        "test": "echo \"Error: no test specified\" && exit 1"
      },
      "repository": {
        "type": "git",
        "url": "git+https://github.com/brewhk/paydollar-node.git"
      },
      "keywords": [
        "paydollar",
        "payments",
        "payment",
        "gateway",
        "asiapay",
        "ecommerce"
      ],
      "author": "Brew <dev@brew.com.hk> (http://brew.com.hk)",
      "license": "MIT",
      "bugs": {
        "url": "https://github.com/brewhk/paydollar-node/issues"
      },
      "homepage": "https://github.com/brewhk/paydollar-node#readme"
    }
    
    
    Is this ok? (yes) 
    

    We can read the package.json file this generated.

    {
      "name": "paydollar",
      "version": "0.0.1",
      "description": "A library for integrating with PayDollar.",
      "main": "index.js",
      "scripts": {
        "test": "echo \"Error: no test specified\" && exit 1"
      },
      "repository": {
        "type": "git",
        "url": "git+https://github.com/brewhk/paydollar-node.git"
      },
      "keywords": [
        "paydollar",
        "payments",
        "payment",
        "gateway",
        "asiapay",
        "ecommerce"
      ],
      "author": "Brew <dev@brew.com.hk> (http://brew.com.hk)",
      "license": "MIT",
      "bugs": {
        "url": "https://github.com/brewhk/paydollar-node/issues"
      },
      "homepage": "https://github.com/brewhk/paydollar-node#readme"
    }
    
  5. Create the entry point (index.js) and write your code logic, remembering to require the modules you need and module.exports the things you want to make public.

    I will be using the lodash and request module, so I first npm install it, remembering to attach the --save flag so that it will be added to the package.json file.

    $ npm install lodash --save
    lodash@3.10.1 node_modules/lodash
    $ npm install request --save
    request@2.65.0 node_modules/request
    

    package.json

    ...
    "dependencies": {
      "lodash": "^3.10.1",
      "request": "^2.65.0"
    }
    ...
    
  6. Publish. To make your package generally available to others, you must publish it to npm. If you have not already, sign up for an account; alternative, you can create an account by following the prompt using npm adduser. If you have an account already, you can use the same command - npm adduser - to log in.

    You can make sure you are logged in as the intended user by running npm whoami.

    Now you can simply run npm publish inside the package directory and you're done!

  7. Updates. npm version, npm publish to update. (https://docs.npmjs.com/cli/version)

  8. Test - Outside your package directory, create a new directory and (https://docs.npmjs.com/getting-started/creating-node-modules)

It's also a good practice to write unit tests and perform integration tests to ensure all parts of the code is tested and working. There are many tools to achieve this - Jasmine, QUnit, Mocha, Karma. Testing is a big and often complicated subject, so we will reserve that for a separate article.