; Date: Wed Aug 14 2019

Tags: Node.JS

We want to automate the boring stuff so we have more time for interesting stuff. Clearly running tests and administrivia like publishing a Node.js package to npm, well, that's boring stuff. It is the same tasks every time, and as an unautomated process you might forget a step, so let's look at how to automate this common workflow for Node.js programmers.

It isn't hard to test a package, then publish it. With a little bit of work it's as easy as

$ npm test && npm publish

The goal here is a workflow where you write some code, then push to your source repository. Once code is pushed to the repository automated systems fire up to handle testing and publishing. While that task is not hard, it is a burden you can easily lift from your shoulders. It'll let you focus more on coding.

I have several packages in npm, primarily AkashaCMS and the various plugins. AkashaCMS is a static website generator - well, this very website is built using AkashaCMS which should give you an idea of what it does. But that's not important for this posting, other than for the latest plugin I developed I decided to look into automating the entire workflow of testing and and publishing the npm package.

The AkashaCMS Authors Plugin source is what I'll be referring to.

This is a fairly simple Node.js package. There's one index.js in the root directory, a subdirectory with some EJS templates, and a package.json. For tests I created a subdirectory, test, containing its own package.json, another index.js containing a Mocha/Chai test suite, and then two subdirectories of test files and layout files.

SInce the code being tested is an AkashaCMS plugin, the test suite had to set up a tiny AkashaCMS project. Files are rendered using different options, and then to verify results the test case uses Cheerio so we can verify DOM content using a jQuery-like API.

The Mocha/Chai test suite for the Author plugin

I prefer using Mocha as the test driver, and using Chai assertions. It doesn't matter if you prefer a different unit test setup. The goal is to run the test suite by:

$ cd test && npm test

In this case it was necessary to do a little bit of setup. To execute the code being tested, I had to install both AkashaRender and Mahabhuta (two components of AkashaCMS) in node_modules in the root directory.

Therefore in test/package.json I have these scripts:

"scripts": {
    "test": "mocha ./index",
    "setup": "npm install && cd .. && npm install akashacms/akasharender akashacms/mahabhuta --no-save",
    "clean": "rm -rf node_modules out"
},

With setup it first runs npm install in the test directory, then goes to the parent directory to install the package dependencies. I used --no-save because those dependencies must not be listed in package-level package.json.

The clean script removes some files that can be safely deleted. The out directory is where AkashaRender renders the documents, and we read the rendered documents in the test for validation.

The test script is where the test suite is run.

That's the setup required in the test directory. Bottom line, one must ensure the test suite reliably runs simply by executing npm test.

Integrating test suite with package-level package.json

In the root directory is the package.json which we publish to npm, and which our customers use.

The scripts here are wholly focused on driving the test suite:

"scripts": {
    "dopublish": "npm publish --access public",
    "test-setup": "cd test && npm run setup",
    "test": "cd test && npm test",
    "clean": "cd test && npm run clean"
},

With test-setup we run the setup script in the test directory. Likewise test and clean run the corresponding scripts in the test directory.

The dopublish script handles publishing. Since this package is published to an NPM Organization (as @akashacms/plugins-authors) we have to use the --access public flag because the package has to be public. We record this as a package.json script so that this is not forgotten.

If we were to do this "by hand" we'd simply execute:

$ npm test && npm run dopublish

Why dopublilsh? I found that if this script were named publish then it ran multiple times. It must be that the npm publish command runs the publish script if it exists. In any case naming the script dopublish meant it published only once.

There is a prerequisite required for npm publish to run from the Jenkins environment. Jenkins jobs run at the shell level on the build server. That means the npm login credentials must exist in that environment for npm publish to work correctly. Therefore you must have run npm login while logged in as jenkins on your server.

Automating this in Jenkins

Now that you have testing and publishing automated in npm package.json scripts it's time to think about integrating this into a continuous integration (CI/CD) system. I use Jenkins for this, but of course there are other similar tools. I just learned the other day that Github is adding CI/CD support.

To setup the job in Jenkins, I used what it describes as a Freestyle project.

Into the job I configured:

What's the || true bit after npm publish?

What's going on is - the build job tries to npm publish every time. What if I've pushed code without bumping the version in package.json? I do that sometimes if I'm building up several changes for one release. I'll push code multiple times without updating the version number.

If the version number in package.json has already been published, then npm publish will throw an error that you cannot publish on top of an existing package version. That makes sense.

But if I were to push code without a version number change, then the build would fail. But using || true is a Bash trick to avoid any error.

Therefore we can push all the changes we want without publishing the package. The npm publish task runs every time and attempts to publish, but will fail if the version number has not changed. As soon as we push a change with an updated version number, then npm publish will succeed to publish the package.

A detail to note here is this job checks the Master branch. If you want to publish from a different branch - maybe you have a beta branch for beta builds? - that should be done in a different Jenkins job.