; Date: Thu Sep 12 2019

Tags: Node.JS

Github's recently added new feature, Github Actions, promises us a powerful Workflow system to handle a huge range of tasks. Those of us who publish Node.js packages can use Actions to automatically run tests and then publish the package to npm. Let us see how to use Github Actions to simplify our lives.

At the moment Github Actions is in public beta and is only open to those who sign up - See: https://github.com/features/actions to sign up, and to read the documentation.

The basic concept of Github Actions is that an Action is triggered on an event. Each Action is a potentially very complex set of steps scripted using a YAML file. In this case we will create an Action triggered by the push event, or whenever a new commit is pushed to the repository. The script will run our unit tests, then attempt to run npm publish.

Once the Actions feature is enabled in your account, you'll find an Actions tab in each repository.

There's a bunch of documentation available, and it might be useful to read it some day. But let's just dive right in and create a Workflow instead. You'll find among the list of starter workflows is one for Node.js, click on it to give us a starting point.

For this tutorial I used one of my packages, Globfs. You'll be able to inspect that package to more fully examine what I did.

I made a lot of modifications, so let's not worry about the Actions script they supply, but look at this:

name: Build and test Globfs

on: [push]

jobs:
  build:

    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [12.x]

    steps:
    - uses: actions/checkout@v1
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    - name: npm install and npm test
      run: |
        npm run setup-test
        npm run test
    - name: npm publish
      run: |
        npm config set //registry.npmjs.org/:_authToken=$NODE_AUTH_TOKEN
        npm run trypublish
      env:
        CI: true
        NODE_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}

When you create this script Github will automatically add it to your repository as .github/workflows/workflow-name.yml. In my case I named it push.yml because it handles the push event.

Walking through the push.yml Action script

The name field simply gives a human-readable name, so use anything you like there. The on field declares the events this will trigger on, in this case the Action triggers on the push event.

The jobs field is executed when the event triggers. The job is run on a Docker container named ubuntu-latest which is perfectly fine for our purpose.

The steps field is a list of steps that are executed. The first, actions/checkout@v1, simply checks out the workspace.

I'll mention in passing that Github Actions lets us reuse other existing actions. In this case we're reusing an Action defined in the Github workspace actions/checkout, and you'll find it checks out the Git repository.

The next step sets up Node.js. The strategy/matrix field lets us set up an array of Node.js releases on which to run the Action. In this case we only want to run on Node.js 12, because that's what is supported for this package. When run actions/setup-node sets up the corresponding Node.js runtime.

The next step, npm install and npm test, handles testing the package. To support this, I added the following scripts in the top-level package.json:

"scripts": {
    "setup-test": "cd test && npm run setup",
    "test": "cd test && npm run test"
}

Each simply goes into the test directory and runs the corresponding script there. In the test directory I added this package.json:

{
  "name": "globfs-test",
  "version": "1.0.0",
  "description": "Test suite for globfs",
  "main": "index.js",
  "scripts": {
    "test": "mocha ./index",
    "setup": "npm install && cd .. && npm install"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "chai": "^4.1.2",
    "mocha": "^5.2.0"
  }
}

Obviously the test script requires a test suite in index.js. I created one using Mocha and the Chai assertions library, which you can read on the Globfs repository if you want. What matters is that the test script reliably execute the test suite, and that it reliably shows an error state if the test suite fails.

If the tests fail the Action script fails and Github will send us the error log by e-mail. If instead the test script succeeds, that's great and the Action script will proceed to the next stage.

The last stage of the Action script, npm publish, is where we publish the package. Theoretically it's just a matter of running npm publish but there are a couple considerations to handle. For example if you run npm publish on a package directory that's already been published, the npm server will just throw an error that you cannot publish a package over an existing package.

My strategy is to always run npm publish, if the tests succeed. If the version string in package.json has changed then npm publish will push the package to the repository. But if the version number did not change, we do not care that the npm publish command fails. To avoid the failure I use this one little weird shell scripting trick from ancient times.

In the top-level package.json I added this script:

"scripts": {
    ...
    "trypublish": "npm publish || true"
},

With trypublish we attempt to run npm publish and then if that fails the error is suppressed by using || true. Look up this shell scripting technique, it's a piece of awesomeness invented in the 1970's.

But take a close look at the npm publish stage in push.yml. There's a bit more stuff there involving an npm config command. What's that about?

In order to publish to npm, we need an authorization token. Getting that to work required a side journey which I hope to help you avoid.

Setting up an npm token for authentication to publish to npm from a Github Actions workflow script

Supposedly to authenticate npm publish, we simply put an npm token in an environment variable, and by magic everything works. According to the npm blog the canonical way to authenticate npm is adding this line to ~/.npmrc.

//registry.npmjs.org/:_authToken=00000000-0000-0000-0000-000000000000

But that's not convenient in the Action script above. The blog post goes on to say we can do this:

$ export NPM_TOKEN="00000000-0000-0000-0000-000000000000"

But I found no environment variable that would work. Nor did this incantation in the documentation of the actions/setup-node action:

- run: npm publish
  env:
    NODE_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}

Instead I spent several hours of searching for advice. Instead I came across several others who'd also wasted hours of their time chasing after the same problem.

But after enough reading, I developed a solution which works.

Let's do this one step at a time, starting with creating an npm token.

When logged-in to npmjs.com you'll find this choice in the account menu. This takes you to a page where you can create (and revoke) tokens.

Simply click on the Create New Token button. It'll ask a couple questions, then bring you to a page showing the new token. Make sure to copy this token somewhere secure, because this is the only time in this universe when you will see that token.

The next step is to go to the Settings tab of your Github repository, and add a Secret containing the token you were given. You will never be able to inspect the content of this secret, once created, using Github.

In the Github Actions script we then access this secret adding it to the environment this way:

env:
    NPM_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}

But this is not enough, despite what the npm documentation on authentication tokens says. No matter what environment variable I used, npm publish did not recognize the token. Only when adding this command did it work:

npm config set //registry.npmjs.org/:_authToken=$NPM_TOKEN

This is the same configuration which must be added to the .npmrc file. This command gets run inside the Docker container that Github spun up to handle this job.

Once you've done this, the npm publish command works as expected.

Conclusion

These are early days for Github Actions. The work involved to set this up is harder than the story we've been told about Github Actions. There are a couple rough edges that could be straightened out and make this work much more smoothly.

This sort of automation will help developers stay sane. Consider what your life would be like if testing a Node.js package and publishing it to the npm registry were not automated. For every publication of a release you'd be typing the required commands away in the terminal. What if you forgot a command, or forgot a parameter? What if you forgot to publish the new release?

There's all kinds of mistakes that can happen. It's surely far more reliable to automate all this so you have repeatable processes.

One thing's for sure - Gitlab has until now had an advantage over Github in that Gitlab has long had a CI/CD subsystem. With that system you add a YAML file to the repository describing the CI/CD workflow, and voila you have automation. The Github Actions feature is far more flexible and could give Gitlab a run for its money.