Use Bash-like command-line environment variables on Windows in Node.js npm scripts

By: (plus.google.com) +David Herron; Date: April 15, 2018

Tags: Node.JS

It is best to automate all things, and in Node.js applications a convenient tool is the scripts section of package.json. Using the scripts section, npm helps you to easily record the necessary commands to build or maintain an application. However, npm does nothing to help with cross platform command-line compatibility, because it simply hands the command string to Bash (on Linux/macOS/etc) or to CMD.EXE on Windows. For the most part this is okay, but what if you need to set environment variables? The Bash way of setting command variables gives an error message on Windows.

I came across this issue while verifying the code in my latest book, Node.js Web Development 4th Edition, would work correctly on Windows. The book extolls the virtues of the Twelve Factor Application model, and for this there are two aspects:

  • Automate all administrative tasks -- for a Node.js app, that includes using npm scripts
  • Use environment variables to configure the application -- This is easy on Bash, but the command format is not portable to Windows

What I mean is to do something like this:

  "scripts": {
    "start": "DEBUG=notes:* node --experimental-modules ./bin/www.mjs", 
    "start-fs": "DEBUG=notes:* NOTES_MODEL=fs node --experimental-modules ./bin/www.mjs", 
    "start-level": "DEBUG=notes:* NOTES_MODEL=level node --experimental-modules ./bin/www.mjs", 
    "start-sqlite3": "SQLITE_FILE=chap07.sqlite3 DEBUG=notes:* NOTES_MODEL=sqlite3 node --experimental-modules ./bin/www.mjs",
    "start-sequelize": "SEQUELIZE_CONNECT=models/sequelize-sqlite.yaml NOTES_MODEL=sequelize node --experimental-modules ./bin/www.mjs",
    "start-mongodb": "MONGO_URL=mongodb://localhost/ MONGO_DBNAME=chap07 NOTES_MODEL=mongodb node --experimental-modules ./bin/www.mjs",
    "server1": "NOTES_MODEL=fs PORT=3001 node --experimental-modules ./bin/www.mjs", 
    "server2": "NOTES_MODEL=fs PORT=3002 node --experimental-modules ./bin/www.mjs",
    "server1-sqlite3": "SQLITE_FILE=chap07.sqlite3 NOTES_MODEL=sqlite3 PORT=3001 node --experimental-modules ./bin/www.mjs", 
    "server2-sqlite3": "SQLITE_FILE=chap07.sqlite3 NOTES_MODEL=sqlite3 PORT=3002 node --experimental-modules ./bin/www.mjs",
    "server1-sequelize": "SEQUELIZE_CONNECT=models/sequelize-sqlite.yaml NOTES_MODEL=sequelize PORT=3001 node --experimental-modules ./bin/www.mjs", 
    "server2-sequelize": "SEQUELIZE_CONNECT=models/sequelize-sqlite.yaml NOTES_MODEL=sequelize PORT=3002 node --experimental-modules ./bin/www.mjs",
    "server1-mongodb": "MONGO_URL=mongodb://localhost/ MONGO_DBNAME=chap07 NOTES_MODEL=mongodb PORT=3001 node --experimental-modules ./bin/www.mjs", 
    "server2-mongodb": "MONGO_URL=mongodb://localhost/ MONGO_DBNAME=chap07 NOTES_MODEL=mongodb PORT=3002 node --experimental-modules ./bin/www.mjs",
    "sqlite3-setup": "sqlite3 chap07.sqlite3 --init models/schema-sqlite3.sql"
  },

What we have here is the same application (bin/www.mjs) being executed in different modes by setting environment variables to configure each mode. For example PORT sets the TCP port the app will listen on, NOTES_MODEL selects the type of storage for the application, and SEQUELIZE_CONNECT is configuration for the Sequelize module to specify the database to be used.

According to the Twelve Factor Application model, environment variables are an incredibly simple mechanism to inject configuration into an application. The application simply queries process.env.ENVIRONMENT_VARIABLE_NAME to pick up the configuration option. No need for baroque parsing of command line options, or yet another configuration file syntax.

The problem with this approach is that this command format only works for Bash. As a Linux and MacOS based software developer, Bash is second nature to me, but there are a significant number of folks using Windows.

Our code needs to be useable by those who use Windows. It's unfortunate that Windows chose such an incompatible usage environment, but it's not up to us to stand in high judgement over Microsoft or those who prefer Microsoft's software.

The problem is - how do we write npm scripts that are portable across Windows and Linux/macOS/etc systems?

If we legitimately have users on a variety of operating systems, we must accommodate them in our software. Otherwise the distribution of our software would be harmed.

It's arguable that since this is such a strong generic problem across all software developers, that the solution should be implemented inside npm rather than the approach I'm about to show. What if npm incorporated something to parse the scripts using Bash-like syntax, for full cross-platform enjoyment?

But that's not what npm does, so we must work with the product we have.

Best solution with current npm capabilities

I came up with several options by searching the npm repository using the phrase "windows script".

The best solution looks to be the cross-env package, see: (www.npmjs.com) https://www.npmjs.com/package/cross-env

This adds a command in node_modules/.bin that interpolates the Bash command-line style of setting environment variables.

Installation via: npm install cross-env@5.1.4 --save

Then you simply modify each npm scripts entry to add cross-env to the front of the command line, like so:

  "scripts": {
    "start": "cross-env DEBUG=notes:* node --experimental-modules ./bin/www.mjs", 
    "start-fs": "cross-env DEBUG=notes:* NOTES_MODEL=fs node --experimental-modules ./bin/www.mjs", 
    "start-level": "cross-env DEBUG=notes:* NOTES_MODEL=level node --experimental-modules ./bin/www.mjs", 
    "start-sqlite3": "cross-env SQLITE_FILE=chap07.sqlite3 DEBUG=notes:* NOTES_MODEL=sqlite3 node --experimental-modules ./bin/www.mjs",
    "start-sequelize": "cross-env SEQUELIZE_CONNECT=models/sequelize-sqlite.yaml NOTES_MODEL=sequelize node --experimental-modules ./bin/www.mjs",
    "start-mongodb": "cross-env MONGO_URL=mongodb://localhost/ MONGO_DBNAME=chap07 NOTES_MODEL=mongodb node --experimental-modules ./bin/www.mjs",
    "server1": "cross-env NOTES_MODEL=fs PORT=3001 node --experimental-modules ./bin/www.mjs", 
    "server2": "cross-env NOTES_MODEL=fs PORT=3002 node --experimental-modules ./bin/www.mjs",
    "server1-sqlite3": "cross-env SQLITE_FILE=chap07.sqlite3 NOTES_MODEL=sqlite3 PORT=3001 node --experimental-modules ./bin/www.mjs", 
    "server2-sqlite3": "cross-env SQLITE_FILE=chap07.sqlite3 NOTES_MODEL=sqlite3 PORT=3002 node --experimental-modules ./bin/www.mjs",
    "server1-sequelize": "cross-env SEQUELIZE_CONNECT=models/sequelize-sqlite.yaml NOTES_MODEL=sequelize PORT=3001 node --experimental-modules ./bin/www.mjs", 
    "server2-sequelize": "cross-env SEQUELIZE_CONNECT=models/sequelize-sqlite.yaml NOTES_MODEL=sequelize PORT=3002 node --experimental-modules ./bin/www.mjs",
    "server1-mongodb": "cross-env MONGO_URL=mongodb://localhost/ MONGO_DBNAME=chap07 NOTES_MODEL=mongodb PORT=3001 node --experimental-modules ./bin/www.mjs", 
    "server2-mongodb": "cross-env MONGO_URL=mongodb://localhost/ MONGO_DBNAME=chap07 NOTES_MODEL=mongodb PORT=3002 node --experimental-modules ./bin/www.mjs",
    "sqlite3-setup": "sqlite3 chap07.sqlite3 --init models/schema-sqlite3.sql",
  },

This is simple, you just use the Bash syntax and proceed with life.

Other alternatives

npmcs: (www.npmjs.com) https://www.npmjs.com/package/npmcs -- Has you implement a Module similar to the npm scripts section that has sub-sections for each operating system. Like:

//npmcs-scripts.js
module.exports = {
    scripts: {
        //nodemon, build and test commands same regardless of running on windows/linux (remove duplication)
        nodemon: 'nodemon --debug src/app.js',
        build: 'start webpack -d --watch',
        test: 'echo "Error: no test specified" && exit 1'
        /*
            Windows scripts, run on a windows environment
        */
        win: {
            start: 'start npm run dev',
            dev: 'SET NODE_ENV=development && npm run build && npm run nodemon',
            prod: 'SET NODE_ENV=production node src/app.js',
        },
        /*
            Unix scripts, run on a unix environment
        */
        nix: {
            start: 'npm run dev',
            dev: 'export NODE_ENV=development && npm run build && npm run nodemon',
            prod: 'export NODE_ENV=production node src/app.js',
        },
    }
}

cross-os: (www.npmjs.com) https://www.npmjs.com/package/cross-os -- Modifies the npm package.json to have subsections for each operating system, like so:

"scripts": {
  "foo": "cross-os bar",
  "bar": {
    "darwin": "echo 'i will only run on Mac'",
    "win32": "echo 'i will only run on Windows'",
    "linux": "echo 'i will only run on Linux'"
  }
}

Windows Subsystem for Linux

Another approach is for Windows developers to install WSL. This is a Linux installation running on Windows - for example, Ubuntu - where you can apt-get to install packages and all the other things you'd do on Linux. The important bit is you have a Bash shell, and the scripts commands will automatically run correctly.

This requires Windows 10 updated with the latest creators pack or whatever it is. I've done this, and the WSL environment is a breath of fresh air. No longer do I feel hampered by CMD.EXE whose usage paradigm was designed in the 1970's. Instead I'm breathing easy in Bash, whose usage paradigm was also designed in the 1970's, but is a paradigm that is far more sensible. Obviously saying "usage paradigm designed in the 1970's" is incorrect, but to me there's something about CMD.EXE that feels crufty and backwards.

We have to recognize WSL may not be the correct solution for all use cases.

It's possible for some to like the CMD.EXE environment, prefer it over Bash, and to be affronted by our suggestion to install WSL.

Some of them will also be affronted by the attitude I'm probably expressing through my word choice. I may be expressing judgmentalism about Windows and Windows users which I'm trying hard to avoid. Windows users are people just like me, and deserve all the respect I deserve.

At the same time, CMD.EXE seems so primitive and I can't imagine why anyone would possibly like it. Having looked into PowerShell, it's really no better. But it's easy to find folks who are ecstatic about PowerShell, and who am I or anyone else to judge them?

Possible modification to npm

What about:

{
  "scripts": {
    "start": {
        "environment": {
            "DEBUG": "notes:*"
        },
        "command": "node --experimental-modules ./bin/www.mjs"
    },
    "start-fs": {
        "environment": {
            "DEBUG": "notes:*",
            "NOTES_MODEL": "fs"
        },
        "command": "node --experimental-modules ./bin/www.mjs"
    },
    "start-level": {
        "environment": {
            "DEBUG": "notes:*",
            "NOTES_MODEL": "level"
        },
        "command": "node --experimental-modules ./bin/www.mjs"
    },
    "start-sqlite3": {
        "environment": {
            "DEBUG": "notes:*",
            "NOTES_MODEL": "sqlite3",
            "SQLITE_FILE": "chap07.sqlite3"
        },
        "command": "node --experimental-modules ./bin/www.mjs"
    },
    "start-sequelize": {
        "environment": {
            "DEBUG": "notes:*",
            "NOTES_MODEL": "sequelize",
            "SEQUELIZE_CONNECT": "models/sequelize-sqlite.yaml"
        },
        "command": "node --experimental-modules ./bin/www.mjs"
    },
    ...
    "sqlite3-setup": "sqlite3 chap07.sqlite3 --init models/schema-sqlite3.sql"
  }
}

In other words the rule is - if the value portion of a scripts entry is a simple string, it will be interpreted as so, but if it is instead an object there are fields to specify the environment as a data structure.

The mechanism could even expose some of the options to the exec function: (nodejs.org) https://nodejs.org/api/child_process.html#child_process_child_process_exec_command_options_callback

« Node.js 10.x released - What's NEW? Fix NVM_NODEJS_ORG_MIRROR deprecation warning when running npm install »
2016 Election Acer C720 Ad block AkashaCMS Amazon Amazon Kindle Amazon Web Services America Amiga and Jon Pertwee Android Anti-Fascism AntiVirus Software Apple Apple Hardware History Apple iPhone Apple iPhone Hardware April 1st Arduino ARM Compilation Artificial Intelligence Astronomy Astrophotography Asynchronous Programming Authoritarianism Automated Social Posting AWS DynamoDB AWS Lambda Ayo.JS Bells Law Big Brother Big Data Big Finish Big Science Bitcoin Mining Black Holes Blade Runner Blockchain Blogger Blogging Books Botnets Cassette Tapes Cellphones China China Manufacturing Christopher Eccleston Chrome Chrome Apps Chromebook Chromebox ChromeOS CIA CitiCards Citizen Journalism Civil Liberties Clinton Cluster Computing Command Line Tools Comment Systems Computer Accessories Computer Hardware Computer Repair Computers Conservatives Cross Compilation Crouton Cryptocurrency Curiosity Rover Currencies Cyber Security Cybermen Daleks Darth Vader Data backup Data Formats Data Storage Database Database Backup Databases David Tenant DDoS Botnet Department of Justice Detect Adblocker Developers Editors Digital Photography Diskless Booting Disqus DIY DIY Repair DNP3 Do it yourself Docker Docker MAMP Docker Swarm Doctor Who Doctor Who Paradox Doctor Who Review Drobo Drupal Drupal Themes DVD E-Books E-Readers Early Computers Election Hacks Electric Bicycles Electric Vehicles Electron Eliminating Jobs for Human Emdebian Encabulators Energy Efficiency Enterprise Node EPUB ESP8266 Ethical Curation Eurovision Event Driven Asynchronous Express Face Recognition Facebook Fake News Fedora VirtualBox Fifth Doctor File transfer without iTunes FireFly Flash Flickr Fraud Freedom of Speech Front-end Development Gallifrey git Github GitKraken Gitlab GMAIL Google Google Chrome Google Gnome Google+ Government Spying Great Britain Green Transportation Hate Speech Heat Loss Hibernate Hoax Science Home Automation HTTP Security HTTPS Human ID I2C Protocol Image Analysis Image Conversion Image Processing ImageMagick In-memory Computing InfluxDB Infrared Thermometers Insulation Internet Internet Advertising Internet Law Internet of Things Internet Policy Internet Privacy iOS Devices iPad iPhone iPhone hacking Iron Man iTunes Janet Fielding Java JavaFX JavaScript JavaScript Injection JDBC John Simms Journalism Joyent Kaspersky Labs Kindle Kindle Marketplace Large Hadron Collider Lets Encrypt LibreOffice Linux Linux Hints Linux Single Board Computers Logging Mac Mini Mac OS Mac OS X Machine Learning Machine Readable ID macOS MacOS X setup Make Money Online March For Our Lives MariaDB Mars Mass Violence Matt Lucas MEADS Anti-Missile Mercurial MERN Stack Michele Gomez Micro Apartments Microsoft Military AI Military Hardware Minification Minimized CSS Minimized HTML Minimized JavaScript Missy Mobile Applications Mobile Computers MODBUS Mondas Monetary System MongoDB Mongoose Monty Python MQTT Music Player Music Streaming MySQL NanoPi Nardole NASA Net Neutrality Network Attached Storage Node Web Development Node.js Node.js Database Node.js Performance Node.js Testing Node.JS Web Development Node.x North Korea npm NVIDIA NY Times Online advertising Online Community Online Fraud Online Journalism Online Photography Online Video Open Media Vault Open Source Open Source Governance Open Source Licenses Open Source Software OpenAPI OpenJDK OpenVPN Palmtop PDA Patrick Troughton Paywalls Personal Flight Peter Capaldi Peter Davison Phishing Photography PHP Plex Plex Media Server Political Protest Politics Postal Service Power Control President Trump Privacy Production use Public Violence Raspberry Pi Raspberry Pi 3 Raspberry Pi Zero ReactJS Recaptcha Recycling Refurbished Computers Remote Desktop Removable Storage Republicans Retro Computing Retro-Technology Reviews RFID Rich Internet Applications Right to Repair River Song Robotics Robots Rocket Ships RSS News Readers rsync Russia Russia Troll Factory Russian Hacking Rust SCADA Scheme Science Fiction SD Cards Search Engine Ranking Season 1 Season 10 Season 11 Security Security Cameras Server-side JavaScript Serverless Framework Servers Shell Scripts Silence Simsimi Skype SmugMug Social Media Social Media Warfare Social Network Management Social Networks Software Development Space Flight Space Ship Reuse Space Ships SpaceX Spear Phishing Spring Spring Boot Spy Satellites SQLite3 SSD Drives SSD upgrade SSH SSH Key SSL Stand For Truth Strange Parts Swagger Synchronizing Files Tegan Jovanka Telescopes Terrorism The Cybermen The Daleks The Master Time-Series Database Tom Baker Torchwood Total Information Awareness Trump Trump Administration Trump Campaign Twitter Ubuntu Udemy UDOO US Department of Defense Virtual Private Networks VirtualBox VLC VNC VOIP Vue.js Walmart Web Applications Web Developer Resources Web Development Web Development Tools Web Marketing Webpack Website Advertising Weeping Angels WhatsApp William Hartnell Window Insulation Windows Windows Alternatives Wordpress World Wide Web Yahoo YouTube YouTube Monetization