Dynamic import lands in Node.js, we can import ES6 modules in CommonJS code

By: (plus.google.com) +David Herron; Date: March 2, 2018

Tags: Node.JS » JavaScript

ES6 modules are a cool new feature coming with Node.js 10.x, and a pre-release version available is avaible in 9.x. ES6 modules implement most of what we can do with Node.js's traditional CommonJS modules. One feature missing is the ability to dynamically select the module to load. The ES6 import statement only takes a static string. A new feature, landing in Node.js 9.7.x, adds an import() function offering a new way to load modules.

The most important feature may be that import() can be used in a CommonJS module to import an ES6 module.

I am not clear on the general use-case targeted by the Dynamic Import feature. But I do have two specific instances demonstrating rationales for using import(). Namely:

  • Dynamically determining the module to load at run time
  • Loading an ES6 module into a Node.js/CommonJS module

As of this writing, Node.js 9.7, using ES6 modules and using import() requires running node with the --experimental-modules flag.

Loading an ES6 module in a Node.js/CommonJS module

A few days ago I wrote about the @std/esm module which enables loading an ES6 module in a CommonJS module using the require() function. That process required replacing the require() function with one created by @std/esm. Replacing a standard platform function is never a preferable thing to do.

With the new Dynamic Import feature there's another route, because import() is available in both ES6 module and CommonJS module contexts. This new function is an API which can import ES6 modules. Because loading an ES6 module is an asynchronous process, import() returns a Promise. Therefore it's not a direct replacement of the require() function we're familiar with.

Consider a simple ES6 module:

$ cat cjs.js 
export default function() {
  console.log('Hello World from ES6 module');

This is easy to use from another ES6 module: import t from 't'; But by the rules of what modules can be loaded where, the big gap was using ES6 modules in a CommonJS module.


$ cat cjs.js 
const util = require('util');

.then(t => {
.catch(err => {

When run we get the following output:

$ node --version
$ node --experimental-modules cjs.js 
(node:10282) ExperimentalWarning: The ESM module loader is experimental.
{ default: [Function: default] }
Hello World from ES6 module

The first thing we learn is that indeed import() is a function returning a Promise. The result of the Promise is the loaded module.

That loaded module object is an anonymous object containing the items exported by the module. We can see this more clearly by adding another item to t.mjs:

$ cat t.mjs 
export default function() {
  console.log('Hello World from ES6 module');

export const meaning = 42;

Then rerunning the app:

$ node --experimental-modules cjs.js 
(node:16121) ExperimentalWarning: The ESM module loader is experimental.
{ default: [Function: default], meaning: 42 }
Hello World from ES6 module

We see the new field showed up in the ES6 module object.

We won't be able to use this as a direct replacement for require() because the global context doesn't support await on a Promise:

$ cat bad.js 

const util = require('util');
const t = await import('./t.mjs');


We expect this to fail because await can only be used inside an async function. When run this fails as expected:

$ node --experimental-modules bad.js 
(node:17135) ExperimentalWarning: The ESM module loader is experimental.
const t = await import('./t.mjs');

SyntaxError: await is only valid in async function
    at new Script (vm.js:51:7)
    at createScript (vm.js:136:10)
    at Object.runInThisContext (vm.js:197:10)
    at Module._compile (module.js:626:28)
    at Object.Module._extensions..js (module.js:673:10)
    at Module.load (module.js:575:32)
    at tryModuleLoad (module.js:515:12)
    at Function.Module._load (module.js:507:3)
    at createDynamicModule (internal/loader/Translators.js:50:15)
    at setExecutor (internal/loader/CreateDynamicModule.js:49:23)

While that doesn't work, we can use import() in other contexts such as inside an async function. See the next section for an example.

Dynamically determining the ES6 module to load at runtime

My book Node.js Web Development, 4th edition, is nearing completion (meaning I'm working hard on editing the book as we speak). The book demonstrates building Node.js app's by walking the reader through building a simple application that stores data in a database. In the 4th edition I'm updating the book to use async functions and ES6 modules.

As demonstration app, it's desired to show how to use several kinds of databases to store data. The cleanest architectural choice is to define an API to be implemented by the several modules that will interface with each database, and we end up with this list of modules:

  • notes-memory.mjs
  • notes-fs.mjs
  • notes-level.mjs
  • notes-sqlite3.mjs
  • notes-sequelize.mjs
  • notes-mongodb.mjs

It should be obvious - each of these modules deal with storing "notes" in the corresponding database system.

The rest of the application needs a simple method to dynamically choose between these modules.

In the 3rd edition, this is the method used:

const notes = require(process.env.NOTES_MODEL
            ? path.join('..', process.env.NOTES_MODEL)
            : '../models/notes-memory');

That was pretty simple but it doesn't work with ES6 modules. As I said above, the ES6 import statement only takes a simple string literal. Meaning you cannot do this:

import notes from process.env.NOTES_MODEL
            ? path.join('..', process.env.NOTES_MODEL)
            : '../models/notes-memory';

For the 4th edition the chosen approach is to use the dynamic import() function. This is not a statement, but a function which returns a Promise that, when resolved, is the module.

This won't work as a top-level code snippet to dynamically choose the module to load:

const notes = await import(process.env.NOTES_MODEL
            ? path.join('..', process.env.NOTES_MODEL)
            : '../models/notes-memory');

That's because await only works inside an async function, and this is outside any function.

What I ended up implementing is this module, called notes.mjs, that will delegate to the dynamically chosen module based on an environment variable.

var NotesModule;

async function model() {
    if (NotesModule) return NotesModule;
    NotesModule = await import(`../models/notes-${process.env.NOTES_MODEL}`);
    return NotesModule;

export async function create(key, title, body) { return (await model()).create(key, title, body); }
export async function update(key, title, body) { return (await model()).update(key, title, body); }
export async function read(key) { return (await model()).read(key); }
export async function destroy(key) { return (await model()).destroy(key); }
export async function keylist() { return (await model()).keylist(); }
export async function count() { return (await model()).count(); }
export async function close() { return (await model()).close(); }

The functions at the bottom match the API supported by these modules.

What's happening is that model() either immediately returns the module object (if it's already been loaded) or uses import() to load the module. We have to use await because import() itself returns a Promise.

In the API delegation methods we have to use (await model()) because model is an async function, and therefore returns a Promise.

You may be thinking (because I am) what's the impact of this layer of indirection? There's several await and async operations going on, shouldn't this be a burden?

The first time through is where the big execution cost occurs. That's because of the time required to load the module.

On subsequent times through, the module will have been loaded and model() immediately returns a resolved Promise. It's fast to construct a resolved Promise, and it's fast to retrieve its value. Therefore the cost is constructing a Promise object then retrieving the resolved value, which sure adds a little bit of overhead but it's minor.

« Deploy Ghost blogging system using Passenger on Dreamhost or other hosting providers Handling unhandled Promise rejections, avoiding application crash »
2016 Election 2018 Elections Acer C720 Ad block Affiliate marketing Air Filters Air Quality Air Quality Monitoring AkashaCMS Amazon Amazon Kindle Amazon Web Services America Amiga and Jon Pertwee Android Anti-Fascism AntiVirus Software Apple Apple Flexgate Apple Hardware History Apple Hardware Mistakes Apple iPhone Apple iPhone Hardware April 1st Arduino ARM Compilation Artificial Intelligence Astronomy Astrophotography Asynchronous Programming Authoritarianism Automated Social Posting AWS DynamoDB AWS Lambda Ayo.JS Bells Law Big Brother Big Data Big Finish Big Science Bitcoin Mining Black Holes Blade Runner Blockchain Blogger Blogging Books Botnets Cassette Tapes Cellphones China China Manufacturing Christopher Eccleston Chrome Chrome Apps Chromebook Chromebox ChromeOS CIA CitiCards Citizen Journalism Civil Liberties Climate Change Clinton Cluster Computing Command Line Tools Comment Systems Computer Accessories Computer Hardware Computer Repair Computers Conservatives Cross Compilation Crouton Cryptocurrency Curiosity Rover Currencies Cyber Security Cybermen Cybersecurity Daleks Darth Vader Data backup Data Formats Data Storage Database Database Backup Databases David Tenant DDoS Botnet Department of Defense Department of Justice Detect Adblocker Developers Editors Digital audio Digital Nomad Digital Photography Direct Attach Storage Diskless Booting Disqus DIY DIY Repair DNP3 Do it yourself Docker Docker MAMP Docker Swarm Doctor Who Doctor Who Paradox Doctor Who Review Drobo Drupal Drupal Themes DuckDuckGo DVD E-Books E-Readers Early Computers eGPU Election Hacks Electric Bicycles Electric Vehicles Electron Eliminating Jobs for Human Emdebian Encabulators Energy Efficiency Enterprise Node EPUB ESP8266 Ethical Curation Eurovision Event Driven Asynchronous Express Face Recognition Facebook Fake Advertising Fake News Fedora VirtualBox Fifth Doctor File transfer without iTunes FireFly Flash Flickr Fraud Freedom of Speech Front-end Development G Suite Gallifrey Gig Economy git Github GitKraken Gitlab GMAIL Google Google Adsense Google Chrome Google Gnome Google+ Government Spying Great Britain Green Transportation Hate Speech Heat Loss Hibernate High Technology Hoax Science Home Automation HTTP Security HTTPS Human ID I2C Protocol Image Analysis Image Conversion Image Processing ImageMagick In-memory Computing Incognito Mode InfluxDB Infrared Thermometers Insulation Internet Internet Advertising Internet Law Internet of Things Internet Policy Internet Privacy iOS iOS Devices iPad iPhone iPhone hacking Iron Man iShowU Audio Capture iTunes Janet Fielding Java JavaFX JavaScript JavaScript Injection JDBC John Simms Journalism Joyent jQuery Kaspersky Labs Kext Kindle Kindle Marketplace Large Hadron Collider Lets Encrypt LibreOffice Linux Linux Hints Linux Single Board Computers Logging Mac Mini Mac OS Mac OS X Mac Pro MacBook Pro Machine Learning Machine Readable ID Macintosh macOS macOS High Sierra macOS Kext MacOS X setup Make Money Online Make Money with Gigs March For Our Lives MariaDB Mars Mass Violence Matt Lucas MEADS Anti-Missile Mercurial MERN Stack Michele Gomez Micro Apartments Microsoft Military AI Military Hardware Minification Minimized CSS Minimized HTML Minimized JavaScript Missy Mobile Applications Mobile Computers MODBUS Mondas Monetary System MongoDB Mongoose Monty Python MQTT Music Player Music Streaming MySQL NanoPi Nardole NASA Net Neutrality Network Attached Storage Node Web Development Node.js Node.js Database Node.js Performance Node.js Testing Node.JS Web Development Node.x North Korea npm NSA NVIDIA NY Times Online advertising Online Community Online Fraud Online Journalism Online News Online Photography Online Video Open Media Vault Open Source Open Source and Patents Open Source Governance Open Source Licenses Open Source Software OpenAPI OpenJDK OpenVPN Palmtop PDA Patrick Troughton PayPal Paywalls Personal Flight Peter Capaldi Peter Davison Phishing Photography PHP Plex Plex Media Server Political Protest Politics Postal Service Power Control President Trump Privacy Private E-mail server Production use Public Violence Raspberry Pi Raspberry Pi 3 Raspberry Pi Zero ReactJS Recaptcha Recycling Refurbished Computers Remote Desktop Removable Storage Renewable Energy Republicans Retro Computing Retro-Technology Reviews RFID Rich Internet Applications Right to Repair River Song Robotics Robots Rocket Ships RSS News Readers rsync Russia Russia Troll Factory Russian Hacking Rust SCADA Scheme Science Fiction SD Cards Search Engine Ranking Search Engines Season 1 Season 10 Season 11 Security Security Cameras Server-side JavaScript Serverless Framework Servers Shell Scripts Silence Simsimi Skype SmugMug Social Media Social Media Networks Social Media Warfare Social Network Management Social Networks Software Development Software Patents Space Flight Space Ship Reuse Space Ships SpaceX Spear Phishing Spring Spring Boot Spy Satellites SQLite3 SSD Drives SSD upgrade SSH SSH Key SSL Stand For Truth Strange Parts Swagger Synchronizing Files Tegan Jovanka Telescopes Terrorism The Cybermen The Daleks The Master Time-Series Database Tom Baker Torchwood Total Information Awareness Trump Trump Administration Trump Campaign Twitter Ubuntu Udemy UDOO US Department of Defense Video editing Virtual Private Networks VirtualBox VLC VNC VOIP Vue.js Walmart Weapons Systems Web Applications Web Developer Resources Web Development Web Development Tools Web Marketing Webpack Website Advertising Website Business Models Weeping Angels WhatsApp William Hartnell Window Insulation Windows Windows Alternatives Wordpress World Wide Web Yahoo YouTube YouTube Adpocalypse YouTube Monetization