; Date: Wed Feb 02 2022

Tags: Node.JS »»»» TypeScript

The core unit of software on Node.js is the package. Therefore using TypeScript for Node.js applications requires learning how to create packages written in TypeScript or sometimes a hybrid of JavaScript and TypeScript. Fortunately this is easy for any Node.js programmer, and amounts to setting up the TypeScript compiler.

Creating a Node.js package from TypeScript code is roughly the same as creating one from JavaScript code. Because TypeScript is compiled to JavaScript, the primary consideration is configuring the compiler to fit your needs. In turn, that depends on the Node.js release you're targeting, as well as the choice between ES6 or CommonJS modules (or both).

Recall that a Node.js package is a directory hierarchy containing:

• A package.json file describing the package
• Code files that are distributed in the package

For this article we're concerned about packages for execution on Node.js. That entails converting TypeScript code to JavaScript code, as just mentioned.

This article is loosely derived from my book: Quick Start to using Typescript and TypeORM in Node.js applications (sponsored link)

Choosing which module format (CommonJS/ES6) and other specifics of target Node.js version

The needs of your application dictate which Node.js versions you'll support. For example, Node.js has not always supported ES6 modules, nor has it always supported async/await functions. Those needing to support old versions of Node.js, will need to target generated code for their preferred Node.js version.

One advantage of TypeScript is that we can code in ultra-modern JavaScript-like constructs, but the compiler can be configured to translate that for any JavaScript environment. That includes the really old implementations.

But, since Node.js is now 2+ years into supporting async/await and ES6 modules, most of us will be coding on at least Node.js 14 if not version 16.

Your project Architect is probably advocating for adption of ES6 modules. That's a wise choice, since the support for this is improving with every Node.js release. But, unfortunately you'll often run into issues since there are many available packages that don't work well when used from an ES6 module. You will sometimes need to generate CommonJS modules from your TypeScript.

We are still at a crossroads moment where many packages are still written for the old days of callback functions in CommonJS modules. There are many popular packages written in that era, which aren't being maintained, which still happen to work. Some of those simply do not work well with ES6 modules.

The recommended best practice is:

• Generate code for ES6 modules
• Generate code for recent platform versions, so you can use async/await and other features

In other words, the best practice is to generate code for the most modern form of JavaScript supported by your target environment.

What's the difference between CommonJS and ES6 module formats?

• CommonJS
  • Uses require to load modules, and this is a regular function that can be used anywhere
  • A CommonJS module can import an ES6 module using the import function
  • JavaScript file names are usually .js, but this can change based on configuraton, and can be .cjs
  • Supports __dirname and __filename global variables
  • Functions and data are exported by assigning to module.exports
  • CommonJS modules are only supported on Node.js
• ES6 modules
  • Uses import to load modules, and this is a statement that can only execute at the top level
  • The import statement supports loading both ES6 and CommonJS modules
  • JavaScript files are usually .mjs, but can be .js depending on configuration
  • Does not support __dirname and __filename, but these values can be computed from module.meta.url
  • Functions and data are exported with the export statement
  • The ES6 Module module format is meant to be universal to all ECMAScript environments.

While the two formats are roughly similar, those differences can be significant.

As you study your application, you may find it impossible to generate code in ES6 module format, and instead be required to use CommonJS. The project Architect who keeps opining about ES6 modules being the future will have to wait awhile until things settle out.

What we'll do here is show the code generation options and explain how to use them. It's up to you to decide what's best for your application.

Generic TypeScript project setup for Node.js

To explore this, let's create a simple Node.js project directory for TypeScript. For a complete discussion of setting up a Node.js project for TypeScript, see: Setting up the Typescript compiler to integrate with Node.js development

Create a directory ts-example, and in that directory another named package. We'll use the package directory to store a simple package, then create a couple directories next to it to use that package. Then run npm to initialize the project.

$ mkdir -p ts-example/package
$ cd ts-example/package
$ npm init -y

$ npm install @types/node --save-dev
$ npm install typescript ts-node --save-dev

That initializes the package.json and installs TypeScript tools. The typescript package is the compiler, and ts-node is a convenience program letting us run TypeScript scripts without compiling them. The @types/node package is a collection of useful definitions so that the TypeScript compiler knows about Node.js objects.

Create a directory, lib, which will hold our TypeScript files. In that directory create example.ts containing:

export class Something {

    constructor(length: number, width: number) {
        this.#length = length;
        this.#width = width;
    }

    #length: number;
    #width: number;

    get length() { return this.#length; }
    set length(l: number) { this.#length = l; }

    get width() { return this.#width; }
    set width(w: number) { this.#width = w; }

    area() { return this.#length * this.#width; }
    ratio1() { return this.#length / this.#width; }
    ratio2() { return this.#width / this.#length; }
}

export function mkSomething(length: number, width: number) {
    return new Something(length, width);
}

This defines a class related in some way to rectangles. The fields #length and #width use a new feature of JavaScript which makes those fields private, with protections enforced by the JavaScript runtime. Since this is a new feature, not all JavaScript versions support it. We've added some getter/setter functions and other functions for computing things about this fine object.

Create a file named tsconfig.json containing:

{
    "compilerOptions": {
        "lib": [ "es6", "es2021", "esnext" ],
        "target": "es2021",
        "module": "es2022",
        "moduleResolution": "Node",
        "outDir": "./dist",
        "rootDir": "./lib",
        "declaration": true,
        "declarationMap": true,
        "inlineSourceMap": true,
        "inlineSources": true
    }
}

This file is one way to specify TypeScript compiler options.

• The lib line describes the JavaScript language features to enable, with these settings saying to turn on all the latest features
• The target line says to output ES2021 code
• The module line describes the output module format which will be used, where es2022 ensures use of ES6 modules
• The moduleResolution parameter says to look up modules in node_modules directories just like NodeJS does.
• The outDir parameter says to compile files into the named directory, and the rootDir parameter says to compile the files in the named directory.
• The declaration and declarationMap parameters says to generate declaration files.
• The inlineSourceMap and inlineSources say to generate source-map data inside JavaScript source files.

What that means is our source code will be in lib and TypeScript will compile it into dist.

Fleshing out the package.json

While a package.json file was automatically created we must take it a little bit further. Add this:

{
    ...
    "main": "./dist/example.js",
    "type": "module",
    "types": "./dist/example.d.ts",
    "scripts": {
        "build": "tsc",
        "watch": "tsc --watch"
    },
    ...
}

By default main shows index.js but we're generating dist/example.js instead. Remember that we configured the lib directory to contain TypeScript source, and that the dist directory will contain the compiled source. Notice that we're generating code in ES6 module format, but the file extension is .js rather than .mjs.

Setting type to module causes Node.js to interpret .js files as ES6 modules.

The type field makes it clear the JavaScript code is ES6 module format.

The types field declares to the world that this module contains type definitions. It is good form to automatically generate type definitions, which the tsconfig.json file shown earlier does, and then make sure the world knows that type definitions are included.

We've included a build script to run the compiler. The watch script starts a persistent process to automatically watch for file changes and recompile.

First, run npm run build and inspect the generated code. Because of the private fields in the object, the code is bordering on gnarlyness, but it makes sense.

Next, run npm run watch. Your screen will clear and a message will be printed that it is Watching for file changes. Add a function to example.ts like this:

export function FAIL(msg: string) {
    throw new Error(msg);
}

And a message prints about File change detected and momentarily it's back to Watching for file changes. After this the files in dist will be updated with corresponding new code.

In other words, it's a big convenience so that your code is always recompiled on every edit. Combine this with another process, like Nodemon, if you're developing an application server the server will be automatically reloaded.

We have a package, and a package.json, to go further we need to write some code to exercise these things.

A simple client to our TypeScript package

Create a directory, client, next to package. Run the following commands:

$ npm init -y
$ npm install typescript @types/node --save-dev
$ npm install ../package --save
$ mkdir lib
$ cp ../package/tsconfig.json .

This sets us up to create a TypeScript file in lib using the same compiler configuration as for the package.

In the lib directory create client.ts containing this:

import * as pkg from 'ts-example';

const thing = pkg.mkSomething(5, 20);
console.log({
    width: thing.width,
    length: thing.length,
    area: thing.area(),
    ratio1: thing.ratio1(),
    ratio2: thing.ratio2()
});

The first line imports the package we created. In this case the name in its package.json was ts-example, and that's the name we've used here.

We use the mkSomething factory function to create a thing, then print its values. We're not here for the greatest of software but to explore creating packages using TypeScript.

In the generated package.json make these changes:

"main": "./dist/client.js",
"type": "module",
"scripts": {
    "build": "npx tsc",
    "watch": "npx tsc --watch"
},

We're again building TypeScript to ES6 module format. The client.ts we just created will land as dist/client.js.

Run npm run build to compile the code. Then running the application we get:

$ node dist/client.js 
{ width: 20, length: 5, area: 100, ratio1: 0.25, ratio2: 4 }

Great, it worked. So let's examine what's here.

Examining the Node.js packages generated from TypeScript

In the package directory we have these files:

$ tree -I node_modules .
├── dist
│   ├── example.d.ts
│   ├── example.d.ts.map
│   └── example.js
├── lib
│   └── example.ts
├── package.json
├── package-lock.json
└── tsconfig.json

2 directories, 7 files

And in the client directory we have:

$ tree -I node_modules .
.
├── dist
│   ├── client.d.ts
│   ├── client.d.ts.map
│   └── client.js
├── lib
│   └── client.ts
├── package.json
├── package-lock.json
└── tsconfig.json

2 directories, 7 files

In each case tsconfig says to generate ES6 modules, and package.json has type set to module to match. This setting tells Node.js to treat .js files as ES6 modules, as if they had the .mjs extension.

In the client directory look at its node_modules directory:

$ ls -l node_modules
lrwxrwxrwx 1 david david   13 Feb  1 17:34 ts-example -> ../../package
drwxrwxr-x 3 david david 4096 Feb  1 17:34 @types
drwxrwxr-x 5 david david 4096 Feb  1 17:34 typescript

The symbolic link is because of how the package was installed. This is how npm installs a package when the dependency uses a file: URL. This is a convenience during development because we can recompile package and the change is automatically available in client.

Switching from ES6 module format to CommonJS module format

Suppose you want the package directory to use CommonJS module format?

All that's required is change module in tsconfig.json to commonjs, and in package.json change type to commonjs. You tne recompile package, and verify that the files in dist are now in CommonJS format. Rerun the client program and it'll still work.

In plain Node.js, an ES6 module is able to use import to load either ES6 or CommonJS modules. That's what we just did with client in ES6 format loading the CommonJS module in package.

Now, in package switch the tsconfig.json and package.json back to using ES6 modules. And in client switch the tsconfig.json and package.json to use CommonJS modules. That makes the package directory in ES6 module format, and client in CommonJS module format.

Rebuild (npm run build) in both directories, then rerunning client gives us this:

$ node dist/client.js /home/david/Projects/ws/techsparx.com/projects/node.js/ts-example/client/dist/client.js:3
const pkg = require("ts-example");
            ^

Error [ERR_REQUIRE_ESM]: require() of ES Module /home/david/Projects/ws/techsparx.com/projects/node.js/ts-example/package/dist/example.js from /home/david/Projects/ws/techsparx.com/projects/node.js/ts-example/client/dist/client.js not supported.

In plain Node.js, a CommonJS module is unable to use require to load an ES6 module. That's what has happened in this case. Instead, CommonJS modules must use the import() function for loading ES6 modules. But, the TypeScript compiler isn't generating the code that way.

Creating a hybrid CommonJS/ES6 module using TypeScript

Starting with Node.js 14 we can create hybrid modules by putting the correct declaration in package.json. A hybrid module contains both CommonJS and ES6 code, and uses a new feature of package.json to direct Node.js which version to load.

In package/package.json the hybrid module is configured this way:

"exports": {
    "require": "./dist-cjs/example.js",
    "import": "./dist-es6/example.js"
},
"main": "./dist-cjs/example.js",
"types": "./dist-cjs/example.d.ts",
"type": "commonjs",

The primary purpose for the exports tag is declaring the entry points to the package. In this case we're using only one entry point, the main (and only) module in the package. The other purpose for exports is to declare which file to load if the package is being loaded by require or by import. In this case, if the package is being loaded by using require, then to use dist-cjs/example.js. If it's loaded using import, to instead use the file in dist-esm.

To build this we need to run tsc twice, once with a tsconfig for CommonJS, and again with a tsconfig for ES6.

In the package directory run these commands:

$ mv tsconfig.json tsconfig-es6.json 
$ cp tsconfig-es6.json tsconfig-cjs.json

This sets up two tsconfig files. In tsconfig-cjs.json change module to commonjs, and change outDir to ./dist-cjs. In tsconfig-es6.json, the module setting is already es2022, so change outDir to ./dist-es6. Then in package.json make an additional change:

"scripts": {
      "build:es6": "npx tsc -p tsconfig-es6.json",
      "build:cjs": "npx tsc -p tsconfig-cjs.json",
},

We now have two build scripts. The -p option tells the compiler to use the selected configuration file. Run these commands:

$ rm -rf dist

$ npm run build:es6

> ts-example@1.0.0 build:es6
> npx tsc -p tsconfig-es6.json

$ npm run build:cjs

> ts-example@1.0.0 build:cjs
> npx tsc -p tsconfig-cjs.json

$ tree -I node_modules .
.
├── dist-cjs
│   ├── example.d.ts
│   ├── example.d.ts.map
│   └── example.js
├── dist-es6
│   ├── example.d.ts
│   ├── example.d.ts.map
│   └── example.js
├── lib
│   └── example.ts
├── package.json
├── package-lock.json
├── tsconfig-cjs.json
└── tsconfig-es6.json

3 directories, 11 files

We have both dist-cjs and dist-es6 built. Examine the files in each, and you'll see it's roughly the same code, but presented using the corresponding module format.

In the client directory, make sure tsconfig.json and package.json is configured for CommonJS modules. Compile and run the program:

$ node dist/client.js
CJS 5 20
{ width: 20, length: 5, area: 100, ratio1: 0.25, ratio2: 4 }

And, yes, it still works. To get the CJS output, we edited dist-cjs/example.js with a console.log invocation, and in dist-es6 we did the same but have it print ES6 instead. This shows CommonJS code loading CommonJS code from a hybrid module.

In client, edit tsconfig.json and package.json to use ES6 module format instead. Recompile, then rerun the application:

$ node dist/client.js 
(node:21428) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
.../ts-example/package/dist-es6/example.js:13
export class Something {
^^^^^^

SyntaxError: Unexpected token 'export'

We have an ES6 module using import to try to load this hybrid package. Notice that the error refers to dist-es6/example.js meaning that Node.js did choose the correct file to load. What went wrong?

The problem here is that the package.json says the package type is commonjs, but our ES6 module has the extension .js. When in CommonJS mode, Node.js expects .js to mean a CommonJS module.

Carefully reading the Node.js documentation we learn that what's needed is for the "nearest" package.json to have type of module. What that means is, another new feature is that Node.js allows a package to have multiple package.json files, and that putting one in a child directory affects files in that sub-hierarchy.

Therefore, we can create dist-es6/package.json containing this:

{
    "type": "module"
}

As the nearest package.json to the files in dist-es6, this file affects how Node.js treats those files. This file therefore overrides the type setting in the main package.json so that the files in dist-es6 can be interpreted as ES6 modules.

Rerun the client:

$ node dist/client.js 
ES6 5 20
{ width: 20, length: 5, area: 100, ratio1: 0.25, ratio2: 4 }

And, it now works correctly, and prints the ES6 message we inserted into the code.

Generating a hybrid ES6/CommonJS module using TypeScript

Let's review what we did to build a hybrid ES6/CommonJS package using TypeScript.

1. Using two tsconfig files, one configured to generate ES6 code, the other configured to generate CommonJS code
2. Using two scripts entries, each to run tsc -p tsconfig-TYPE.json
3. Configuring the main package.json to use the hybrid exports feature
4. Configuring the main package.json with type of commonjs
5. Adding a second package.json that sets type of module

The last step is at the moment manual. The build:es6 tag should be refactored to copy a package.json into the dist-es6 directory.

This approach would be used for any situation where we're targeting multiple platforms from the same TypeScript code. We simply create multiple tsconfig files, one for each target, and rerun tsc for each target.

But, carefully reading the Node.js documentation we learn that this is not recommended.

Why shouldn't we use this hybrid module approach?

The Node.js documentation points out a flaw in this approach. It's possible for the same application to load both the CommonJS and ES6 version of the package. As the documentation says:

This potential comes from the fact that the pkgInstance created by const pkgInstance = require('pkg') is not the same as the pkgInstance created by import pkgInstance from 'pkg' (or an alternative main path like 'pkg/module'). This is the “dual package hazard,” where two versions of the same package can be loaded within the same runtime environment.

Depending on the package, this can lead to problems. For example, some packages store internal state, and then there would be two internal states for what should be the same package. Another risk is whether TypeScript generates bugs in one module format that do not exist in the other module format.

What's recommended instead is to use a wrapper module.

Implementing an ES6 wrapper module for a CJS module

To understand what that means, create a directory package/wrapper-es6 containing example.mjs, containing this:

export { 
    Something, mkSomething, FAIL
} from '../dist-cjs/example.js';

This is called re-exporting. This exports from this module the named items from the named module.

You then configure package.json to use wrapper-es6/example.mjs for the import side of the exports tag.

"exports": {
      "require": "./dist-cjs/example.js",
      "import": "./wrapper-es6/example.mjs"
},
"main": "./dist-cjs/example.js",
"types": "./dist-cjs/example.d.ts",
"type": "commonjs",

In client, with it configured for ES6 module format, rerun the code:

$ node dist/client.js 
CJS 5 20
{ width: 20, length: 5, area: 100, ratio1: 0.25, ratio2: 4 }

It works, and the message we left in the code shows that the CommonJS module expected.

Summary

We have learned how to generate either CommonJS or ES6 modules from TypeScript source. And we've learned how to generate type files for either. With a simple reconfiguration, we can easily switch between the two.