; Date: Mon Aug 24 2020

Tags: Node.JS »»»» Front-end Development »»»» Bootstrap

In this project we'll create a small TODO application using Node.js 14.x, the latest version of Express, Bootstrap v5, Bootstrap Icons, Sequelize for the database, and Socket.IO for multi-user real time interaction. In this posting we'll initialize the project directory, and put together the initial code.

This is part 1 of a 4 part series in which we will build a simple TODO application. For an overview of the project, and index of the articles, see: Single page multi-user application with Express, Bootstrap v5, Socket.IO, Sequelize

The application demonstrates how to build a pseudo-real-time multi-user web app using Node.js and Socket.IO. Along the way we are exploring the latest Node.js release (v 14.x) and some of its new features.

In this tutorial we will do the initial setup of the project directory for our Node.js/Express application, and the support for Bootstrap, Bootstrap Icons, Sequelize, and Socket.IO. That means using npm to initialize a package.json, and installing the required packages. Then we'll create the main scripts, app.mjs, approotdir.mjs and appsupport.mjs, in which we "wire up" the Express application and all the parts.

In passing we will look at using the await keyword at the top level of a Node.js module. This is a new feature added to Node.js in version 14.8, and is an important advance since previously using await had to happen inside an async function. We can do it now, but there is a special consideration for error and exception handling.

The full source code for this project is at: https://github.com/robogeek/nodejs-express-todo-demo ... To use the application, clone the repository, and follow further directions in the project README. It's very simple to run, and there are more directions in Running the TODO application

Before starting this project make sure to have Node.js 14.x installed on your laptop. As of this writing, version 14.8 is the current release. If you prefer to use Yarn rather than npm for package management, you'll have to translate the npm commands to Yarn commands.

To start with, create a directory named something like todo, then run:

$ mkdir todo
$ cd todo
$ npm init -y

This gives you a blank project with a default package.json.

Next let's install required packages:

$ npm install cookie-parser cross-env cross-var \
    express-session express-session-sequelize helmet \
    js-yaml morgan socket.io socket.io-redis nunjucks sequelize sqlite3 \
    bootstrap@next popper.js@1.x bootstrap-icons --save

Maybe I'm old fashioned, but I prefer to explicitly use the --save option rather than have npm automatically act as if I wanted the package automatically saved as a dependency. In any case this sets up the required modules for:

• ExpressJS v4
• Bootstrap v5 (as of this writing we must specify bootstrap@next for that purpose)
• The required Popper.js dependency, as well as the current version of Bootstrap Icons
• Sequelize for handling databases
• Socket.IO for handling pseudo-real-time interaction with the client code
• The YAML parser, which we'll use to read a configuration file for Sequelize configuration
• The Nunjucks template engine, which we'll use both in server-side and browser-side template processing
• A few extra packages that are useful for Express development

Some of these packages do not contain server-side code for execution on Node.js. Instead they contain browser-side code, and are distributed through the npm repository for convenience.

For example:

$ ls node_modules/bootstrap/dist
css js
$ ls node_modules/bootstrap-icons/
LICENSE.md  README.md  bootstrap-icons.svg  icons   package.json
$ ls node_modules/popper.js/
README.md  dist  index.d.ts  index.js.flow  package.json  src
$ ls node_modules/nunjucks/
CHANGELOG.md  LICENSE  README.md  bin  browser  index.js  package.json src

Therefore, in node_modules/bootstrap/dist/css is the Bootstrap CSS files, and node_modules/bootstrap/dist/js is the JavaScript files. Likewise node_modules/bootstrap-icons/icons/ contains the collection of SVG files. Similarly, node_modules/popper.js/dist contains the Popper.js browser-side library, and node_modules/nunjucks/browser contains the browser side of the Nunjucks library.

We'll see shortly how to use these libraries in an Express-based application. But it's worthy first of discussing why we're installing these libraries in this way.

The Bootstrap project recommends using libraries from the corresponding CDN's. While that would be easy, ask yourself what happens if the CDN goes down? Is it a good idea to have a dependency on a 3rd party service? If that 3rd party service dies, then your application dies, and your users won't care that it's not your fault.

In this case it seems like a best practice to host all such dependencies on your own infrastructure.

Setting up an Express application for use with Bootstrap v5 et al

The initial application setup is very simple, comprising three files.

Because Node.js 14 has native support for ES6 modules, we will use them exclusively in this application.

Start with a file named approotdir.mjs containing:

import * as path from 'path';
import * as url from 'url';

const __filename = url.fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
export const approotdir = __dirname;

In Node.js, the traditional CommonJS-based module format came along with a pair of very useful variables, __dirname and __filename. When Node.js executes ES6 modules those variables are not available.

Instead the import.meta object contains import.meta.url which is the file:// URL for the module. From that we can calculate __dirname and __filename. Namely we can take apart that URL to get the __filename then use path.dirname to compute __dirname.

So.. what is the primary need for the __dirname variable in the first place? Node.js applications often need to compute a file name relative to the project root directory. This file, approotdir.mjs, is in the project root directory, and therefore its __dirname is the same as the pathname of the project directory. Therefore this module exports approotdir so that other modules can calculate file names.

That's one module down, and two to go. Let's do both, app.mjs and appsupport.mjs, simultaneously.

In app.mjs start with this:

import { default as express } from 'express';
import { default as DBG } from 'debug';
const debug = DBG('todos:debug');
const dbgerror = DBG('todos:error');
import { default as logger } from 'morgan';
import { default as cookieParser } from 'cookie-parser';
import { default as bodyParser } from 'body-parser';
import helmet from 'helmet';
import * as http from 'http';
import * as path from 'path';
import { default as nunjucks } from 'nunjucks';
import { approotdir } from './approotdir.mjs';
const __dirname = approotdir;
import {
    normalizePort, onError, onListening, handle404, basicErrorHandler
} from './appsupport.mjs';

import { 
    router as indexRouter,
    init as homeInit
} from './routes/index.mjs';
import { connectDB } from './models/sequlz.mjs';

import socketio from 'socket.io';

This is every required import for the main module. In structuring an Express application, I prefer to have one module (app.mjs) where the purpose is solely to wire everything together. This module will contain no functions of its own, but instead call functions exported from other modules.

In appsupport.mjs we'll have a few functions that are useful for managing an Express application. In routes/index.mjs we'll have the Express router functions required for the home page of the application, as well as server-side Socket.IO functions to drive the application. And, in models/sequelz.mjs we'll have database interface code.

export const app = express();

export const port = normalizePort(process.env.PORT || '3000');
app.set('port', port);

export const server = http.createServer(app);

server.listen(port);
server.on('request', (req, res) => {
    debug(`${new Date().toISOString()} request ${req.method} ${req.url}`);
});
server.on('error', onError);
server.on('listening', onListening);

This sets up the Express application and the HTTPServer. There is a debugging printout of every request on the server. And we get the listening on message as well as any errors that happen to the server.

This relies on a couple appsupport.mjs functions, so let's start that file now:

import util from 'util';
import { server, port } from './app.mjs';
import { default as DBG } from 'debug';
const debug = DBG('todos:debug');
const dbgerror = DBG('todos:error');

export function normalizePort(val) {
    const port = parseInt(val, 10);

    if (isNaN(port)) {  // named pipe
        return val;
    }

    if (port >= 0) {   // port number
        return port;
    }

    return false;
}

The PORT variable could be specified with a numerical port number, or the path name of a named pipe. This handles both cases.

export function onError(error) {
    dbgerror(error);
    if (error.syscall !== 'listen') {
        throw error;
    }

    const bind = typeof port === 'string'
        ? 'Pipe ' + port
        : 'Port ' + port;

    // handle specific listen errors with friendly messages
    switch (error.code) {
        case 'EACCES':
            console.error(`${bind} requires elevated privileges`);
            process.exit(1);
            break;
        case 'EADDRINUSE':
            console.error(`${bind} is already in use`);
            process.exit(1);
            break;
        default:
            throw error;
    }
}

The onError function handles a few useful error conditions with useful messages.

export function onListening() {
    const addr = server.address();
    const bind = typeof addr === 'string'
        ? 'pipe ' + addr
        : 'port ' + addr.port;
    debug(`Listening on ${bind}`);
}

This function gives us a nice user-friendly printout of the initial state for the application.

Since the setup of the bind variable is duplicated in two functions we should think about moving that to a top-level external function. Like:

const describeAddress = (addr) => {
    return typeof addr === 'string'
        ? 'pipe ' + addr
        : 'port ' + addr.port;
};

Back in app.mjs, add this:

export const io = socketio(server);

nunjucks.configure('views', {
    autoescape: true,
    express: app
});
app.set('view engine', 'njk')

app.use(logger(process.env.REQUEST_LOG_FORMAT || 'dev', {
    // immediate: true,
}));
// app.use(helmet());

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());

This initializes Socket.IO, attaching it to the HTTPServer. Afterward, the Socket.IO server-side code will intercept and handle certain URL's.

Next, we configure Nunjucks as the default Express rendering engines. We'll use .njk as the file name for Nunjucks templates.

Next we configure the Morgan logger, and we can use an environment variable to configure the logging format.

Finally we add a few useful middleware functions. One we've skipped over is Helmet, which is a framework for adding security features. By all means look up how to configure this since it is an extremely useful module.

The next task is setting up paths for some static files:

app.use(express.static(path.join(__dirname, 'public')));

app.use('/assets/vendor/bootstrap/js', express.static(
    path.join(__dirname, 'node_modules', 'bootstrap', 'dist', 'js')));
app.use('/assets/vendor/bootstrap/css', express.static(
    path.join(__dirname, 'node_modules', 'bootstrap', 'dist', 'css')));
app.use('/assets/vendor/bootstrap/icons', express.static(
    path.join(__dirname, 'node_modules', 'bootstrap-icons', 'icons')));
app.use('/assets/vendor/nunjucks', express.static(
    path.join(__dirname, 'node_modules', 'nunjucks', 'browser')));

app.use('/assets/vendor/popper.js', express.static(
    path.join(__dirname, 'node_modules', 'popper.js', 'dist', 'umd')));

Express has a module, express.static, that lets you serve static files from the file system. We're using this feature to supply the Bootstrap, Popper, Nunjucks libraries to the browser, as well as our custom JavaScript that will be in the public directory.

You might see a recommendation to not do this, and instead serve the static files through another server. In theory Node.js/Express won't offer good performance for delivering these files to the browsers. Often projects set up another server, like NGINX, to handle these files. But for a demo application like this, we can easily serve those files using Express like this.

In our HTML we will use a tag like this to access the files:

<link rel="stylesheet" href="/assets/vendor/bootstrap/css/bootstrap.min.css">

The /assets/vendor/bootstrap/css URL translates to this directory:

path.join(__dirname, 'node_modules', 'bootstrap', 'dist', 'css')

The other directories have corresponding mappings to directories in node_modules. We've already looked in these directories to see that they do indeed contain the corresponding libraries.

In app.mjs the next thing to do is to set up the router module:

app.use('/', indexRouter);

The app.use function used this way configures an Express Router instance to handle requests on certain URL's. This Router will come from routes/index.mjs and we discuss its implementation in Using Bootstrap and Socket.IO for the user interface in an Express Node.js 14 application .

Next is these functions:

app.use(handle404);
app.use(basicErrorHandler);

These set up error handling functions for an Express application.

Let's add those functions to appsupport.mjs:

export function handle404(req, res, next) {
    const err = new Error('Not Found');
    err.status = 404;
    next(err);
}

export function basicErrorHandler(err, req, res, next) {
    // Defer to built-in error handler if headersSent
    // See: http://expressjs.com/en/guide/error-handling.html
    if (res.headersSent) {
        debug(`basicErrorHandler HEADERS SENT error ${util.inspect(err)}`);
        return next(err)
    }
    res.locals.message = err.message;
    res.locals.error = req.app.get('env') === 'development' ? err : {};

    res.status(err.status || 500);
    res.render('error');
}

This is the handler for 404 error status's, and a basic error handler. The errors this will generate are not at all user friendly. I hear that an error page depicting a flock of birds pulling a whale out of the ocean is popular.

To finish off app.mjs, add this:

export const todostore = await TodoStore.connect();
await homeInit(todostore);

Please take a minute to let this sink in. This is top-level code in a Node.js ES6 module, and this top-level code is using await. Until now that was impossible. In Node.js 14.7 and earlier using await at the top level of a module threw an error reading SyntaxError: Unexpected reserved word. With Node.js 14.8, using await at the top-level of a Node.js module became possible.

It looks so natural and serene there, as if that's the way it is supposed to be.

The first function, TodoStore.connect, connects to the database, which is an asynchronous operation. It returns a TodoStore instance, which has functions for interacting with the database. Likewise, the second function initializes some code in routes/index.mjs, providing the TodoStore instance, and that initialization is also asynchronous. See: Node.js Script writers: Top-level async/await now available

As nice as this look there is a very important consideration. What happens if either function call throws an error? Where does that error go? What does it do to the execution of the program?

One answer is to add a .catch to those calls to capture any errors. But that wouldn't look very good. Or we could throw a try/catch around them, which would look okay.

But there is another way to handle these errors, and it's something we should do anyway. In Node.js there is the requirement to create handler functions for the uncaughtException or unhandledRejection errors. Either will be thrown if an Exception or Rejected Promise occurs that is not correctly handled.

For that purpose, we need to add this in appsupport.mjs:

process.on('uncaughtException', function(err) { 
    console.error("I've crashed!!! - "+ (err.stack || err)); 
});

process.on('unhandledRejection', (reason, p) => {
    console.error(`Unhandled Rejection at: ${util.inspect(p)} reason: ${util.inspect(reason)}`);
});

Both of these handle unhandled errors. It is extremely important to handle these errors, if for no other reason than the Node.js team is threatening to cause both of these to simply crash the application if they are not handled.

Then, we have a few more niceties to add in appsupport.mjs:

import { close as TodoClose } from './models/sequlz.mjs';

async function catchProcessDeath() {
    debug('urk...');
    await TodoClose();
    await server.close();
    process.exit(0);
}

process.on('SIGTERM', catchProcessDeath);
process.on('SIGINT', catchProcessDeath);
process.on('SIGHUP', catchProcessDeath);

process.on('exit', () => { debug('exiting...'); });

The process.on invocations set up handlers on events emitted from the Process object. If the process needs to die, we can capture that and print a useful message. We can also make sure to "close" the database connection, since for some database engines it is important to close the connection.

Summary

We have covered a lot of ground in this tutorial. We've learned how to set up the starting point of an Express application, and prepared the way for the other application components. The application implements a few best practices, such as minimizing the number of 3rd party dependencies, and handling the uncaught errors.

But, we do not have a complete application at this time. The remaining tasks are covered in other tutorials in this series:

To learn about the database layer: Using Sequelize for the database layer in an Express Node.js 14 application

To learn about the Bootstrap user interface: Using Bootstrap and Socket.IO for the user interface in an Express Node.js 14 application

To learn about running the Todo application: Running the TODO application

To return to the project overview: Single page multi-user application with Express, Bootstrap v5, Socket.IO, Sequelize