Pages with tag Node.js

A brief look at Sequelize, an ORM for Node.js with MySQL, PostgreSQL or SQLITE3 Want to do some database code but not think too much about it?  Such as, avoid SQL?  You can have your SQL and a simplified model of your database thanks to a module I just found for Node.js called Sequelize.  It adds an ORM-like layer on top of MySQL, PostgreSQL or SQLITE3, allowing you to do database interactions using JavaScript code rather than SQL.  It's fairly nice and easy to use, however I think it's likely there are some limitations to the complexity of what you can do with Sequelize.
A deep dive guide to JavaScript Symbols (ES-6 and beyond)

ES2015 added a ton of useful new features, and among them is the Symbol type. Symbol lets us avoid property name collisions in a way where we can almost implement private properties in JavaScript objects.

A file-system browser component for Electron/Vue.js applications Many kinds of applications need to browse files in the file system. For example most programmers text editors (I'm writing this in Microsoft Visual Studio Code) include a sidebar showing the file-system. Therefore it's important to have such a component to use in Electron/Vue.js applications. In this article we'll look at one way to implement such a component.
A popular Node.js package, request, has been deprecated What do we do when a venerated elder is supplanted by young whipper-snappers? Node.js is barely 10+ years old, and the availability of async/await and other JavaScript features means older packages must either adapt or be replaced. The community around the request package chose the latter, to let Request become deprecated, and to officially recommend folks to use other packages.
A recommended Node.js/Express application folder structure

Neither Node.js nor ExpressJS have an opinion about the best folder structure. As a non-opinionated application framework, Express doesn't much care how you wish to organize your code. Since the folder organization is up to you, please consider the following as one among many structures that at least one programmer thinks is an efficient way to organize Express application code. The structure here more-or-less follows the Model-View-Controller paradigm, meaning we have separated the code for data models and data persistence from the business logic from the view code.

A simple multi-tier Node.js and Nginx deployment using Docker In a world of Microservices, our Docker deployments are probably multiple small containers each implementing several services, with an NGINX container interfacing with the web browsers. There are many reasons for doing this, but lets instead focus on the how do we set up NGINX and NodeJS in Docker question. It is relatively easy, and to explore this we'll set up a two-container system with an NGINX container and a simple NodeJS app standing in as a back-end service.
Asynchronous array operations in ES7 JavaScript async/await functions

A glaring problem with the most excellent async/await feature in JavaScript is all the existing coding structures that only work with synchronous code. In an async/await function it's trivially easy to write a simple "for" loop that contains whatever async/await calls you want. But as soon as you try "Array.forEach" with an async callback function it will fail miserably. That's because this function, like so many existing interfaces which take a callback function, assumes the callback is synchronous. Instead of waiting for the Promise to resolve, it plows on ahead executing the next iteration.
Autogenerating TypeScript types and data validation for OpenAPI Schemas OpenAPI lets one describe web API methods and data types in a language-neutral format. While we can manually write code matching the schema, automatic code generation is more agile. This article focuses on autogeneration of data types and data validation for TypeScript on Node.js.
Automatically handling runtime data validation in TypeScript TypeScript does excellent compile time type checking. But that does not mean TypeScript code ensures runtime type or data validation or checking. TypeScript compiles code to JavaScript with no runtime checking, leaving the door open to runtime data problems.
Automating Node.js/JavaScript code quality checks with eslint

With eslint, coders can save a lot of time (in theory) by checking code for simple (but hard to find) errors. Take a function where you typed one parameter name, and used a different name, and then your unit testing did not touch that function. You'll find that error after a customer reports a bug. With eslint such an error can be caught before shipping the code.

Avoid killing performance with asynchronous ES7 JavaScript async/await functions

While async functions in JavaScript are a gift from heaven, application performance can suffer if they're used badly. The straightforward approach in using async functions in some cases makes performance go down the drain. Some smart recoding can correct the problem, unfortunately at the cost of code clarity.  
Avoiding rogue Node.js packages by using good version dependencies in package.json Several times now, widespread failures in Node.js applications were due to Node.js packages containing malware. If bad code makes it into our applications, what sort of trouble can ensue? That bad code has the same privileges your application has. What if the bad code isn't immediately obvious, but is stealthy malware that steals data?
Books and videos so you can easily learn Node.js programming

Have you heard about Node.js, but aren't sure what it is? Maybe you know about this exciting new software development platform, Node.js, but don't know where to start learning it? Maybe someone told you Node.js is JavaScript for server-side environments, which left you extremely confused? Or maybe you've never heard of server-side-JavaScript, and think JavaScript only runs inside web pages? The following resources will help you learn how to develop in JavaScript not inside a web browser, but on the server, using the Node.js platform.

Node.js is an exciting new platform for developing web applications, application servers, any sort of network server or client, and general purpose programming. It is designed for extreme scalability in networked applications through an ingenious combination of server-side JavaScript, asynchronous I/O, and asynchronous programming. Its claimed that the event-driven architecture gives a low memory footprint, high throughput, a better latency profile under load, and a simpler programming model. All this makes it an attractive platform for web application development.
Build A Restful Api With Node.js Express & MongoDB We are going to cover how to create a restful api using node.js express and mongodb together with mongoose. REST apis help us decouple our backend code from our front end so we can use it across multiple application (mobile apps, web apps, etc). We are going to learn how to build a simple blog post type api with all the useful methods(GET,POST,DELETE,PATCH).
Building Node.js REST API servers from OpenAPI specifications OpenAPI specifications are meant to assist creating either server or client API implementations. In this article we will explore quickly implementing server code from an OpenAPI spec, running on Node.js, preferably using TypeScript.
Bun is baking its way towards Node.js compatibility Following up on earlier Bun testing, as of v0.1.8, it is improving rapidly, but is still missing important functionality.
COMET as a justification for using Node.js? A lot of excitement is circulating around Node.js. To an extent this strikes me as symptomatic of a pattern in internet software development where a flashy new idea comes along, a bunch of leading edge thinkers get excited and start using and promoting it, there's a wave of excitement, and a call to rewrite everything under the sun using this new bit of flashy technology. But is the reinvestment of rewriting everything using that new piece of technology worth the result? That is, will Node.js be the death of LAMP? Who knows about whether the excitement around Node.js will lead to that result, but in researching and evaluating Node.js I've come across a specific use case that directly calls for server architecture identical with Node.js. There's an exciting class of web applications which use asynchronous browser updates to increase interactivity, that as we'll see in a minute require an event driven server architecture which seems well suited to Node.js. Leaving aside the question of a general purpose Node.js based server stack, let's look at COMET style applications.
Can I monkey patch a Node.js module installed from the npm repository so the patch is maintained after npm install?

Suppose you've found a Node.js module that almost does what you want, but it needs another feature or two, or some other tweak? How to modify the package, and perhaps temporarily modify it at runtime? Technically, monkey patching is not simply maintaining a local modification to someone elses code, but to dynamically modify that code at runtime. Some say this is evil, but it is a practice that is required at times. Let's see how to do this.

Complete guide to using ES6 modules to create Node.js packages that are easily usable from CJS modules It is very tempting to write all JavaScript code on Node.js with ES6 modules. But that creates a problem of interoperability with the large number of existing CommonJS (CJS) modules. Namely, it is difficult to use ES6 modules from a CJS module. Node.js makes it easy to use both CJS and ES6 modules from an ES6 module, and CJS modules can easily use CJS modules, but a CJS module using an ES6 module is painful. At the same time ES6 modules offer enough attractive benefits to warrant rewriting our code to switch from CJS to ES6 modules. This means we need a strategy for easily using Node.js ES6 modules from CJS modules.
Complete guide to using TypeORM and TypeScript for data persistence in Node.js module TypeORM is an advanced object-relations-management module that runs in Node.js. As the name implies, TypeORM is meant to be used with TypeScript. In this article we'll learn about using TypeORM to set up Entity objects to store data in a database, how to use a CustomRepository instance to manipulate a database table, and to use Relations between Entity instances to simulate database joins.
Convert JPG's into PNG's using Node.js and ImageMagick - Possible Photo workflow?

I'm pondering a new photography workflow where, instead of working with the JPG's directly out of the camera, that I instead convert them to PNG first.  The reasoning is that JPG is a lossy image format meaning every time you edit a JPG it loses a bit of precision, whereas PNG is a lossless image format and you don't lose precision.  That idea might not be the best, I haven't checked how well Picasa works with PNG's, in any case it let me play with what kind of image manipulation we can do in Node.js.
Convert SVG to PNG in Node.js using Sharp, no headless browser in sight SVG is an excellent portable, XML-based, graphics format that lets us show high fidelity graphics in modern browsers without requiring large image files. It works by using graphics operations written in XML tags. But, sometimes we need to convert an SVG to an image format (PNG, JPG, etc) in Node.js for use in non-browser contexts. Unlike most solutions to this problem, the Sharp library lets you do this without using a headless browser, decreasing the overhead of SVG image conversion.
Correctly match URL against domain name without killing yourself with regular expressions

The Internet relies on domain names as a more user-friendly humane address mechanism than IP addresses. That means we must often write code checking if a URL is "within" domain A or domain B, and to act accordingly. You might think a regular expression is the way to go, but it has failings because while a URL looks like a text string it's actually a data structure. A domain name comparison has to recognize that it's dealing with a data structure, and to compare correctly. Otherwise a URL with domain name "loveamazon.com" might match the regular expression /amazon.com$/i and do the wrong thing.

Could Node.x unseat Node.js? Event driven asynchronous server side platform duel in the making? Node.js is getting a lot of excited attention, but is it the only way to build an event driven server? For example the other day I summarized some work by Twitter to re-architect their systems including an event driven asynchronous server, but that server was written in Java and in general the article proved that a single-language solution like Node.js might not be the best because there are plenty of useful languages besides JavaScript (see Java, Twitter, and asynchronous event driven architecture). Today on Twitter I saw an announcement of Node.x, which is a general purpose framework for asynchronous event driven programming but rather than built on top of Node.js/V8 it's built on top of the JVM. And by being on top of the JVM it immediately opens the door to a multi-language solution, where Node.js is a single language solution.
Could Storify be implemented in Drupal?
Creating either CommonJS or ES6 modules for Node.js packages using Typescript The core unit of software on Node.js is the package. Therefore using TypeScript for Node.js applications requires learning how to create packages written in TypeScript or sometimes a hybrid of JavaScript and TypeScript. Fortunately this is easy for any Node.js programmer, and amounts to setting up the TypeScript compiler.
Crunching/minifying HTML, CSS and JavaScript in Node.js with the minify module

I've just added HTML/CSS/JavaScript minification to AkashaCMS (link is external), to minimize the size of static website files rendered by AkashaCMS. This will reduce page load times by decreasing the number of bytes required to be downloaded. We humans like blank space because spreading out information makes it easier to understand. With programming languages that becomes all the white space and indenting conventions. But the white space makes no difference to the computers that interpret the web pages, in fact it makes the web browsing experience slower because it takes longer to download or execute the web pages, or the CSS or JavaScript referenced by the web pages.

Data validation decorators in TypeScript Decorators are an interesting feature of TypeScript, which may become a standard part of JavaScript. They resemble annotations in other languages, in that they're attached to the declaration of something and add behavior or information to that thing. To get some familiarity, let us use a library of data validation decorators along with TypeORM.
Debugging Syntax Error Unexpected Reserved Word in Mocha on Modern Node.js The error Unexpected Reserved Word usually means you're using await in a non-async function. But what if Mocha throws this error, citing a test suite file that's clean of such problems? To save spending fruitless hours, read this short article.
Deeper testing of Bun's performance and compatibility against Node.js Bun is a new project aiming to be compatible with Node.js, but with huge performance gains. Not even a month into public availability, and people are claiming both the Node.js and Deno projects are dead.
Deploy Ghost blogging system using Passenger on Dreamhost or other hosting providers

Ghost is a very popular blogging platform which, because it's written in Node.js, is not what you'd expect to install on a traditional web hosting provider who supports PHP. Dreamhost is one of the many dozens or hundreds of web hosting providers whose bread-and-butter is PHP application hosting. Dreamhost, for example, has a specially configured Wordpress hosting service, Dreampress, that looks very useful. However, Dreamhost also allows hosting of Node.js applications. Let's see how to get Ghost running on a Dreamhost Virtual Private Server.
Deploy Node.js application using Passenger on Dreamhost or other hosting providers

On regular web hosting PHP rules the world, and you might think Node.js has no chance. I haven't cross-checked among current web hosting providers, but Dreamhost does offer the option of hosting Ruby or Node.js applications on a VPS using Passenger. In case you've never heard of Passenger, it is an "app server" for hosting Ruby, Python or Node.js applications. Dreamhost supports all three, but for our purpose we're interested in using this to run a Node.js app.
Deploying a simple multi-tier Node.js and Nginx deployment to AWS ECS Amazon's AWS Elastic Container Service (ECS) lets us deploy Docker containers to the AWS cloud. ECS is a very complex beast to tame, but Amazon offers a method of using Docker Compose to describe an ECS Service. That hugely simplifies the task of launching tasks on AWS ECS.
Deploying an Express app with HTTPS support in Docker using Lets Encrypt SSL Getting HTTPS support in all websites is extremely important for a variety of security purposes. HTTPS both encrypts the traffic as it goes over the Internet, and it validates the website is correctly owned. Let's Encrypt is a fabulous service providing free SSL certificates, which are required to implement HTTPS. For this posting we'll be implementing a simple Express app, deploying it using Docker and Docker Compose, using Let's Encrypt to acquire the SSL certificates, and hosting the app on a cloud hosting service.
Deprecating buggy npm packages, dealing with deprecations It seems several critical npm packages deprecated older releases. Installing the old version of some packages causes npm to print a warning saying the package was deprecated, and to use a newer version of the package. Sometimes the message suggests a way to figure out where the package is being required. Due to the way an npm package can pull in other npm packages, it can be tricky to figure out where the deprecated package version is being used.
Distributing, publicly or privately, Node.js modules without using npm repository The default assumption for distributing a Node.js module is to publish it in the public npm registry. It's a simple declaration in the package.json, and then you tell your customers to simply type "npm install". The public npm registry takes care of the details, and you can even use versioning to make sure your customers use tested module versions. But what if you don't want to publish modules in the public npm registry? Maybe your modules are proprietary, or maybe the modules are too intertwined with a parent product to be of general use?
Don't rip your hair out when Vows tells you "Errored callback not fired" - try this one weird trick instead

When your vows.js based tests for a Node.js application says "Errored » callback not fired" -- well, it can be very confusing. In my case the code clearly handled all paths ensuring the Vows callback would be called. No matter how many tweaks I performed to try and catch some possible error in test or code, I couldn't figure out what caused this problem. But after some yahoogling, the answer was not only difficult to find, but surprisingly simple.
Dynamic import lands in Node.js, we can import ES6 modules in CommonJS code

ES6 modules are a cool new feature coming with Node.js 10.x, and a pre-release version available is avaible in 9.x. ES6 modules implement most of what we can do with Node.js's traditional CommonJS modules. One feature missing is the ability to dynamically select the module to load. The ES6 import statement only takes a static string. A new feature, landing in Node.js 9.7.x, adds an import() function offering a new way to load modules.

The most important feature may be that import() can be used in a CommonJS module to import an ES6 module.
Easily deploy files or directory hierarchies to a server using Grunt Something we geeks need to do all the time is deploy files between machines. Such as, deploying a directory hierarchy over to a server for staging or production use. There's a ton of ways to do this. The old-school way is a shell script with carefully crafted rsync commands. In my case I build websites using (akashacms.com) AkashaCMS and need to deploy them to the destination webserver. Until now I'd added a command to AkashaCMS solely for deployment, but I'm experimenting with Grunt to see how much can be done using the Grunt ecosystem rather than having to maintain code in AkashaCMS.
Easily export or import data between CSV files and SQLite3 or MySQL databases with Node.js

It's convenient to import a CSV file into a spreadsheet to crunch numbers. CSV's (comma-separated-values) and it's brethren (TSV, for tab-separated, etc) are widely used to deliver information. You'll see CSV downloads available all over the place, whether it's your stock broker, a bank, or a government agency, and on and on. Spreadsheet programs like LibreOffice Calc offer direct import of these files, and then as a spreadsheet you've got a zillion tools available to crunch numbers, make graphs, etc. Sometimes, though, you need to load the CSV into a database to do other sorts of work. That's what we'll look at, the twin tasks of autoloading a CSV file into a MySQL database, and of exporting data from MySQL into a CSV file.
Easily offload your CPU-intensive Node.js code with simple Express-based REST server

Node.js is horrible with CPU bound processing, supposedly. Why? Because CPU-intensive algorithms block the event loop from handling events, blocking the Node.js platform from doing its core competency. Actually, as I demonstrate in my book Node Web Development (see sidebar for link), it's possible to use "setImmediate" to dispatch work through the Node.js event loop, and perform intensive computation while not blocking the event loop. The example I chose for the book is a simplistic Fibonacci algorithm, and requesting "large" Fibonacci values (like "50") would take a loooong time. But, by recoding the Fibonacci algorithm using setImmediate, it can do calculation for any Fibonacci value without blocking the event loop.
Electron considered harmful because it rides on Chromium

Electron is perhaps the greatest gift to cross platform application development in ages. I've been working on cross platform application development for over 20 years, with over 10 years spent IN the Java SE team at Sun. Today I'm a happy Node.js developer, and have spent the last couple months learning Electron and am totally enjoying the application-writing experience in Electron. It is so easy and quick to develop desktop applications using a repurposed web browser. Development is fast and intuitive, assuming you're familiar with HTML and CSS and JavaScript website development.

But a nagging thought came to mind yesterday. That Electron application requires running a large portion of the Chrome web browser. Meaning, my laptop not only has the regular Chrome browser running, but another in GitKraken, another in Visual Studio Code (and I'd been running Atom before discovering VSCode), and yet another for the application I'm developing. That's four Chrome instances, and the result simply is not scalable. Each additional running Electron-based app causes a whole new Chrome instance to exist. Is Electron an unscalable mess?
Express enters Beta 1, proving the project is still alive Express is the most popular web application framework for Node.js, even while its development has seemingly stagnated for years. Work on the Express repository is very slow, and the Express 5.x release has been promised for years; for example 5.alpha.1 was released in November 2014. But if you look under the covers, there is significant work happening in Express sub-projects.
Fargo: a Scheme for Node.js? Node.js supports only one language! The guys who developed Node.js implemented it on top of a virtual machine which supports only one programming language - JavaScript. If they'd wanted us to use multiple programming languages they could have implemented Node.js on top of the Java/Hotspot VM and it's rich and mature support for multiple languages. But they didn't. There are a couple examples of "other" languages being used to write Node.js programs, such as CoffeeScript.
Fix NVM_NODEJS_ORG_MIRROR deprecation warning when running npm install The nvm script is an exceedingly useful tool for maintaining multiple Node.js versions while keeping your sanity. Running "npm install", I noticed a strange warning message from node-gyp about "NVM_NODEJS_ORG_MIRROR" being deprecated. Looking for the cause, I found a cause centered on nvm. Let's see how to fis this problem.
Fix Webpack's ability to load CSS while packaging an app Web applications obviously will need CSS, and therefore Webpack must support packaging applications with CSS files. However with the sample Webpack build procedure I'm using, it fails while loading a CSS file. Let us take a look at the configuration changes required to fix this problem.
Fix Webpack's ability to package an app using the mime module Application development with Vue.js, Electron, and other frameworks, requires using Webpack. Being a Webpack newbie I'm not sure what it does, but it is required for the tasks I'm doing right now. One of which failed with a strange error message while trying to build a Vue.js component which used the mime module. Namely, we have to instruct Webpack to recognize the JSON files included in the mime module.
Fixing "Maximum call stack size exceeded" in async Node.js code

I've happily used the async module for Node.js for years to simplify asynchronous operations over arrays. This is a difficulty in the Node.js paradigm, since the "normal" way to process an array is with a simple for loop. But if the operation is asynchronous the for loop has no way of knowing when to iterate to the next item, or even when processing is finished. Instead the async.eachSeries function does the trick, because your code tells async.eachSeries when to go to the next item, when there's an error, and it knows when the last item is processed, etc. It's been great.

Fixing CannotPullContainerError when deploying to AWS ECS using ECR registry The AWS Elastic Container Service is supposed to be an excellent place to deploy Docker containers at scale. But unless you're careful with the configuration it is easy to get stuck with the dreaded CannotPullContainerError. AWS's documentation about this is unclear as to the precise cure for this problem. In this post we'll go over the correct way to configure a VPC and deploy a Docker container in an ECS cluster attached to the VPC.
Generating API documentation for TypeScript packages with TypeDoc With TypeDoc, one adds structured comments (JSDoc style comments) to TypeScript source code. With data in those comments, TypeDoc generates nice API documentation you can easily publish to a website. With the addition of another tool, gh-pages, it is easy to publish the documentation to GitHub Pages.
Getting image metadata (EXIF) using Node.js We can store extra data inside images, such as geolocation coordinates, text strings, and more. While there are fancy GUI applications for dealing with this data, we sometimes need to process it in our Node.js programs. For example, a static website generator platform, like AkashaCMS, might use metadata stored in images such as to automatically provide captions, or tooltip text. What we'll do in this tutorial is explore reading metadata in Node.js.
Getting started with asynchronous read/write between Node.js 8 and MongoDB

Node.js and MongoDB work very well together, by marrying a JavaScript programming language with a JavaScript-oriented database engine. In this article we'll look at usage patterns for asynchronous coding in Node.js using async functions against the MongoDB database. Async functions are a powerful new feature in JavaScript, and we want to be sure to use them as widely as possible. Async functions gives cleaner database code because results plop into a convenient place.

Github buys npm: Might cause more angst about npm as de-facto package manager for Node.js? From the beginning of Node.js, npm has been a faithful companion providing useful package management service to the Node.js community. Node.js would not have risen so high without a good package manager, and npm served that role. By rights we should celebrate that Github is buying npm since a big question mark about npm has its level of funding. But - it raises a big question mark about the continued independence of the npm registry once it falls into the clutches of a big corporation (Microsoft owns Github).
Handling unhandled Promise rejections, avoiding application crash

With Promises and async functions, errors are automatically caught and are turned into a Promise rejection. What happens if nothing inspects the rejected Promise? Turns out that, currently, Node.js detects the unhandled Promise rejection and it prints a warning. In the warning, it says the behavior of catching unhandled Promise rejections is deprecated and will be returned, and instead the application will crash. Therefore it's worth our while to figure out what to do with unhandled Promise rejections.
Hiding data, creating encapsulation, in JavaScript ES-2015 classes

ES2015 added a ton of useful new features, and with ES2016/2017/2018 we're getting a whole new language with a list of exciting new capabilities. One of them is the new Class defintion syntax that brings Class definitions closer to what's done in other languages. Unfortunately the new Class syntax does not provide a robust implementation hiding mechanism. Hiding the implementation is important in object oriented programming, if only to have the freedom to change the implementation at any time.
High Performance Apps with JavaScript and Rust, It's Easier Than You Think
How do function(err,data) callbacks work in Node? Those new to Node programming might find the callback function pattern a little difficult to understand.  For example how does the data get into the arguments to the function call, and how does the callback function get called?
How do you choose between Node.js or other web application technologies?

There are plenty of new server side web application development technologies being developed. With the blizzard of choices before us, how do you choose between one or another? Will the newly hot web app technology really take off, or will it fizzle in a few years? For example, Node.js is getting a lot of excitement, but what about Go, or what about the mature platforms like PHP/Symfony or CakePHP?
How does Node.js compare to a traditional MVC platform like Spring?

Node.js is a young software development platform. It's only about 6 years old, and many software developers are still unsure about where Node.js fits into the landscape. It's JavaScript, which many people pigeon-hole as a browser language, but Node.js doesn't run on browsers but on servers. The question above illustrates a fundamental confusion about what Node.js is, and its role in the world, because it is not an "MVC" but a complete programming platform.

How to avoid vulnerability to malicious modules distributed through npm repository Recently a pair of modules in the npm repository became the vector for distributing malicious code. As much as npm Inc is working to improve security in the npm repository, there are steps we all must take as Node.js software coders to reduce the vulnerability of our packages. While its unlikely, in theory malicious code could be introduced through any package and how would we notice? It's not enough to trust npm Inc, we have our own work to do to strengthen the modules we publish.
How to correctly calculate path to a Node.js module to access files in that module directory Many Node.js modules contain files meant to be accessed as data. For example the Bootstrap, jQuery and Popper.js modules do not contain Node.js code, but exist to distribute the libraries. We are supposed to configure Express (or other framework) to serve the files in those directories to a web browser. The question is -- what is the best way to calculate the filesystem path of a file installed as a Node.js module inside the node_modules hierarchy?
How to create a copy-on-write clone of a file in Node.js It's been refreshing to learn of an advance in Linux/Unix file system semantics over the last 35 years. In 1984 I first used symbolic links, on a Vax running 4.2BSD. That was a significant advance over the hard links that had existed for many years before. The other day I learned that both Mac OS X and Linux support a new kind of link, a reflink, that is a form of copy-on-write for certain file systems.
How to deploy Express applications to AWS Lambda AWS Lambda and the AWS API Gateway is a powerful combination for deploying Node.js application code on an automagically scaling deployment platform. Out of the box you do not have the Express API. This webinar shows how to setup API Gateway and Lambda to support Express applications.
How to fix Node.js require returning an empty module object

In Node.js the require function is synchronous, meaning you're guaranteed that when require returns the module object is fully populated. But what happens when that doesn't happen? It can happen that the module object is empty, and then what do you do?
How to generate unique temporary file names in Node

Often we want to write data to a file, don't really care what the file name is, but want to be sure our code is the only code accessing that data.  A trivial algorithm might be to form a file name like:
How to get URL params after '?' in Express.js or Node.js?

Express makes it easy to route parameterized URL's to a handler function, and easily grab data out of the request URL.  This makes it easy to build a web application, that puts data into the URL, with clean URL's out of the box.  That's so much nicer than other application frameworks that require abominations like mod_rewrite to get clean URL's.  However nice the parameterized URL's are in Express, they do not handle other parts of the URL such as query parameters (the part of the URL after the '?').
How to install NAVE, an alternative Node.js version manager to nvm Those of us wanting to keep up with the latest Node.js version, or to test our code on alternative Node.js versions, probably have installed nvm. It is an easy-to-use tool for quickly installing the latest Node.js version, or to easily switch between Node.js versions to test our code. But nvm doesn't fit all use cases, and that's where nave comes in. Its purpose is extremely similar to nvm, but it satisfies a slightly different set of use cases that one may find interesting.
How to run JavaScript in the command-line environment

JavaScript is an extremely popular programming language deserving of a bigger role than browser-based programming. Fortunately we can use JavaScript outside web browsers, because of several non-browser JavaScript implementations. The most popular, Node.js, is not the only choice for running JavaScript outside the web browser.

How to use Passport in ExpressJS to handle Node.js app login/logout functionality Almost every web application requires login/logout functionality. In the Node.js/Express world, the Passport module handles that functionality.
How to use npm/yarn/Node.js package.json scripts as your build tool To Node.js the package.json simply tracks a few things about a package like its main module. To npm and yarn this file serves more purposes, from package management data, to dependencies lists, to package installation and project management scripts. The scripts tag may have a humble originating purpose, but by overusing it package.json becomes powerful platform for running command-line software processes like building websites or building software products.
If Wordpress is switching from PHP to Node.js, how should they do it? Supposedly the Wordpress team is migrating Wordpress from PHP to Node.js.
Implementing WebHooks and RESTHooks using TypeSript for Node.js and ExpressJS The WebHooks architecture lets one webservice send event notifications to another. RESTHooks build on them by advertising available events, allowing software to manage event subscriptions without human intervention.
Implementing __dirname in ES6 modules using import.meta.url

ES6 modules are a nifty new way to write modules. In Node.js, our traditional CommonJS modules had several injected variables one of which, __dirname, was incredibly useful. None of those variables exist in ES6 modules. The Node.js team is working on replacements that work in ES6 module contexts, and a new feature has been added in Node.js 10.0.0-pre giving us a way forward to implement the __dirname variable in ES6 modules.

Implementing worker queues for processing datasets in Node.js Processing large datasets can be done with a simple for loop. But, processing one data item at a time, eliminates possible efficiency gains from processing data in parallel. But, parallel processing is not scalable, since it can easily swamp CPU or memory resources. Worker queue packages help by constraining the number of parallel tasks executing at any one time.
Importing an ES6 modules over HTTP/HTTPS in a Node.js In Node.js 17.6.0, an experimental new feature allows us to import modules from an HTTP or HTTPS URL. That will close one of the differences between Node.js and Deno, which is that Deno allows packages to be imported using HTTPS. Further, ES6 modules in the browser allow importing modules over HTTPS. Until now, the Node.js team did not allow this, citing security concerns.
In JavaScript (Node.js), how do I read a text file from a different directory and store into a string? Built-in to Node.js are a few packages providing core services, including the "fs" package. As the name implies, it interfaces with the file-system, and among its functions is readFile that reads a file completely into memory. So, the short answer to this question is to simply call fs.readFile but we should take a longer look than that.
In Jenkins, automatically test an npm package, then publish if tests succeed We want to automate the boring stuff so we have more time for interesting stuff. Clearly running tests and administrivia like publishing a Node.js package to npm, well, that's boring stuff. It is the same tasks every time, and as an unautomated process you might forget a step, so let's look at how to automate this common workflow for Node.js programmers.
Installing MongoDB on Mac OS X Mavericks for Node.js development How do you upload files to a server to deploy application or website code?  FTP?  rsync?  While it's easy enough to call a command line tool like rsync from a Node.js script, what if you're using a Windows computer that doesn't have those command line tools.  When I use Windows it's like stepping back into the dark ages where directory listings looked like we were making fire by rubbing sticks together.  Okay, there does appear to be an rsync for Windows but I had no confidence in it.  Also, I did not want to have a dependency on something like Cygwin.
Installing some native-code npm packages on Ubuntu 20.04 fails on not finding python command Node.js 14.x was just released, as was Ubuntu 20.04. Testing my application on that combination failed because native code npm packages fail because python is not found. WTF? I followed the instructions and installed build-essential which supposedly brings in every compiler tool we would need. But ... fortunately there is a simple solution.
Introduction to Node.js with the Serverless framework on AWS Lambda AWS Lambda lets programmers focus on their code, with AWS taking care of all the deployment drama issues.
Is Node.js / Express scalable? How to grow Node.js app deployment to handle more traffic? Since Node.js is billed as being very fast, high performance, application development platform, you might think it automatically "scales" to fill out server resources and satisfy a huge workload. While you can do this with a Node.js application, you have to design the scaling mechanism yourself. With no additional configuration, a Node.js application can saturate a single CPU core and handle as many transactions as that core can sustain. That doesn't help when your manager asks why that 32 core server you just ordered has 31 idle CPU cores and one running flat out. Out of the box Node.js doesn't scale, but it's possible to make it do so.
Is Node.js a cancer? No!! It's quite nice, really

A recent blog post by Ted Dziuba claims that Node.js "is a cancer" and fills out a few hundred words of inflammatory laden "proof" to make his point. The post makes a few good points but is largely off base. Perhaps the most sticking point is that CPU intensive tasks make Node servers become unresponsive, but I have a clever answer for him. Another issue he raises ("JavaScript sux") is, well, as a longtime Java programmer who used to work in the Java SE team at Sun, I've had my own moments like that, but really JavaScript is quite a nice language especially if you stick to the good bits of JavaScript.
Is Node.js one of the most widely used scripting languages on the Internet? Really? I can believe JavaScript is one of the most widely used scripting languages, but Node.js? Huh? But, over there on embed.ly they have published two libraries implementing the embed.ly API for PHP and Node.js, and claimed "These two libraries give you server-side access via the two most widely used scripting languages today."
Java, Twitter, and asynchronous event driven architecture

I'm reading a blog post about what Node.js is, and there's a glaring question "why in the world would anyone want to run JavaScript outside of a browser, let alone the server?" As the author of a (www.amazon.com) book about Node.js it's a question I've thought about quite a bit, especially coming as I do from the Java SE team at Sun where the work of that team was highly focused on server side software (hey, it's Java) to the detriment of the client side of Java. The question comes from a belief cemented into place by 15+ years of JavaScript in the browser that it's a browser only language. However, JavaScript has a long history on server-side and is a rather cool language, and Node.js is an excellent base for developing web applications.

JavaScript or SQL injection attacks in the Node.js platform?

Traditionally the server side of web applications has been written in PHP, Perl, Python, Java, C/C++, etc. Javascript traditionally was implemented only in web browsers, and hence Javascript programming has been almost completely focused on the client end of web application development. It's arguably better to have the same programming language on both client and server sides of web application development, maybe. Several attempts have been made to implement javascript for server side web application development. A new javascript stack, Node.JS, is getting a lot of attention.
Javascript (specifically Node.JS) for server-side web application programming

Traditionally the server side of web applications has been written in PHP, Perl, Python, Java, C/C++, etc. Javascript traditionally was implemented only in web browsers, and hence Javascript programming has been almost completely focused on the client end of web application development. It's arguably better to have the same programming language on both client and server sides of web application development, maybe. Several attempts have been made to implement javascript for server side web application development. A new javascript stack, Node.JS, is getting a lot of attention.
Joyent webinar on Node.js and "Carriers" (?phone companies?)

Recently the primary supporter of Node.js, Joyent, posted a video webinar about "Node.js overview for Carriers". Eric Burns talked about a broad range of services offered by Joyent, features of Node.js, spinning it all around the needs of mobile device carriers. That is, the phone companies who provide mobile device services and run the cell phone networks, not those individuals who carry around mobile devices.
Learn the MERN stack with this video tutorial series MERN (MongoDB, ExpressJS, ReactJS, Node.js) is a popular alternative to the MEAN stack, and that has a nicer-sounding name. Therefore many of us want to learn the MERN stack, and this video tutorial series is an excellent way to start.
Loading an ES6 module in a Node.js CommonJS module

ES6 modules are a nifty new way to write modules. They're rather similar to the CommonJS module format traditionally used by Node.js, but there are significant differences. When ES6 module support was first added to Node.js, it was an experimental feature with significant limitations. One limitation was whether you could use ES6 modules from CJS modules, and there was a 3rd party package to fill that gap. Today, ES6 modules are a first class feature in Node.js, and ES6 modules can be used from CJS without any add-on support.

Loading jQuery and Bootstrap in Electron app has a curious bug What about using jQuery/Bootstrap to design an Electron app? It's easy to get started, right? You just load the jQuery and Bootstrap JavaScript libraries in that order. The process is well documented on the Bootstrap website, and since Electron is a web browser the same technique should work. But -- you get an inscrutible error that jQuery needs to be loaded. Which is odd because the HTML does indeed load jQuery, and in the Developer tools you can inspect the loaded files and see that jQuery is loaded. WTF?
MacOS X setup for Node.js development - Installing Node I'm setting up a new Mac, rather, I'm doing a clean install of my Mac Mini as a way to upgrade to Mavericks.  I've had the Mini since 2009 and the disc has some cruft so I thought it would be best to start with a clean slate.  That means reinstalling EVERYTHING including all the stuff I use for Node.js development. Hence, the idea came to mind that this would be a useful starting point for a series of blog posts.  I've always wondered what it would take to set up the Node.js equivalent to the MAMP application.
Make a bash script detect the directory it's stored in, to access data there

You may need to write a bash shell script that accesses data stored alongside the script, while your current working directory might be elsewhere. In my case the shell script needed to use Node.js scripts stored next to the shell scripts -- the shell script acting to simplify running the Node.js scripts. The "data" to be accessed in this case is the Node.js scripts, plus the support modules required to run them. You may have other data like a list of hostnames or who-knows-what.

Managing Node.js servers on Mac OS X with forever - works best for development If, like me, you're doing Node.js development on a Mac, you might have a yearning for a tool like MAMP but which works for Node.  A couple weeks ago I wrote a blog post covering the first step, (nodejs.davidherron.com) setting up a Node and npm instance on your computer.   If you don't know what MAMP is, go read that blog post, and then come back here.  What I want to go over today is a way to manage/monitor one or more Node processes on your computer.
Memory-efficient CSV transformation into CSV or text reports in Node.js

Those of us who consume/edit/modify/publish CSV files must from time to time transform a CSV file. Maybe you need to delete columns, rearrange columns, add columns, rename volumes, or compute some values taking one CSV file and producing another CSV file. Or maybe you need to convert the CSV to a simple text report. You might have a raw CSV file with no column headers, and need to supply column names. Or your CSV might, instead, have header names.
Message send from Electron Main process not received by Renderer Sending messages between Electron's main process and the Renderer looks simple. In the Renderer you use ipcRenderer to send a message to the main process, and in the main process you use windowObject.webContents.send to send the message. But what about when the process doesn't work and the documentation doesn't make it clear what to do?
New Book: Asynchronous JavaScript with Promises, Generators and async/await The JavaScript language is changing, and among the changes are three new features which will revolutionize the way we write asynchronous code in JavaScript. The problem with the old callback-oriented asynchronous coding practice is the Callback Hell resulting from stacking callbacks within callbacks within callbacks. Between Promises, Generators and the proposed async/await feature, JavaScript programmers can largely free themselves from Callback Hell, and write clean asynchronous code that's easier to write, easier to understand, easier to maintain, and more robust.
Node Cookbook is great for deeper understanding of Node.js programming
Node Web Development 2nd edition has been released!!
Node v0.8.17 released - fixes security vulnerability - we're urged to upgrade ASAP Isaac Schlueter just posted this warning ..
Node.js 10.x released - What's NEW?

After a year of development Node.js 10 has been released. It represents a huge step forward because of some high profile features and fixes. In October Node.js 10 will become the active Long Term Support branch, with 11.x becoming the new experimental branch. Let's take a look at what's been included.

Node.js 4.0.0 is out - quick tip for use while testing compatibility

Node.js v 4.0.0 was just released. This is a long-awaited release representing the healing of the schism in the Node.js community created when the io.js project forked Node.js. That had been done over a disagreement about the policies and maintainership of Node.js. Joyent had been in control of the project (because Ryan Dahl had been employed by Joyent) and not all in the community liked the decisions made by Joyent. They instead forked the project to create io.js, and implemented a bunch of useful ideas for example to adopt the latest-and-greatest version of the V8 JavaScript engine in order to get all the modern ES6 goodies.

Node.js Performance and Highly Scalable Micro-Services - Chris Bailey, IBM
Node.js Script writers: Top-level async/await now available Async/await functions are a god-send to JavaScript programmers, simplifying writing asynchronous code. It turns difficult-to-write-and-debug pyramids of doom into clean clear code where the results and errors land in the natural place. It's been wonderful except for one thing, we could not use the await keyword in top-level code in Node.js. But, with Node.js 14.8 that's fixed.
Node.js Web Development, 4th edition, coming soon

With the rapid advances in the Node.js platform, Packt Publishing (the publishers of Node.js Web Development) and I both felt a new edition was required. The 3rd edition published in mid-2016 updated the text to support Promises and some advanced techniques like deployment using Docker. Since then, async functions have emerged on the scene, and with Node.js 10.x we'll have ES6 modules available. I just submitted the first draft of all 12 chapters to the editors, meaning that the book is about a month from being finished.

Node.js and Bell's Law of computer classes

Node.js offers us an exciting new model for developing web applications and a statement in a recent episode of the TWiT Network's Triangulation podcast gave me an interesting lens through which to interpret the excitement around this new thing. That episode featured an interview with Gordon Bell, formerly the VP of Engineering at Digital Equipment Corporation, a company which at the time (1972) was revolutionizing the computer industry from being dominated by Mainframes to being dominated by Minicomputers. DEC's reason for existence was to develop and promote Minicomputers, but they were later supplanted by the Microcomputer era and DEC was unable to make the transition and later died. Gordon Bell is currently a researcher at Microsoft. He spoke about many things during the interview, he has over 40 years experience in developing key aspects of the computer industry, and his discussion about Bells Law struck me as a very apt way to describe the excitement around Node.js.

Node.js is a big win at PayPal

In my book Node Web Development (see sidebar), I spent the first chapter trying to sell the reader on using JavaScript on the server.  That's because the typical server side languages do not include JavaScript, meaning everyone has to be scratching their head and wondering why they should use JS on the server.  I suggested that, theoretically, one big win will come because the front end coding and back end coding will both be in the same language, JavaScript, which will make it possible for front engineers to talk with server engineers in the same language.  Or perhaps even be the same person.
Node.js team adopts the Contributor Code of Conduct, fostering a welcoming environment for contributors

Yesterday the Node.js project adopted the Contributor Covenant Code of Conduct which is an important step towards fostering an open and welcoming environment ensuring the project is a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. While this isn't a cool piece of technology, it is a step which should give the Node.js project a firmer standing in the world. In the past some open source projects have run into serious problems when team members acted with discrimination or harassment towards other team members. Having taken a stand like this, the team has put a firm stake in keeping the community open to all.

Node.js toolkit for mobile iOS and Android devices announced by Janea Systems

Node.js is no longer limited to server-side application development. The Electron platform, popularized through the Atom editor, is an excellent way to develop desktop applications. Now it's possible to target mobile devices running either iOS or Android using a Node.js implementation. Janea Systems is offering a "library" for both iOS and Android systems allowing an app to host a full Node.js execution environment, and offering UI implementation either with Cordova or React Native.
Node.js, the exciting web development platform, from a Java programmer point of view Hey, it's been awhile since I last posted on this blog, largely because I didn't have much to say about Java for awhile, and partly because I've been working with Node.js over the last year. It's a new software platform that I think the Java community needs to know about, and as the author of a book about Node (a.k.a. Node.js) I may be the person to explain a thing or two to y'all. Since it's been so long since my last post here, a bit of re-introduction should be useful. I'd spent 10 1/2 years working for Sun in the Java SE team, as an architect of the Software Quality Engineering team (SQE). I was also someone who had some interesting ideas to share about Java (hence this blog). Well, I hope they were interesting. During that time, I was privileged to meet the FreeJava community (classpath, harmony, etc), participating in launching the OpenJDK project, the JDK-Distros project, the Mustang Regressions contest, and some other interesting projects. It was an amazing time of meeting the broader Java community. But 2.5 yrs ago things changed (ahem) and I stopped being active with Java and didn't want to post here because I wasn't sure I had anything to say. Until now, when I feel a need to talk about Node.
Node.js: JavaScript on the Server - Ryan Dahl's original presentation at Google The following is the original presentation by Ryan Dahl showing the ideas behind Node.js and some of the performance results which have wow'd people.
Overriding console.log in Node.js, and other thoughts about logging in Node apps

How should Node.js programs best do logging?  When that question came up in the (groups.google.com) node users mailing list, Tim Caswell answered to just use console.log, and if you want to send the log to somewhere other than stdout then to override the function.  That makes the Java programmer recoil with "eeew, overriding a system function is wrong" but hey we're in a JavaScript environment.  This sort of thing is par for the course in JavaScript.  So, how do you go about doing this?
Potential for integrating Node.js with Drupal and speed up Drupal page processing Besides some experience with Node.js, enough to write the book linked in the side bar, I've also spent a lot of time building and configuring Drupal websites.  I've been pondering the possibilities for marrying Node with Drupal and have also seen a few projects spring up with that purpose.  However the core issue is that Drupal page processing is not an asynchronous process like Node's query handling, instead Drupal implements the typical synchronous start at the beginning and go to the end step by step model.  You know, the model we're trying to get away from by adopting Node.
Project setup for Node.js 14, Express, Bootstrap v5, Socket.IO application In this project we'll create a small TODO application using Node.js 14.x, the latest version of Express, Bootstrap v5, Bootstrap Icons, Sequelize for the database, and Socket.IO for multi-user real time interaction. In this posting we'll initialize the project directory, and put together the initial code.
Quickly start a new Electron and Vue.js application project, It's relatively easy to setup a blank Electron application, but of course the modern best practice is to use an application framework when creating an HTML+JavaScript application. The Electron development model more-or-less-exactly matches the model in regular web browsers. That means we can use a framework like React or Vue.js to make a powerful Electron app. In this tutorial we'll set up a simple Electron app, then add to it a simple bit of Vue.js code, and see that it is pretty simple to bring the two together.
Quickly start a new Electron software project If Electron encapsulates the Chromium web browser to support running GUI applications, isn't it a complex platform to use? Isn't it complex to get started with Electron? Yes, there is a fair amount of complexity under the hood of Electron. Electron really is the guts of the Google Chrome web browser, encapsulated in a Node.js runtime, letting Node.js programmers easily develop desktop GUI applications using web technology. That was a mouthfull to describe, and getting a fully polished application up and running has some complexity. Thanks to starter projects maintained by the Electron team, getting a skeleton application running is quick.
Real-Time Machine Learning with Node.js by Philipp Burckhardt, Carnegie Mellon University
Regarding the recent security vulnerability in event-stream and other npm packages Recently security vulnerabilities were discovered in the event-stream package, and at least one other. Malicious code was added to specific packages in a way that could be done much more broadly. While the specific vulnerability was tightly focused on one specific target and did not affect most of us, the problem could have been extremely wide-spread. As a result we, the Node.js community, need to rethink how packages are managed.
Resizing images using Node.js image processing tools, without external dependencies or image servers While getting ready to post a set of images on another blog, I wondered how to most effectively resize the images. They were high resolution product images that came with a press release, and to be nice to my readers it's important to shrink the image size to promote faster downloads. What if I want to supply several image sizes based on the device used by the visitor? It's not enough to resize the image on my own, it would be better to generate several image sizes. Which led to exploring Node.js tools for resizing images, and finding them lacking.
Running the TODO application In this article series we've developed a Todo application using Node.js 14.x, Express, Bootstrap v5, Socket.IO, and using Sequelize for the database layer. It was developed to explore some of the new features in Node.js, and to explore the post-jQuery world that Bootstrap v5 offers.
Ryan Dahl: History of Node.js In this talk, Ryan Dahl gives an expansive history of Node.js, not just the technological history but his personal history leading to developing Node.js. As a Mathematics PhD student in up-state New York, he had a "what the f**k am I doing" moment, dropped out of the program, and moved to Chile and ended up writing PHP websites. One thing led to another, and he became embroiled in the question of why Rails is so slow and doesn't scale well. And that led to the realizations leading to developing Node.js as a single threaded system relying on asynchronous code execution.
Ryan Dahl: Introduction to Node.js Ryan Dahl's presentation at Yahoo In May 2010, Yahoo held a meeting billed as Cinco de Node.js, and Ryan Dahl gave this presentation about Node.js.
Safely detect if a Node.js module is installed before using require() to load it Sometimes you need to detect whether a Node.js module is installed, and print a useful error message if it isn't. For example Grunt does this. The instructions are to install "grunt-cli" then add "grunt" to the package.json for the project for which you want to use Grunt. If you haven't installed grunt locally, typing "grunt" gives you a nice error message telling you what to do. It does that instead of printing an ugly message from the "require" function saying your module wasn't loaded.
Serious security hole found in Electron framework, Electron app's can be easily backdoored An attacker that can gain access to a machine can easily patch an Electron application to run bad code that does bad things. The problem is Electron does not secure the contents of ASAR files, and they can be easily opened and modified.
Serverless MongoDB, using ForerunnerDB to store local data in a Node.js application ForerunnerDB is a JavaScript library for storing data using a MongoDB-like API. It runs in both the browser and server-side in Node.js, and requires no setup of any server making it convenient for inline database functionality. It supports persisting data either to browser local storage, or the Node.js supports persisting data to a directory containing JSON objects.
Setting up the Typescript compiler to integrate with Node.js development TypeScript is growing rapidly in popularity within the JavaScript ecosystem. TypeScript offers us important compile-time data type enforcement, for what is otherwise JavaScript code. Any large project written in JavaScript will, in theory, find this useful because, in theory, the larger the project the more likely the coders will have keeping track of data types. The TypeScript tools run on Node.js, and while it can generate code for Node.js, it can be configured for any JavaScript environment.
Simple data export and manipulation using Node.js and the node-mysql module What follows is a fairly simple data processing script to extract data from a Drupal installation in a MySQL database. The purpose was to export nodes from a Drupal site, reformatting them for some software I'd written to import content into a blogger blog. In other words, this is the sort of simple data extraction and manipulation tool we write all the time.
Simplify catching uncaughtException, unhandledRejection, and multipleResolves in Node.js When native Promise and async/await support was added to Node.js, the process object gained some error events to notify the program about un-handled or un-caught error conditions. It's important to handle such errors, but of course we sometimes neglect to set up error handlers for every possible error condition. The primary purpose for these event handlers is so we know about program errors that we failed to catch. Node.js provides default error handlers which leave a bit to be desired. The log-process-errors assists with these issues.
Simplify your npm publish workflow using Github Actions Github's recently added new feature, Github Actions, promises us a powerful Workflow system to handle a huge range of tasks. Those of us who publish Node.js packages can use Actions to automatically run tests and then publish the package to npm. Let us see how to use Github Actions to simplify our lives.
Single page multi-user application with Express, Bootstrap v5, Socket.IO, Sequelize The Node.js world changed a lot this year with the Node.js 14 release, and its support for ES6 modules, async/await functions, and top-level async code. Another important change was the Bootstrap project which is preparing version 5 where the leading feature is to drop jQuery. These and other changes have come, and are coming, to application developers. This creates a good opportunity to revisit building web applications using Node.js, exploring not only the new features in Node.js but life with Bootstrap without relying on the jQuery crutch. To that end, let's write a little TODO application using Node.js 14, Express, Bootstrap 5, Sequelize, and Socket.IO.
Streamline Mocha/SuperTest REST tests with async functions

Mocha is an excellent unit testing framework for Node.js, and SuperTest is an excellent library for testing REST services. SuperTest is based on SuperAgent which makes it pretty darn easy to write code interfacing with a REST service. The problem with this combination is the test code can get pretty darn caught up in boilerplate callback functions, if you use Mocha and SuperTest the way its documented. Fortunately it's pretty easy to make SuperTest return a Promise, which then plays well with async functions, and Mocha is easily able to use an async function in a test scenario. So let's get started.
Terraform deployment of a simple multi-tier Node.js and Nginx deployment to AWS ECS Amazon's AWS Elastic Container Service (ECS) lets us deploy Docker containers to the AWS cloud. ECS is a very complex beast to tame, but Terraform offers a way to easily describe infrastructure builds not only AWS but on many other cloud services. Using Terraform is hugely simpler than any tool offered by AWS ECS.
Terraform deployment on AWS ECS a multi-tier Node.js and Nginx system using Service Discovery Amazon's AWS Elastic Container Service (ECS) lets us deploy Docker containers to the AWS cloud. In earlier postings of this series we have hosted a NGINX/Node.js application stack using Docker on our laptop, and on AWS ECS, then we hosted it on AWS ECS using Terraform. In this tutorial we switch to hosting the application using two AWS ECS Service instances, and use AWS Service Discovery between the instances.
Testing Bun's compatibility with Node.js and speed with a complex application Recently a new JavaScript server-side runtime, Bun, was announced, promising a massive speed improvement over Node.js. Instead of testing with a small application, let's try with a complex app to see real-world performance.
The advent of async/await for Node.js - Node.js v7 has now arrived

With today's first release of Node.js version 7, we now have async/await as a base feature of the platform. This is a significant milestone that wasn't even mentioned in the commit message. The async/await feature is so important that I'm preparing a short book to discuss JavaScript asynchronous programming patterns, that highly features async/await.

The difference between Node.js require, and Wordpress plugins or Drupal modules

Someone experienced with using Wordpress, or Drupal, to build websites are accustomed to "plugins" or "modules". Both are software modules which extend the functionality of Wordpress or Drupal websites, giving more features to the system than what's available out of the box. For example it's typical for a Drupal site to install Views to support building fancy data displays, and on (longtailpipe.com) my Wordpress blog I've added the PODS Framework to simplify defining custom post types along with custom fields.
The new test framework built-in to Node.js 18.8.0 New in Node.js v18.8.0 is a test framework highly remniscient of Mocha. It is marked as experimental, but points to a future where our unit tests will not require an external dependency like Mocha.
The rumors of Express.js's death are greatly exagerrated Wandering across a question on Quora (Is Express.js dying? What are the alternatives?) had me stop and take a look at the facts. I'd noticed a few months ago that contributions to the main Express repository had dried up, and therefore I was worried the Express project was dying from neglect. So let's talk about this, because it is important to clear up the air.
Troubles with Asynchronous code flows in JavaScript, and the async/await solution of ES-2017

Asynchronous coding is one of those powerful tools that can bite, if you're not careful. Passing around anonymous so-called "callback" functions is easy, and we do it all the time in JavaScript. Asynchronous callback functions are called, when needed, as needed, sometime in the future. The key result is that code execution is out-of-order with the order-of-appearance in the code.
TypeScript adds to programmer productivity and expressiveness, and is not dying Recently the teams for a couple front-end libraries abandoned using TypeScript and returned to vanilla JavaScript. That leads some to claim TypeScript is dying, and anyway that it doesn't provide useful productivity advantages to developers. While I do not have data on software development trends, I do find TypeScript adds significantly to my development work, without costing significant time.
Unit testing Express route handlers in isolation from everything, including Express

When we write unit tests it's good practice to "mock" out extraneous bits to the code being tested. It's almost like the scientific method in that testing, in the unit testing paradigm, means exercising each small portion of your code in isolation if only to eliminate unwanted variables. While there are other testing paradigms, unit testing has its value. A big question for Node.js web application programmers is - how do you mock out HTTP requests for unit testing? In other words, how do you test the route handler method in isolation from the Node.js HTTPServer object, or Express, or whatever app framework used in your application?
Use Bash-like command-line environment variables on Windows in Node.js npm scripts It is best to automate all things, and in Node.js applications a convenient tool is the scripts section of package.json. Using the scripts section, npm helps you to easily record the necessary commands to build or maintain an application. However, npm does nothing to help with cross platform command-line compatibility, because it simply hands the command string to Bash (on Linux/macOS/etc) or to CMD.EXE on Windows. For the most part this is okay, but what if you need to set environment variables? The Bash way of setting command variables gives an error message on Windows.
Use Electron-Vue to build Electron/Vue.js application with Bootstrap v4 -- DISRECOMMENDED Using Vue.js in an Electron app means dealing with using Webpack to build/package the components and other assets. The most effective way to put this together is the Electron-Vue package, which builds itself as "Boilerplate for making electron applications built with Vue.js". What that means is Electron-Vue is a pre-built application template and build infrastructure to simplify running Electron/Vue applications in development mode, and to package Electron/Vue applications for distribution.
Useful reading to understand the Promises, Generators and the async/await feature for Node.js/JavaScript

The long-awaited async/await feature for JavaScript promises to make our lives much easier. Instead of asynchronous JavaScript programming being tricky and error-prone, with async/await our code will look like regular synchronous code we write in other languages, but will accommodate asynchronous code execution while retaining the single-threaded nature of JavaScript. With this feature we declare async functions, and within their boundary magic occurs - we can put the "await" keyword in front of a function which produces a Promise, and automagically JavaScript will wait for the Promise to resolve and give us the result or else throw the error. What follows is a list of posts describing how to use async/await.

Using Bootstrap and Socket.IO for the user interface in an Express Node.js 14 application The Bootstrap project has released the first Alpha for Bootstrap v5. It contains a number of major changes, such as dropping the requirement to use jQuery. Nothing is said about when the first official release will ship, but we can get a taste of what Bootstrap v5 will be like by using the Alpha to build a small application. This article goes over developing a Bootstrap v5 user interface for a TODO application.
Using Dynamic import in Node.js lets us import ES6 modules in CommonJS code, and more - UPDATED With ES6, JavaScript programmers finally gained a standard module format that worked on both browser-side and server-side (Node.js) JavaScript runtimes. Plus, ES6 modules offer many interesting features. But Node.js programmers need to know how to integrate ES6 modules with the CommonJS module format we're accustomed to using. Now that Node.js v14 is coming, and bringing with it full ES6 module support as a standard feature, it is time to update a previous blog post about Dynamic Import on Node.js. The Dynamic Import feature, a.k.a. the import() function, gives JavaScript programmers much more freedom over the standard import statement, plus it lets us import an ES6 module into CommonJS code.
Using Git submodules to streamline Node.js package development When developing a Node.js package, we typically create a test application with which to exercise the package. That means for every source code change in the package, we must reinstall the package into the node_modules directory of the test application, which can be very frustrating. Git's submodule feature can streamline this by letting you directly edit both the test application and package with no reinstalls required.
Using HTMLParser2, DOMUtils, to process HTML and XML in Node.js HTMLParser2 is part of a cluster of Node.js packages (domhandler, domutils, css-select, dom-serializer) that enable powerful manipulation of both HTML and XML DOM object trees. These packages can be used not just for web scraping, but for server-side DOM manipulation, and they form most of the underpinning of Cheerio, the Node.js package for jQuery-like DOM manipulation on Node.js.
Using Protocol Buffers with Node.js applications Protocol Buffers is a Google-developed toolchain for binary encoding of data and objects that works between programming languages. It is the basis of gRPC, a cross-language remote procedure call system, but can be used separately.
Using SSL to connect to MySQL database in Node.js Encrypting database connections can be extremely important for security. The documentation for the Node.js MySQL driver briefly mentions SSL support, and does not give adequate documentation. What follows is an example showing how to connect using PEM certificates to a MySQL server that was configured with a self-signed root CA.
Using Sequelize for the database layer in an Express Node.js 14 application Like many web applications, the TODO application we are developing with Express and Bootstrap v5 must persist its data somewhere. For that purpose we will use Sequelize to to interface with an SQL database (SQLite3). Sequelize is a high level ORM (Object Relations Manager) for Node.js that supports a long list of SQL databases.
What is Babel, and when should one use Babel?

Fast advances in the JavaScript language mean the new features are not evenly distributed. Fortunately the JavaScript language is flexible enough that in many cases new features can be retrofitted by using a polyfill among other techniques. One of those techniques is Babel, a so-called Transpiler that can convert code using new features into equivalent code using old features, so that a programmer can code with the new features and still deploy to older JavaScript implementations.

What is Node.js Exactly? - a beginners introduction to Nodejs Node.js is JavaScript running outside of the web browser. If you're expecting a Window or Document object because JavaScript has that, you'll be disappointed because those objects are side effects of running in a browser. This introduction to Node.js is a very basic intro, showing the bare bones of beginning to understand what Node.js is, and what its role is in the software industry.
What might the excitement about Node.js be about? JavaScript on the server? Events? Or, what? There's a lot of excitement brewing around Node.js with a rapidly growing community of people and projects surrounding the platform, and at least five books either published or on their way to being published. I myself am working in a team at a company I cannot yet name on a large Node.js project that could be very significant. The team is excited about the platform but also have some real eyes on the question of whether this makes sense for the company, or what. I thought it would be interesting to run through some attributes of Node.js and ponder what might or might not contribute to the excitement about Node.js and its eventual success. I'm also interested in what others think, so consider leaving your thoughts below.
What's the best open source license for Node.js modules ?? I've been developing a new software project with Node.js, a (akashacms.com) "content management" system that produces static HTML websites named AkashaCMS.  Last week I made my initial release into npm and knew I needed to put an open source license on the code, and by default I put in the GPL license.  But then I thought to ponder, is the GPL the best choice of license?  Will that license prevent others from using AkashaCMS?  In general, what's the best license for a Node.js open source project?  Does use of a Node module invoke the viral nature of the GPL and infect GPLness into modules that use the GPL'd module?  I didn't think about this myself, but would the GPL license interfere with others who want to contribute patches?  So I sent a query to the Node.js group on Google Groups (linked below), which touched off an insightful conversation.
Where should you put routes in an Express/Node.js web application for simple easy coding enjoyment?

The Express.js app framework for Node.js is a pretty cool system that makes it easy to implement web apps and even REST API's. But the Express team doesn't give you any guidance on structuring the application code. They give you an API and it's up to you to decide how or even if you structure the model-view-controller paradigm or any other paradigm you wish to use.

Why publish Node.js packages to npm using Babel/Webpack? Babel and Webpack are widely used tools for front-end-engineering. Between the two you can code JavaScript using the latest ES2016/2017/2018/etc features, with multiple modules, and transpile it down to a simple JavaScript file using ES5 syntax. This is a big step forward in JavaScript tooling. But, since Node.js natively supports a huge fraction of ES2016/2017/2018/etc features, why do Node.js programmers even need to know Babel/Webpack? After reading a tutorial on using Babel/Webpack with Node.js modules, I have the beginning of an inkling for why.
With a Node.js based REST service, is it okay to not use a front-end framework like React or Vue.js?

React and Vue.js are very popular front-end application frameworks. One might think either one is the best and only way to develop the front end of an application. Lots of folks will tell you, use React because it's the best framework, while others are flocking to Vue.js because whatever. A few years ago The MEAN Stack (Mongo-Express-Node-Angular) was all the rage, and now The MERN Stack (Mongo-Express-React-Node) is taking its place. Is it therefore ridiculous folly to skip using any front end framework? Or to skip using one of the popular choices?
Yarn versus npm, what's the big deal? A couple years ago Yarn came onto the Node.js scene promising to be a better npm than npm. Apparently a lot of people have taken to Yarn, given the frequency with which I see instructions to install packages using Yarn. Since npm is installed along with Node.js, and since Yarn just uses the npm package repository, I'm left wondering what all the fuss is about Yarn. If it's just doing the same thing as npm then what is its reason to exist? What advantage does Yarn have over npm? Let's take a look.
npm version 5 has major usability bug with installing packages locally

With npm version 5 we gained a lot of welcome new features and performance improvements. I've been happily using npm@5 for several months, but recently discovered a major problem that dramatically affects my workflow. When I'm updating a package, I want to test that package locally WITHOUT pushing changes to the Git repository. To do so, I found it best to install that package into another project to test/run the code. This worked great with npm versions prior to npm@5, but now I have two major problems. First, npm modifies the package.json to insert a "file:" dependency, overwriting the existing dependency, and second it makes a symlink to the package rather than doing a proper installation.