Easily offload your CPU-intensive Node.js code with simple Express-based REST server

By: (plus.google.com) +David Herron; Date: 2014-07-03 23:47

Tags: Node.JS

Node.js is horrible with CPU bound processing, supposedly. Why? Because CPU-intensive algorithms block the event loop from handling events, blocking the Node.js platform from doing its core competency. Actually, as I demonstrate in my book Node Web Development (see sidebar for link), it's possible to use "setImmediate" to dispatch work through the Node.js event loop, and perform intensive computation while not blocking the event loop. The example I chose for the book is a simplistic Fibonacci algorithm, and requesting "large" Fibonacci values (like "50") would take a loooong time. But, by recoding the Fibonacci algorithm using setImmediate, it can do calculation for any Fibonacci value without blocking the event loop.

However, this isn't always what you want to do. The CPU-intensive processing is still running in the Node.js process, it's just not blocking the event loop. That's an improvement, but not always what we want.

In particular it should be good to move any CPU-intensive stuff out of the user-facing server. That way the outside edge of your application will remain as responsive as possible.

With the Fibonacci algorithm there's a fairly obvious move, and that's to not use such a simplistic Fibonacci algorithm. For example, caching Fibonacci values is quite easy, and will result in nearly instantaneous Fibonacci calculations of any value. But, calculating Fibonacci values is not an example of real world CPU-intensive work. Maybe you can make drastic performance improvements in your algorithm, but probably not.

What you can do is farm CPU-intensive work out to other servers. The question is - how?

A really simple way, and one that I show in Node Web Development, is that Express makes it trivially easy to create a REST server. You may think of Express as being a framework for HTML webapp development, but it's also useful for REST services.

It's as simple as building a server with an Express route definition like this:

app.get('/fibonacci/:n', function(req, res, next) {
  calculateCPUintensiveValue(req.params.n, function(err, result) {
      if (err) next('ERROR OCCURRED');
      else {
        res.send({ n: req.params.n, result: result });

Of course you should use a pathname appropriate to your application, and your application is probably going to expose more API points. This is the pattern, though, of putting request parameters in the URL, and when done with the CPU-intensive calculation you call res.send. That function formats the object it's given as JSON, automatically.

Once you've coded your server, it can be tested easily:

$ curl -f http://localhost:3000/fibonacci/10
{ "n": "10", "result": 55 }

And then to call the REST service from inside another Express application, do this:

var httpreq = require('http').request({
  host: "localhost", port: 3000,
  path: "/fibonacci/"+ Math.floor(fiboRequest),
  method: "GET"
}.function(httpresp) {
  httpresp.on('data', function(chunk) {
    var data = JSON.parse(chunk);
    res.render(... );

The simplicity of doing this is one of the big wins from using Node.js. The platform and the major libraries, like Express, work together to make this sort of task easy as pie.

« How do you choose between Node.js or other web application technologies? Unit testing Express route handlers in isolation from everything, including Express »
2016 Election Acer C720 Ad block AkashaCMS Amazon Amazon Kindle Amiga Android Anti-Fascism AntiVirus Software Apple Apple Hardware History Apple iPhone Apple iPhone Hardware April 1st Arduino ARM Compilation Astronomy Asynchronous Programming Authoritarianism Automated Social Posting Ayo.JS Bells Law Big Brother Big Finish Bitcoin Mining Black Holes Blade Runner Blockchain Blogger Blogging Books Botnet Botnets Cassette Tapes Cellphones Christopher Eccleston Chrome Chrome Apps Chromebook Chromebooks Chromebox ChromeOS CIA CitiCards Citizen Journalism Civil Liberties Clinton Cluster Computing Command Line Tools Computer Hardware Computer Repair Computers Cross Compilation Crouton Cryptocurrency Curiosity Rover Cyber Security Cybermen Daleks Darth Vader Data backup Data Storage Database Database Backup Databases David Tenant DDoS Botnet Detect Adblocker Developers Editors Digital Photography Diskless Booting DIY DIY Repair DNP3 Do it yourself Docker Docker Swarm Doctor Who Doctor Who Paradox Drobo Drupal Drupal Themes DVD E-Books E-Readers Early Computers Election Hacks Electric Bicycles Electric Vehicles Electron Emdebian Encabulators Energy Efficiency Enterprise Node EPUB ESP8266 Ethical Curation Eurovision Event Driven Asynchronous Express Facebook Fake News Fedora VirtualBox File transfer without iTunes FireFly Fraud Freedom of Speech Gallifrey git Gitlab GMAIL Google Google Chrome Google Gnome Google+ Government Spying Great Britain Heat Loss Hibernate Hoax Science Home Automation HTTPS Human ID I2C Protocol Image Analysis Image Conversion Image Processing ImageMagick InfluxDB Infrared Thermometers Insulation Internet Internet Advertising Internet Law Internet of Things Internet Policy Internet Privacy iOS Devices iPad iPhone iPhone hacking Iron Man Iternet of Things iTunes Java JavaScript JavaScript Injection JDBC John Simms Journalism Joyent Kaspersky Labs Kindle Marketplace Lets Encrypt LibreOffice Linux Linux Hints Linux Single Board Computers Logging Mac OS Mac OS X Machine Readable ID MacOS X setup Make Money Online MariaDB Mars Matt Lucas MEADS Anti-Missile Mercurial Michele Gomez Micro Apartments Military Hardware Minification Minimized CSS Minimized HTML Minimized JavaScript Missy Mobile Applications MODBUS Mondas MongoDB Mongoose Monty Python MQTT Music Player Music Streaming MySQL NanoPi Nardole NASA Net Neutrality Node Web Development Node.js Node.js Database Node.js Testing Node.JS Web Development Node.x North Korea Online advertising Online Fraud Online Journalism Online Video Open Media Vault Open Source Governance Open Source Licenses Open Source Software OpenAPI OpenVPN Personal Flight Peter Capaldi Photography PHP Plex Plex Media Server Political Protest Postal Service Power Control Privacy Production use Public Violence Raspberry Pi Raspberry Pi 3 Raspberry Pi Zero Recycling Remote Desktop Republicans Retro-Technology Reviews Right to Repair River Song Robotics Rocket Ships RSS News Readers rsync Russia Russia Troll Factory Russian Hacking SCADA Scheme Science Fiction Search Engine Ranking Season 1 Season 10 Season 11 Security Security Cameras Server-side JavaScript Shell Scripts Silence Simsimi Skype Social Media Social Media Warfare Social Networks Software Development Space Flight Space Ship Reuse Space Ships SpaceX Spear Phishing Spring Spring Boot SQLite3 SSD Drives SSD upgrade SSH SSH Key SSL Swagger Synchronizing Files Telescopes Terrorism The Cybermen The Daleks The Master Time-Series Database Torchwood Total Information Awareness Trump Trump Administration Trump Campaign Ubuntu UDOO Virtual Private Networks VirtualBox VLC VNC VOIP Web Applications Web Developer Resources Web Development Web Development Tools Web Marketing Website Advertising Weeping Angels WhatsApp Window Insulation Wordpress YouTube YouTube Monetization