How to use Node.js features

In JavaScript (Node.js), how do I read a text file from a different directory and store into a string?

(2017-06-18 22:57) Built-in to Node.js are a few packages providing core services, including the "fs" package. As the name implies, it interfaces with the file-system, and among its functions is readFile that reads a file completely into memory. So, the short answer to this question is to simply call fs.readFile but we should take a longer look than that.

Troubles with Asynchronous code flows in JavaScript, and the async/await solution of ES-2017

(2017-01-22 21:25)

Asynchronous coding is one of those powerful tools that can bite, if you're not careful. Passing around anonymous so-called "callback" functions is easy, and we do it all the time in JavaScript. Asynchronous callback functions are called, when needed, as needed, sometime in the future. The key result is that code execution is out-of-order with the order-of-appearance in the code.

Useful reading to understand the Promises, Generators and the async/await feature for Node.js/JavaScript

(2016-11-20 17:01)

The long-awaited async/await feature for JavaScript promises to make our lives much easier. Instead of asynchronous JavaScript programming being tricky and error-prone, with async/await our code will look like regular synchronous code we write in other languages, but will accommodate asynchronous code execution while retaining the single-threaded nature of JavaScript. With this feature we declare async functions, and within their boundary magic occurs - we can put the "await" keyword in front of a function which produces a Promise, and automagically JavaScript will wait for the Promise to resolve and give us the result or else throw the error. What follows is a list of posts describing how to use async/await.

Memory-efficient CSV transformation in Node.js

(2016-11-14 17:06)

Those of us who consume/edit/modify/publish CSV files must from time to time transform a CSV file. Maybe you need to delete columns, rearrange columns, add columns, rename volumes, or compute some values taking one CSV file and producing another. In my case, I have a raw CSV file with no column headers that's organized in a way which makes sense for one team in our company, but we need that same data organized a different way, with different column names and containing selected fields. The following is what came from that need, which I managed to write in a fairly generic way. It not only extracts and renames columns, but with a bit of coding could perform other transformations.

The advent of async/await for Node.js - Node.js v7 has now arrived

(2016-10-25 08:35)

With today's first release of Node.js version 7, we now have async/await as a base feature of the platform. This is a significant milestone that wasn't even mentioned in the commit message. The async/await feature is so important that I'm preparing a short book to discuss JavaScript asynchronous programming patterns, that highly features async/await.

Easily import your CSV data into MySQL with Node.js

(2016-09-16 10:59)

It's convenient to import a CSV file into a spreadsheet to crunch numbers. CSV's (comma-seperated-values) and it's brethren (TSV, for tab-separated, etc) are widely used to deliver information. You'll see CSV downloads available all over the place, whether it's your stock broker or bank, or a government agency, and on and on. Spreadsheet programs like LibreOffice Calc offer direct import of these files, and then as a spreadsheet you've got a zillion tools available to crunch numbers, make graphs, etc. Sometimes, though, you need to load the CSV into a database to do other sorts of work.

Fixing "Maximum call stack size exceeded" in async Node.js code

(2016-09-09 13:20)

I've happily used the async module for Node.js for years to simplify asynchronous operations over arrays. This is a difficulty in the Node.js paradigm, since the "normal" way to process an array is with a simple for loop. But if the operation is asynchronous the for loop has no way of knowing when to iterate to the next item, or even when processing is finished. Instead the async.eachSeries function does the trick, because your code tells async.eachSeries when to go to the next item, when there's an error, and it knows when the last item is processed, etc. It's been great.

Node.js 4.0.0 is out - quick tip for use while testing compatibility

(2015-09-09 17:58)

Node.js v 4.0.0 was just released. This is a long-awaited release representing the healing of the schism in the Node.js community created when the io.js project forked Node.js. That had been done over a disagreement about the policies and maintainership of Node.js. Joyent had been in control of the project (because Ryan Dahl had been employed by Joyent) and not all in the community liked the decisions made by Joyent. They instead forked the project to create io.js, and implemented a bunch of useful ideas for example to adopt the latest-and-greatest version of the V8 JavaScript engine in order to get all the modern ES6 goodies.

The difference between Node.js require, and Wordpress plugins or Drupal modules

(2015-07-19 18:25)

Someone experienced with using Wordpress, or Drupal, to build websites are accustomed to "plugins" or "modules". Both are software modules which extend the functionality of Wordpress or Drupal websites, giving more features to the system than what's available out of the box. For example it's typical for a Drupal site to install Views to support building fancy data displays, and on my Wordpress blog I've added the PODS Framework to simplify defining custom post types along with custom fields.

Where should you put routes in an Express/Node.js web application for simple easy coding enjoyment?

(2015-06-09 15:20)

The Express.js app framework for Node.js is a pretty cool system that makes it easy to implement web apps and even REST API's. But the Express team doesn't give you any guidance on structuring the application code. They give you an API and it's up to you to decide how or even if you structure the model-view-controller paradigm or any other paradigm you wish to use.

Don't rip your hair out when Vows tells you "Errored callback not fired" - try this one weird trick instead

(2015-01-07 18:10)

When your vows.js based tests for a Node.js application says "Errored » callback not fired" -- well, it can be very confusing. In my case the code clearly handled all paths ensuring the Vows callback would be called. No matter how many tweaks I performed to try and catch some possible error in test or code, I couldn't figure out what caused this problem. But after some yahoogling, the answer was not only difficult to find, but surprisingly simple.

Easily offload your CPU-intensive Node.js code with simple Express-based REST server

(2014-07-03 23:47)

Node.js is horrible with CPU bound processing, supposedly. Why? Because CPU-intensive algorithms block the event loop from handling events, blocking the Node.js platform from doing its core competency. Actually, as I demonstrate in my book Node Web Development (see sidebar for link), it's possible to use "setImmediate" to dispatch work through the Node.js event loop, and perform intensive computation while not blocking the event loop. The example I chose for the book is a simplistic Fibonacci algorithm, and requesting "large" Fibonacci values (like "50") would take a loooong time. But, by recoding the Fibonacci algorithm using setImmediate, it can do calculation for any Fibonacci value without blocking the event loop.

Unit testing Express route handlers in isolation from everything, including Express

(2014-06-07 23:48)

When we write unit tests it's good practice to "mock" out extraneous bits to the code being tested. It's almost like the scientific method in that testing, in the unit testing paradigm, means exercising each small portion of your code in isolation if only to eliminate unwanted variables. While there are other testing paradigms, unit testing has its value. A big question for Node.js web application programmers is - how do you mock out HTTP requests for unit testing? In other words, how do you test the route handler method in isolation from the Node.js HTTPServer object, or Express, or whatever app framework used in your application?

Avoid tearing your hair out on variable values in deeply nested JavaScript/Node.js callback chains

(2014-05-22 14:51)

JavaScript, and hence Node.js, supports a useful model for variable scoping and callback functions where the available variables build up as the code nests. It's very convenient because your code doesn't have to pass values through function parameters to code in an in-line callback function. But it's possible for a variable to not have the expected value by the time the callback function executes. When that's the case, it's tempting to start tearing your hair out in frustration screaming about the variable's value is incorrect. The culprit can be that the code is executing asynchronously even though it looks like linear code.

Is there a way to pass variables down to a nested callback without passing them to each function along the way, unnecessarily?

JavaScript doesn't tell you the Date object is bad, here's how to figure that out before crashing your program

(2014-05-04 21:41)

In JavaScript, creating a Date object from a string is real convenient ("var foo = new Date(dateString)"), but what if dateString has a bad format? How will your code know about this? The Date object doesn't have a getter to tell you the date is bad.

In AkashaCMS I just added a feature so the website author can set the file date using metadata in the content. Previously the file date was set from the date code on the input file, but of course that's probably not accurate. It's better if the website author can specify the date they want to appear in the website, and as the file date.

Managing Node.js servers on Mac OS X with forever - works best for development

(2014-01-03 17:07) If, like me, you're doing Node.js development on a Mac, you might have a yearning for a tool like MAMP but which works for Node.  A couple weeks ago I wrote a blog post covering the first step, setting up a Node and npm instance on your computer.   If you don't know what MAMP is, go read that blog post, and then come back here.  What I want to go over today is a way to manage/monitor one or more Node processes on your computer.

How to generate unique temporary file names in Node

(2013-12-17 16:35)

Often we want to write data to a file, don't really care what the file name is, but want to be sure our code is the only code accessing that data.  A trivial algorithm might be to form a file name like:

How to get URL params after '?' in Express.js or Node.js?

(2013-06-09 11:38)

Express makes it easy to route parameterized URL's to a handler function, and easily grab data out of the request URL.  This makes it easy to build a web application, that puts data into the URL, with clean URL's out of the box.  That's so much nicer than other application frameworks that require abominations like mod_rewrite to get clean URL's.  However nice the parameterized URL's are in Express, they do not handle other parts of the URL such as query parameters (the part of the URL after the '?').

Overriding console.log in Node.js, and other thoughts about logging in Node apps

(2012-08-06 22:57)

How should Node.js programs best do logging?  When that question came up in the node users mailing list, Tim Caswell answered to just use console.log, and if you want to send the log to somewhere other than stdout then to override the function.  That makes the Java programmer recoil with "eeew, overriding a system function is wrong" but hey we're in a JavaScript environment.  This sort of thing is par for the course in JavaScript.  So, how do you go about doing this?

Getting image metadata using the Node.js imagemagick module

(2011-11-03 21:27)

Continuing on in my quest for Node.js scripts to manipulate images (see my earlier JPG to PNG conversion script), today I'm looking at how to access image metadata. The result I'm interested in is to store a caption, keywords and other information in the image itself and to display the caption and keywords on web pages. I found it pretty easy to add a caption and keywords and other metadata using Picasa.

Convert JPG's into PNG's using Node.js and ImageMagick - Possible Photo workflow?

(2011-10-30 22:09)

I'm pondering a new photography workflow where, instead of working with the JPG's directly out of the camera, that I instead convert them to PNG first.  The reasoning is that JPG is a lossy image format meaning every time you edit a JPG it loses a bit of precision, whereas PNG is a lossless image format and you don't lose precision.  That idea might not be the best, I haven't checked how well Picasa works with PNG's, in any case it let me play with what kind of image manipulation we can do in Node.js.

2016 Election Acer C720 Ad block AkashaCMS Android Apple Apple Hardware History Apple iPhone Hardware April 1st Arduino ARM Compilation Asynchronous Programming Authoritarianism Automated Social Posting Bells Law Big Brother Blade Runner Blogger Blogging Books Botnet Botnets Cassette Tapes Cellphones Christopher Eccleston Chrome Chrome Apps Chromebook Chromebooks Chromebox ChromeOS CIA CitiCards Civil Liberties Clinton Cluster Computing Command Line Tools Computer Hardware Computer Repair Computers Cross Compilation Crouton Cyber Security Cybermen Daleks Darth Vader Data backup Data Storage Database Database Backup Databases David Tenant DDoS Botnet Detect Adblocker Digital Photography DIY DIY Repair Docker Doctor Who Doctor Who Paradox Drobo Drupal Drupal Themes DVD Election Hacks Emdebian Enterprise Node ESP8266 Ethical Curation Eurovision Event Driven Asynchronous Express Facebook Fake News File transfer without iTunes FireFly Fraud Freedom of Speech Gallifrey git Gitlab GMAIL Google Google Chrome Google Gnome Google+ Government Spying Great Britain Home Automation HTTPS I2C Protocol Image Conversion Image Processing ImageMagick InfluxDB Internet Internet Advertising Internet Law Internet of Things Internet Policy Internet Privacy iOS Devices iPad iPhone iPhone hacking Iron Man Iternet of Things iTunes Java JavaScript JavaScript Injection JDBC John Simms Joyent Lets Encrypt LibreOffice Linux Linux Hints Linux Single Board Computers Logging Mac OS Matt Lucas MEADS Anti-Missile Mercurial Michele Gomez Military Hardware Missy Mobile Applications MODBUS Mondas Monty Python MQTT Music Player Music Streaming MySQL NanoPi Nardole Net Neutrality Node Web Development Node.js Node.js Database Node.js Testing Node.JS Web Development Node.x North Korea Online advertising Online Fraud Open Media Vault Open Source Software OpenAPI OpenVPN Personal Flight Peter Capaldi Photography Plex Media Server Political Protest Power Control Privacy Production use Public Violence Raspberry Pi Raspberry Pi 3 Raspberry Pi Zero Recycling Republicans Retro-Technology Reviews Right to Repair River Song Rocket Ships RSS News Readers rsync Russia Russia Troll Factory Scheme Science Fiction Season 1 Season 10 Season 11 Security Security Cameras Server-side JavaScript Shell Scripts Silence Simsimi Skype Social Media Warfare Social Networks Software Development Space Flight Space Ship Reuse Space Ships SpaceX Spring SQLite3 SSD Drives SSD upgrade SSH SSH Key SSL Swagger Synchronizing Files Terrorism The Cybermen The Daleks The Master Time-Series Database Torchwood Total Information Awareness Trump Trump Administration Ubuntu Virtual Private Networks VirtualBox VLC VOIP Web Applications Web Developer Resources Web Development Web Development Tools Weeping Angels WhatsApp Wordpress