In JavaScript (Node.js), how do I read a text file from a different directory and store into a string?

By: (plus.google.com) +David Herron; Date: 2017-06-18 22:57

Tags: Node.JS

Built-in to Node.js are a few packages providing core services, including the "fs" package. As the name implies, it interfaces with the file-system, and among its functions is readFile that reads a file completely into memory. So, the short answer to this question is to simply call fs.readFile but we should take a longer look than that.

The first little caveat is the phrase "from a different directory" in the question. That requires we also familiarize ourselves with the path package, because of the path.join function.

const path = require('path');
const fs   = require('fs');

const srcdirectory = "path/to/some/where";
const file2read    = "file.txt";

const text = fs.readFileSync(path.join(srcdirectory, file2read), 'utf8');

The path.join function joins two strings as directory path names. We use it rather than srcdirectory+'/'+file2read because path.join ensures correctness. It knows when it's running on Windows where the slashes go the wrong way around, and it knows to check for slashes before blindly inserting one.

That second parameter is, however, making a huge assumption that you're reading a text file and that the text can be encoded in utf8.

That second argument is the options argument, and when used this way it signifies the text encoding. It can also be an object, and I recommend reading the documentation.

The glaring thing about this code snippet is that it is synchronous (fs.readFileSync). Node.js famously follows an asynchronous programming model. Reading text from a file inherently takes a long time (for a computer), during which time that Node.js process could be doing other things.

Typical Node.js code would do that function in this way:

const path = require('path');
const fs   = require('fs');

const srcdirectory = "path/to/some/where";
const file2read    = "file.txt";

fs.readFile(path.join(srcdirectory, file2read), 'utf8', (err, text) => {
    if (err) ... do something with the error;
    else {
        ... do something with the text;
    }
});

The key observation behind Node.js's very existence is the time difference between local memory access and other operations that take awhile, such as reading a file from disk or making a database query. A "blocking" operation, where execution has to wait for a long-running request to finish, does bad things to throughput of a network server handling thousands of requests a minute. Typical architecture for such servers is to rely on threads to handle lots of requests, but that carries with it lots of memory/processing overhead not to mention how tricky it is to get multi-threaded systems to work right.

Instead, Node.js paradigm is that when the result is ready it will invoke the provided callback function. That way Node.js can get back to the business of handling requests with very little overhead.

The cost is that the resulting value lands in an inconvenient place. Rather than landing at the next line of code, it's inside a callback function.

The newest version of Node.js, v8.x, includes native support for async/await functions. This is an amazing advance that tames asynchronous programming patterns in Node.js, making it a little bit like synchronous coding patterns in other languages.

const path = require('path');
const fs   = require('fs');

async function readFile(srcdirectory, file2read) {
    const text = await new Promise((resolve, reject) => {
        fs.readFile(path.join(srcdirectory, file2read), 'utf8', (err, text) => {
            if (err) reject(err);
            else resolve(text);
        });
    });
    ... do something with text;
    return text;
}

readFile("path/to/some/where", "file.txt")
.then(text => {
    ... do something with text
})
.catch(err => {
    console.error(err);
});

Inside an async function, asynchronous code must either return a Promise or else be called within the Promise object as shown here. A Promise is a new way of handling code that will execute in the future. Once the result of the code is known you either call reject to indicate an error, or resolve to indicate the resulting value.

The await keyword arranges things such that the result of the Promise is returned. It creates some magic allowing asynchronous code to execute, but your code looks more like synchronous code of other programming languages.

Unfortunately most Node.js packages have not been written to return Promise's and therefore we have to jump through a small hoop to get fs.readFile to return a Promise. There is a simple way get a version of this function, or others, that instead returns a Promise.

const path = require('path');
const fs   = require('fs');
const util = require('util');
const fs_readFileAsync = util.promisify(fs.readFile);

async function readFile(srcdirectory, file2read) {
    const text = await fs_readFileAsync(path.join(srcdirectory, file2read), 'utf8');
    ... do something with text;
    return text;
}

readFile("path/to/some/where", "file.txt")
.then(text => {
    ... do something with text
})
.catch(err => {
    console.error(err);
});

The util.promisify function takes a function using the normal Node.js convention, and makes a version that returns a Promise. The normal Node.js convention is for the final function argument to be the callback function, and that the first argument to the callback function is an error indicator.

All async functions themselves return a Promise, and we show in this code snippet the typical way of dealing with a Promise.

« Books and videos so you can easily learn Node.js programming Troubles with Asynchronous code flows in JavaScript, and the async/await solution of ES-2017 »
2016 Election Acer C720 Ad block AkashaCMS Amiga Android Anti-Fascism Apple Apple Hardware History Apple iPhone Apple iPhone Hardware April 1st Arduino ARM Compilation Astronomy Asynchronous Programming Authoritarianism Automated Social Posting Bells Law Big Brother Big Finish Black Holes Blade Runner Blogger Blogging Books Botnet Botnets Cassette Tapes Cellphones Christopher Eccleston Chrome Chrome Apps Chromebook Chromebooks Chromebox ChromeOS CIA CitiCards Citizen Journalism Civil Liberties Clinton Cluster Computing Command Line Tools Computer Hardware Computer Repair Computers Cross Compilation Crouton Curiosity Rover Cyber Security Cybermen Daleks Darth Vader Data backup Data Storage Database Database Backup Databases David Tenant DDoS Botnet Detect Adblocker Developers Editors Digital Photography DIY DIY Repair DNP3 Docker Doctor Who Doctor Who Paradox Drobo Drupal Drupal Themes DVD Early Computers Election Hacks Electric Bicycles Electric Vehicles Electron Emdebian Energy Efficiency Enterprise Node ESP8266 Ethical Curation Eurovision Event Driven Asynchronous Express Facebook Fake News Fedora VirtualBox File transfer without iTunes FireFly Fraud Freedom of Speech Gallifrey git Gitlab GMAIL Google Google Chrome Google Gnome Google+ Government Spying Great Britain Heat Loss Home Automation HTTPS I2C Protocol Image Analysis Image Conversion Image Processing ImageMagick InfluxDB Infrared Thermometers Insulation Internet Internet Advertising Internet Law Internet of Things Internet Policy Internet Privacy iOS Devices iPad iPhone iPhone hacking Iron Man Iternet of Things iTunes Java JavaScript JavaScript Injection JDBC John Simms Journalism Joyent Kindle Marketplace Lets Encrypt LibreOffice Linux Linux Hints Linux Single Board Computers Logging Mac OS Mac OS X MacOS X setup Make Money Online MariaDB Mars Matt Lucas MEADS Anti-Missile Mercurial Michele Gomez Military Hardware Minification Minimized CSS Minimized HTML Minimized JavaScript Missy Mobile Applications MODBUS Mondas MongoDB Mongoose Monty Python MQTT Music Player Music Streaming MySQL NanoPi Nardole NASA Net Neutrality Node Web Development Node.js Node.js Database Node.js Testing Node.JS Web Development Node.x North Korea Online advertising Online Fraud Online Journalism Online Video Open Media Vault Open Source Governance Open Source Licenses Open Source Software OpenAPI OpenVPN Personal Flight Peter Capaldi Photography PHP Plex Media Server Political Protest Postal Service Power Control Privacy Production use Public Violence Raspberry Pi Raspberry Pi 3 Raspberry Pi Zero Recycling Remote Desktop Republicans Retro-Technology Reviews Right to Repair River Song Rocket Ships RSS News Readers rsync Russia Russia Troll Factory SCADA Scheme Science Fiction Season 1 Season 10 Season 11 Security Security Cameras Server-side JavaScript Shell Scripts Silence Simsimi Skype Social Media Warfare Social Networks Software Development Space Flight Space Ship Reuse Space Ships SpaceX Spring Spring Boot SQLite3 SSD Drives SSD upgrade SSH SSH Key SSL Swagger Synchronizing Files Telescopes Terrorism The Cybermen The Daleks The Master Time-Series Database Torchwood Total Information Awareness Trump Trump Administration Ubuntu UDOO Virtual Private Networks VirtualBox VLC VNC VOIP Web Applications Web Developer Resources Web Development Web Development Tools Web Marketing Website Advertising Weeping Angels WhatsApp Window Insulation Wordpress YouTube