; Date: Fri May 27 2022

Tags: Node.JS »»»» TypeScript

TypeScript does excellent compile time type checking. But that does not mean TypeScript code ensures runtime type or data validation or checking. TypeScript compiles code to JavaScript with no runtime checking, leaving the door open to runtime data problems.

Writing TypeScript code with an IDE that automatically shows type errors as you write them is reassuring that your code correctly deals with the data it receives. In theory this makes our applications safer. But, this can lull you into complacency. At the end of the day, TypeScript code becomes JavaScript, and at execution time there is no checking because it is JavaScript.

This means your production code is vulnerable to all kinds failures from bad data. That is, unless you know about defensive programming and correctly check your data. How many of us remember to do that in every place where it is needed? We might have the best of intentions to check data validity at every step every time, but let's be honest and admit that we sometimes forget to do so.

It may be best to implement automatic runtime data checking. Data checking can be built into the definition of a TypeScript class such that it occurs every time you assign data to a field in a class, or every time you invoke a class method.

To see the problem, consider the following:

import { promises as fs } from 'fs';

class jsdata {
    title: string;
    range: number;
}

async function readData(fn: string): Promise<jsdata> {
    const txt = await fs.readFile(fn, 'utf-8');
    const d = JSON.parse(txt);
    const ret = new jsdata();
    ret.title = d.title;
    ret.range = d.range;
    return ret;
}

readData(process.argv[2])
.then(data => {
    console.log(data);
})
.catch(err => {
    console.error(err);
});

It is common for an application to be given JSON data, parse it, store the data in an object, then send that object elsewhere. That's what this example demonstrates.

Let's run it with some good data:

{
    "title": "Fantastic book title",
    "range": 42
}

Run it like so:

$ npx ts-node ./json1.ts ./data1.json
jsdata { title: 'Fantastic book title', range: 42 }

That works as expected, and gives the right values, and the values fit the declared data type.

But, what happens if we give our fantastic algorithm some bad data?

{
    "range": "Fantastic book title",
    "title": 42
}

It's the same data but somehow a bug messed up the labels. This can happen, right?

$ npx ts-node ./json1.ts ./data2.json
jsdata { title: 42, range: 'Fantastic book title' }

Whoops, there were no errors, and now our data object has the wrong data types. The title field is declared as string but it has number instead, with the same kind of problem with range. In a real production application, this sort of error can have real world significant consequences.

We might scratch our head and wonder how that could happen, because the TypeScript class clearly declared the data types. But, take a look at the actual compiled JavaScript:

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = require("fs");
class jsdata {
}
async function readData(fn) {
    const txt = await fs_1.promises.readFile(fn, 'utf-8');
    const d = JSON.parse(txt);
    const ret = new jsdata();
    ret.title = d.title;
    ret.range = d.range;
    return ret;
}
readData(process.argv[2])
    .then(data => {
    console.log(data);
})
    .catch(err => {
    console.error(err);
});

Do you see any data types or type checking here? In fact, the jsdata class has an empty declaration here. This code contains zilch in the way of runtime data validation. This is the code which is executed at runtime, not the TypeScript code shown above.

In other words - if you write the above code and believe you're safe, you've been lulled into lala-land.

What's required is runtime data validation. The TypeScript team recommends using type guards, which are little functions to ensure data is correctly typed. For example, after the JSON.parse line, there would be a line reading isJSData(d) which would check that the required fields are present.

Another approach is the runtime data validation decorators in the runtime-data-validation package. They allow for automatic type checking every time a value is assigned to a field, or every time a method is called.

Rewriting the above example using those decorators looks like this:

import { promises as fs } from 'fs';
import {
    IsInt, IsAscii,
    ValidateParams, ValidateAccessor
} from 'runtime-data-validation';

class jsdata {
    #title: string;

    @ValidateAccessor<string>()
    @IsAscii()
    set title(nt: string) { this.#title = nt; }
    get title(): string { return this.#title; }

    #range: number;

    @ValidateAccessor<number>()
    @IsInt()
    set range(nr: number) { this.#range = nr; }
    get range(): number { return this.#range; }

}

async function readData(fn: string): Promise<jsdata> {
    const txt = await fs.readFile(fn, 'utf-8');
    const d = JSON.parse(txt);
    const ret = new jsdata();
    ret.title = d.title;
    ret.range = d.range;
    return ret;
}

readData(process.argv[2])
.then(data => {
    console.log({ title: data.title, range: data.range });
})
.catch(err => {
    console.error(err);
});

This is largely the same as above. The decorators add some value to the code by making it explicitly clear, in the code, what is the expected data.

While designing the runtime-data-validators package it was found that we could not implement runtime data validation using decorators attached to class attributes. It was only possible to implement this on accessor functions. Therefore, this implementation relies on making the fields title and range as private fields, and to use accessor functions instead.

For the success case there is no change:

$ npx ts-node ./lib/json/json2.ts ./lib/json/data1.json
{ title: 'Fantastic book title', range: 42 }

The code still executes as expected, and produces the correct results.

$ npx ts-node ./lib/json/json2.ts ./lib/json/data2.json
TypeError: Expected a string but received a number
    at assertString (/home/david/Projects/nodejs/runtime-data-validation-typescript/node_modules/validator/lib/util/assertString.js:17:11)
    at Object.isAscii (/home/david/Projects/nodejs/runtime-data-validation-typescript/node_modules/validator/lib/isAscii.js:17:29)
    at /home/david/Projects/nodejs/runtime-data-validation-typescript/lib/decorators/strings.ts:65:31
    at vfunc (/home/david/Projects/nodejs/runtime-data-validation-typescript/lib/index.ts:87:14)
    at jsdata.descriptor.set (/home/david/Projects/nodejs/runtime-data-validation-typescript/lib/index.ts:247:21)
    at readData (/home/david/Projects/nodejs/typescript-decorators-examples/simple/lib/json/json2.ts:29:14)

The failure case is now an actual failure. At runtime the data type was correctly enforced. Further, the stack trace both clearly describes the problem and indicates the correct line of code as the culprit.

Summary

The runtime-data-validation package contains a very long list of data validation decorators. With it your applications should run more safely because data will be automatically checked before it is stored into class fields, or used in class methods.

This package isn't the only solution to the problem of runtime data validation in JavaScript. There are packages like AJV or JOI that aid with data validation. And, JSON can be validated using a JSON Schema along with the @hyperjump/json-schema package.

The problem with those approaches is that they do not prevent invalid data from being assigned to objects after the objects are validated. Suppose you read the JSON as above, then validate it, and assign the JSON data to the jsdata instance. What's to stop your code from then assigning incorrect data to the instance?

With runtime-data-validation, the protection using validation decorators exists for every assignment to a protected field and every use of a protected method.

To learn how these validation decorators were implemented see: Runtime data validation in TypeScript using decorators and reflection metadata