Tags: Node.JS »»»» TypeScript
Decorators are an interesting feature of TypeScript, which may become a standard part of JavaScript. They resemble annotations in other languages, in that they're attached to the declaration of something and add behavior or information to that thing. To get some familiarity, let us use a library of data validation decorators along with TypeORM.
Our reason for using TypeScript is mostly because the compiler helps us catch certain types of errors through enforcing data types on the objects we use. But, TypeScript's type enforcement only happens at compile time, and there's no data type verification at runtime.
We could have an application maintaining data in a database. We write the code in TypeScript, and the compiler is helping us ensure the code is correct. But, bad data can get into the database unless we have runtime data verification.
This issue isn't just about runtime verification of data types. It's also about whether the data fits constraints for the data. For example your application might be a student database for a University registrar office. The year the student enters the University won't be in 1231 A.D. nor will their entry date be 2341 A.D. In other words, the likely range that a currently active student will have entered the University starts in the 1960's, and ends with "this year".
It's possible to generate a Student record that could be a row in a database, like this:
class Student {
name: string;
entered: number;
grade: number;
gender: Gender;
}
But, not all "strings" are legitimate as human names. The year a student enters a University is an integer, and as we just discussed there are likely values for this field. The grade field is meant to cover whether they're a junior or sophomore or senior in the school. And, gender, has a constrained set of possible values.
But this simple data type declaration allows for many incorrect values.
In other words, it is important to validate application data. A part of validation is ensuring the values are the correct type, and TypeScript helps us greatly with this. But it doesn't help us ensure a particular value in the correct range, or the correct format, for our application.
Using TypeScript gave us automated compile time type checking. In this tutorial we'll explore using a TypeScript decorator library with which one can automate runtime validation of data values. We'll do this for TypeORM entities, as well as regular objects.
This article is loosely derived from my book:
TypeScript Decorators
Before we start start on code, we need to discuss what TypeScript calls Decorators. These are markers which can be attached to classes or methods that have a passing similarity to what Java calls Annotations. This means that Decorators can attach information or functionality to classes or methods in JavaScript.
The TypeScript documentation describes Decorators this way:
.... Decorators provide a way to add both annotations and a meta-programming syntax for class declarations and members. Decorators are a stage 2 proposal for JavaScript and are available as an experimental feature of TypeScript.
A Decorator is a special kind of declaration that can be attached to a class declaration, method, accessor, property, or parameter. Decorators use the form @expression, where expression must evaluate to a function that will be called at runtime with information about the decorated declaration.
For a detailed description see:
-
https://github.com/tc39/proposal-decorators -- In the due course of time it is expected that decorators will be a standard part of the ECMAScript language. This is the ECMAScript committee working on making that happen.
Decorators look like this:
@ClassDecorator()
class A {
@PropertyDecorator()
name: string;
@MethodDecorator()
fly(
@ParameterDecorator()
meters: number
) {
// code
}
@AccessorDecorator()
get egg() {
// code
}
}
In other words, it is the @ symbol, followed by a function name. It's not that the decorator name looks like a function name, it is a function name. The parentheses may be optional, because the available examples show them used with and without parentheses without explanation. Decorators can receive arguments, in which case the parentheses are required.
Because decorators are an experimental feature of TypeScript, the feature must be enabled in tsconfig.json like so:
{
"compilerOptions": {
...
"experimentalDecorators": true,
"emitDecoratorMetadata": true
...
}
}
Because decorators are on their way to becoming a standard part of JavaScript, it's well worth our time to understand what they do, and even how to implement them. They may look magical, but each decorator is implemented by a function, and with enough study of both the decorator and reflection API's we too could start implementing them.
For those of us who've spent time writing Java or C# code, we've probably had experience with Annotations. These decorators look very similar to Annotations, and serve roughly the same purpose.
TypeORM
TypeORM is an Object-Relations-Manager (ORM) for JavaScript that has good support for use within TypeScript. Generally the goal for ORM's is to act as a high level abstraction over SQL databases. Instead of writing SQL, and dealing with serializing data between SQL and database tables, the ORM layer takes care of those details.
TypeORM is very similar to Hibernate (from the Java EE world) and also cites influence from Doctrine and Entity Framework.
See:
https://typeorm.io/
Writing code in TypeORM requires using Decorators. For part of this tutorial we'll discuss using decorators to validate TypeORM entities.
Setting up a Node.js project with TypeScript and TypeORM
That was a lot of theory and introduction, so lets get started on some code.
Start with these commands to set up a project directory.
$ mkdir validation-example
$ cd validation-example
$ npm init -y
$ npm install typescript @types/node --save-dev
$ npm install typeorm --save
$ npm install sqlite3 --save
$ npm install class-validator reflect-metadata --save
$ npm install log-process-errors --save
You'll find a package.json that's been setup, and you have some packages that will be useful for this little demo. Installing TypeScript and TypeORM will give us a platform to code with. We'll save some data in a database using SQLite3. The class-validator package will handle the data validation we want to examine. My book, Quick Start to TypeScript etc, goes over these tools in more detail.
The log-process-errors package will automatically catch errors and show them to us. To learn about this package, see Simplify catching uncaughtException, unhandledRejection, and multipleResolves in Node.js
After installing the typeorm package, you'll see that a typeorm command has been installed. Do not get tempted to run typeorm init to initialize a project. The project generated that way uses an ancient version of TypeScript that doesn't support anything modern.
Modify the package.json with these scripts entries.
"scripts": {
"build": "npx tsc",
"watch": "npx tsc -w"
},
These two scripts help us build our code. The watch command will let us automatically rebuild code as we change it, which is a great convenience.
Create a file named tsconfig.json containing:
{
"compilerOptions": {
"lib": [ "es6", "es2021", "esnext" ],
"target": "es2021",
"module": "commonjs",
"moduleResolution": "Node",
"outDir": "./dist-cjs",
"rootDir": "./src",
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
"declaration": true,
"declarationMap": true,
"inlineSourceMap": true,
"inlineSources": true
}
}
This file configures the TypeScript compiler. These settings configure it to produce CommonJS modules because TypeORM can only be used from a CJS module. The generated code uses the latest JavaScript. The source code will be in src and the generated code is in dist-cjs. The other settings are required for properly supporting TypeScript and the use of Decorators.
Now, create a file named ormconfig.yml containing this:
default:
type: "sqlite"
database: "registrardb.sqlite"
synchronize: true
logging: false
We're going to configure TypeORM so that it can read this configuration file. It sets up TypeORM to use SQLite and the database will land in registrardb.sqlite.
Create a directory, src, and in that directory create a file named index.ts containing:
import "reflect-metadata";
import {
createConnection, getConnectionOptions,
getRepository
} from "typeorm";
import { Student } from "./entity/Student";
export { Student } from './entity/Student';
export async function connect() {
// One option is this... to pass in a hard-coded configuration
// _connection = await createConnection({
// ... hard-coded connection object
// });
// The next option is to read the configuration from
// the environment or an ormconfig file, and then
// to insert the entities configuration.
const connectionOptions = await getConnectionOptions();
Object.assign(connectionOptions, {
entities: [
Student
]});
return await createConnection(connectionOptions);
}
This imports some packages. The connect function handles connecting to TypeORM. The getConnectionOptions function looks either at environment variables or ormconfig.yml to find dynamically supplied connection options.
As the comments say, it's possible to use a hard-coded configuration object. But that's not how one would use TypeORM in production.
Coded this way we have the greatest flexibility. Configuration settings can come either from the ormconfig, from environment variables, or from selected hard-coded values. In particular, does it make sense to force a configuration file to list the entities used in the application? Nope. Hence, this is coded so that the only hard-coded value is the list of Entity classes, and everything else comes from external configuration.
Defining a TypeORM Entity with validation decorators
We need a data type with which we can experiment with data validation. Let's take the Student example discussed earlier and run with it.
Create a directory named src/entities and in that directory create a file named Student.ts containing:
import {
Entity, Column,
PrimaryGeneratedColumn,
BeforeInsert, BeforeUpdate
} from "typeorm";
import {
validateOrReject,
ValidatorConstraint,
ValidatorConstraintInterface,
IsInt, Matches, Min, Max,
IsString, IsIn, IsAscii,
IsOptional,
} from 'class-validator';
// Regular expression to match against names
const re_name = /^[a-zA-Z ]+$/;
@ValidatorConstraint()
class IsName implements ValidatorConstraintInterface {
validate(text: string) {
let matcher = text.match(re_name);
let ret = typeof matcher !== 'undefined'
&& Array.isArray(matcher)
&& matcher.length > 0;
return ret;
}
}
@Entity()
export class Student {
@PrimaryGeneratedColumn() id: number;
@Column({ length: 100 })
@IsString() @IsAscii()
@Matches(re_name)
// @Validate(IsName, {
// message: 'Is not a name',
// })
name: string;
@Column("int")
@IsInt() @Min(1900) @Max(2040)
entered: number;
@Column("int")
@IsInt() @Min(1) @Max(8)
grade: number;
@Column()
@IsString() @IsAscii()
@IsIn(['male', 'female'])
gender: string;
@BeforeInsert()
async validateInsert() {
await validateOrReject(this);
}
@BeforeUpdate()
async validateUpdate() {
await validateOrReject(this);
}
}
The imports from the typeorm and class-validator packages are all functions implementing TypeScript decorators. For every TypeScript decorator, there is a function that is the actual implementation. We won't go into the implementation details, so if you're curious to learn more look in the TypeScript documentation.
A TypeORM entity is a TypeScript class, but you use TypeORM decorators to attach behavior to the class.
For example, @Column says the field is a database column, and you pass in a descriptor saying what data type to use in the column. With @PrimaryGeneratedColumn we tell TypeORM to generate a column that automates generating an ID number.
The other decorators used here come from the class-validator package, and are used to describe various validation checks to use. The name of each does a good job of describing its purpose, for example @IsInt clearly asserts the item is an integer.
The @BeforeInsert and @BeforeUpdate decorators come from TypeORM. These trigger a function call before TypeORM inserts data into the database, or before it updates data in the database. Notice that these decorators trigger functions which call validateOrReject comes from class-validator. That function checks the data in the object instance against the validation decorators attached to each field.
These two functions are therefore the point of this exercise. They are how we validate data in TypeScript. The method is precisely this:
- Attach decorators from
class-validatordescribing the legitimate values for each field - Call
validateOrReject
The last function will throw an error if the object does not pass the validation checks.
TypeORM functions to add/query Entities
To finish up the example code, let's go back to src/index.ts and add a couple utility functions.
export async function addStudent(student: Student)
: Promise<number> {
const stud1 = new Student();
stud1.name = student.name;
stud1.entered = student.entered;
stud1.grade = student.grade;
stud1.gender = student.gender;
await getRepository(Student).save(stud1);
return stud1.id;
}
This takes an object describing a Student object, creates an instance of Student, then calls a TypeORM method to save that object to the database. In the process of doing this, TypeORM will invoke the @BeforeInsert decorator, and therefore run the validation checks we just discussed.
TypeORM also has an update operation available. We won't exercise that operation, but it invokes @BeforeUpdate.
export async function findStudent(id: number)
: Promise<Student[]> {
return getRepository(Student).find({
where: { id: id }
});
}
export async function findStudentByName(name: string)
: Promise<Student[]> {
return getRepository(Student).find({
where: { name: name }
});
}
export async function listStudents()
: Promise<Student[]> {
return getRepository(Student).find();
}
These functions handle looking up Student objects in the database.
Remember to build the source code, generating the JavaScript for execution:
$ npm run build
> typeorm1@1.0.0 build
> npx tsc
If you like, examine the generated files in the dist-cjs directory. It's interesting comparing the source with these files to see what TypeScript is doing.
Testing data valuation in TypeScript/TypeORM Entities
We need some kind of tool to exercise the code we just created. For that purpose let's create a file run.mjs in the same directory where package.json sits. As the file name implies, this will be a regular JavaScript file, written in ES6 module format.
import {
connect, Student, addStudent, listStudents
} from './dist-cjs/index.js';
import logProcessErrors from 'log-process-errors'
/*
const env = process.env;
env.TYPEORM_CONNECTION = 'sqlite';
env.TYPEORM_DATABASE = './registrardb.sqlite';
env.TYPEORM_SYNCHRONIZE = "true";
env.TYPEORM_LOGGING = "false";
*/
logProcessErrors({});
const connection = await connect();
await addStudent({
name: 'Joe Bloe',
entered: 1992,
grade: 2,
gender: 'male'
});
await addStudent({
name: 'Josephine Bloe',
entered: 1992,
grade: 2,
gender: 'female'
});
await addStudent({
name: 'John 123dmc',
entered: '1992',
grade: '2',
gender: 'female-like'
});
console.log(await listStudents());
At the top we import some things, including the functions we just created. We also import log-process-errors and call its function. This package catches uncaught errors, and prints them out, which will be useful to verify that we know about all failures.
The lines about using process.env and setting env.BLAH_DE_BLAH to values are an alternate way of configuring TypeORM. We can set environment variables to set its configuration, and this is one way of setting environment variables. Because those lines are commented out, the only source for configuration is ormconfig.yml.
Below that we connect to the database, call addStudent a few times, and then call listStudents.
For now, comment out the third addStudent call. Then run this:
$ node run.mjs
[
Student {
id: 1,
name: 'Joe Bloe',
entered: 1992,
grade: 2,
gender: 'male'
},
Student {
id: 2,
name: 'Josephine Bloe',
entered: 1992,
grade: 2,
gender: 'female'
}
]
Joe and Josephine get added to the database just fine. We can verify this quite easily by using SQLite3 tools to query the database:
$ sqlite3 registrardb.sqlite
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> select * from student;
1|Joe Bloe|1992|2|male
2|Josephine Bloe|1992|2|female
sqlite>
Now, remove the comments around the third addStudent call, and then pay attention to the values. Every one of them violate a validation check.
Rerun the script:
$ rm registrardb.sqlite
$ node run.mjs
✖ uncaughtException (an exception was thrown but not caught) [
ValidationError {
target: Student {
name: 'John 123dmc',
entered: '1992',
grade: '2',
gender: 'female-like'
},
value: 'John 123dmc',
property: 'name',
children: [],
constraints: { matches: 'name must match /^[a-zA-Z ]+$/ regular expression' }
},
...
]
$ sqlite3 registrardb.sqlite
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> select * from student;
1|Joe Bloe|1992|2|male
2|Josephine Bloe|1992|2|female
sqlite>
First, we deleted the database to make sure we start fresh.
This time a ValidationError was generated because we purposely fed invalid data. The output includes all validators that failed, we're just showing the first here in the interest of space. This means our validation worked.
Notice that it says uncaughtException. That's because no attempt was made to catch anything. There should have been try/catch around each addStudent call because of the possibility that an error would be thrown. Instead we installed log-process-errors which ensured the uncaught errors were caught.
Finally, we can rerun the sqlite3 command to inspect the database. That says the first two items were successfully created, but the third was not because it failed.
Validation does not depend on TypeORM
So far we demonstrated data validation while saving data to the database using TypeORM. But it looks like we can call validatorOrReject separately at any time we like.
In src/index.ts we can add this function:
export function mkStudent(student: Student): Student {
const stud1 = new Student();
stud1.name = student.name;
stud1.entered = student.entered;
stud1.grade = student.grade;
stud1.gender = student.gender;
return stud1;
}
This generates a Student object, as in addStudent, but does not save this object to the database. You might find it useful to refactor addStudent to call this function.
We did this to have a code path that does not execute TypeORM code. That means validateOrReject is not called as a byproduct of inserting the object into the database. This will lets us verify that validation decorators do nothing until validateOrReject is called.
To exercise this, create a new script called mk.mjs containing:
import { mkStudent } from './dist-cjs/index.js';
import { validateOrReject } from 'class-validator';
import logProcessErrors from 'log-process-errors';
logProcessErrors({});
const stud1 = mkStudent({
name: 'Joe Bloe',
entered: 1992,
grade: 2,
gender: 'male'
});
await validateOrReject(stud1);
console.log(stud1);
const stud2 = mkStudent({
name: 'Joe Bloe123',
entered: '1992',
grade: '2',
gender: 'ro-male'
});
await validateOrReject(stud2);
console.log(stud2);
This imports mkStudent and validateOrReject. We construct a pair of Student objects using mkStudent, and for each call validateOrReject. The second of these objects is constructed to fail validation.
Remember to run npm run build to rebuild the source. Then run mk.mjs as so:
$ node mk.mjs
Student { name: 'Joe Bloe', entered: 1992, grade: 2, gender: 'male' }
✖ unhandledRejection (a promise was rejected but not handled) [
ValidationError {
target: Student {
name: 'Joe Bloe123',
entered: '1992',
grade: '2',
gender: 'ro-male'
},
value: 'Joe Bloe123',
property: 'name',
children: [],
constraints: { matches: 'name must match /^[a-zA-Z ]+$/ regular expression' }
},
...
]
The first item is successfully constructed, and because we see it printed out we know that validateOrReject felt it is okay. But, the second failed validation, because every one of its data values do not match the constraints. At no time did either object go through TypeORM, instead we called validateOrReject directly.
To make this even clearer, let's create a class that does not have any TypeORM decorators. In src create a file named foo.js containing:
import {
IsInt, Min, Max,
} from 'class-validator';
export class Foo {
@IsInt() @Min(1900) @Max(2040)
year: number;
}
This class has a member, year, with the constraint of being between 1900 and 2040.
Now, create a script named foorun.mjs containing:
import { Foo } from './dist-cjs/foo.js';
import { validateOrReject } from 'class-validator';
import logProcessErrors from 'log-process-errors';
logProcessErrors({});
const foo1 = new Foo();
foo1.year = 1999;
validateOrReject(foo1);
const foo2 = new Foo();
foo2.year = 1999999;
validateOrReject(foo2);
This loads the Foo class, the validateOrReject function, and other tools. We create two instances of this object, the first with a valid value, and the second with one that fails validation.
Run npm run build to rebuild the source. Then run the script as so:
$ node foorun.mjs
✖ unhandledRejection (a promise was rejected but not handled) [
ValidationError {
target: Foo { year: 1999999 },
value: 1999999,
property: 'year',
children: [],
constraints: { max: 'year must not be greater than 2040' }
}
]
And, we see that the first item passed validation, and the second failed. We can tell because the value reported as failing is the second one.
In this case there is nothing special about the Foo class. It's not attached to anything, but is a normal class we defined ourselves. We decided when and where to validate the contents of each instance. And, we received a useful indication of which was incorrect.
Conclusion
With the class-validator library we can implement semi-automatic data validation in TypeScript. All that's required is to attach suitable decorators to fields in a class, then run validateOrReject against instances of that class. Voila, Q.E.D., we know the instance fits the constraints we attach to the class.
We didn't mention this earlier, but it is easy to implement a custom validator. You see an example in Student.ts. This involves creating a class whichs implements a given interface, and has the @ValidatorConstraint decorator. The validate method then implements the desired validation check. Once you've implemented the class, it is invoked as shown in the commented-out code in the Student class.
One technique recommended by the TypeScript team is creating what type guard functions. These test an object to verify that it is, or is not, of the given type. It seems feasible to use class-validator decorators as part of a type guard function.
Books by David Herron
(Sponsored)
