Uploading/mirroring files to remote server in Node.js without using rsync

; Date: Tue Feb 12 2013

Tags: Synchronizing Files »»»» rsync »»»» AkashaCMS »»»» Command Line Tools

How do you upload files to a server to deploy application or website code?  FTP?  rsync?  While it's easy enough to call a command line tool like rsync from a Node.js script, what if you're using a Windows computer that doesn't have those command line tools.  When I use Windows it's like stepping back into the dark ages where directory listings looked like we were making fire by rubbing sticks together.  Okay, there does appear to be an rsync for Windows but I had no confidence in it.  Also, I did not want to have a dependency on something like Cygwin.

Anyway the primary user of AkashaCMS (my girlfriend) likes her Windows machine very much.  After watching her struggle mightily with understanding how to use an FTP program to upload her website, I developed a Node.js script for uploading files that does not rely on rsync.

The script works fairly well but the approach was tricky enough it'd be nice to have some other eyes take a look at it.  The github repository is at https://github.com/robogeek/node-ssh2sync

The approach was to directly use the SSH2 protocol to use SFTP to copy files and to remove excess files on the remote server.  This is done using the SSH2 module for Node.

Previous to developing this tool I'd coded this up, to use rsync:

var user = config.deploy_rsync.user;
var host = config.deploy_rsync.host;
var dir  = config.deploy_rsync.dir;
var rsync = spawn('rsync',
                    [ '--verbose', '--archive', '--delete', config.root_out+'/', user+'@'+host+':'+dir+'/' ],
                    {env: process.env, stdio: 'inherit'});

The purpose of this command was to make the remote directory a duplicate of the local directory.

That's the full extent of functionality in ssh2sync.  To upload files, attempt to set their file times to match the local file times, and remove any excess files on the remote server.

I've tested it and it appears to work between my laptop (Mac OS X) and server (Debian Linux).  That's not a very exhaustive set of tests.  In the workspace is a script, try.js, that I'm using to test the tool.  The options object is the same as is documented for the underlying library.  https://github.com/mscdex/ssh2

It doesn't appear to be setting the file times on the remote server.  But it does appear to be doing everything else I want it to do.

It is, however, rather limited in that it does only the one thing.  The rsync command has a long list of other options, and in theory it appears possible to implement everything rsync does using SFTP.  That means ssh2sync could theoretically be a more comprehensive tool.  But if that is to be, then someone else will have to do it.  At the moment it does everything I require, other than possibly some bugfixing after testing it on my girlfriend's laptop.

Usage in AkashaCMS is fairly simple.  Add configuration in config.js, then run the command

akashacms deploy

It'll be very simple.  Honest.  Much simpler than a GUI FTP program.