; Date: Mon Apr 20 2020

Tags: Node.js »»»» Node.js Installation »»»» Ubuntu

Node.js 14.x was just released, as was Ubuntu 20.04. Testing my application on that combination failed because native code npm packages fail because python is not found. WTF? I followed the instructions and installed build-essential which supposedly brings in every compiler tool we would need. But ... fortunately there is a simple solution.

Remember that when Node.js installs a native-code package, a build tool (node-gyp) is used. That build tool relies on Python. For Node.js 14.x, the team transitioned the requirements such that node-gyp must be executed with Python3, because the Python team has end-of-life'd Python2.

As we'll see, some of the packages in the npm repository are affected by being unable to install because there is no python command. By default Ubuntu installs Python 3, but it follows the Python community requirements of installing it as python3. Therefore a script trying to run python will fail because there is no python command.

The actual bug here is the script which tries to run python. The Python community has required for several years now that Python be installed as python2 and python3, and therefore scripts must accommodate either one. Therefore an npm package that fails to install because python is not found is in error.

I don't know where it's clearly stated the installation requirements to not only install Node.js, but to support installing native code modules. The npm documentation doesn't document the requirements. The document on building Node.js from source talks about Python 2 being eschewed in favor of Python 3, but suggests that Python 2 is used if it is found. The NodeSource page on installing on Linux from their package repository says to do only this:

apt-get install -y build-essential

Further, when you install Node.js on Ubuntu from the Nodesource package repository, you're told this:

## Run `sudo apt-get install -y nodejs` to install Node.js 13.x and npm
## You may also need development tools to build native addons:
     sudo apt-get install gcc g++ make

Hence, that results in Python 3 being the only Python release installed. If we instead install build-essential, Python 2 does get installed alongside Python 3.

I have installed - using Multipass - an Ubuntu 20.4 instance. To set up Node.js, I ran these commands:

    1  curl -sL https://deb.nodesource.com/setup_13.x | sudo -E bash -
    2  sudo apt-get update
    3  sudo apt-get upgrade -y
    4  sudo apt-get install -y nodejs build-essential

You'll note this is installing Node.js 13.x (13.13 to be exact) but the same result happens if I install Node.js 14.x instead.

Installing build-essential is useful because it installs all compiler tools, including Python. Supposedly.

$ ls /usr/bin/python*
/usr/bin/python2  /usr/bin/python2.7  /usr/bin/python3  /usr/bin/python3.8

We have python2 and python3 installed, but no command named python. Indeed, if I execute python, I get this:

$ python

Command 'python' not found, did you mean:

  command 'python3' from deb python3
  command 'python' from deb python-is-python3

There is no python command installed, and a couple suggestions are made. Ubuntu kindly makes suggestions like this to help guide us in which packages to install.

$ python3
Python 3.8.2 (default, Mar 13 2020, 10:14:16) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

Because Python 3 is installed, we can run it using the python3 command. Bottom line is that on Ubuntu 20.04 there is not a command named python, only python2 and python3.

As we'll see, for some native code packages in the npm repository, because there is no python command. But.. that's by intention rather than a bug.

But, wait, the Python community wants us to use python2 and python3 commands

This issue will affect any other tool that expects to run the python command. But it seems that my lack of knowledge about Python has led me astray. I thought that the python command must be updated to equate to Python 3 because Python 3 is now the default. But looking at the Ubuntu Wiki, and then at PEP-0394, I see the situation is more nuanced than that.

Namely, the Python community has been in a transition process for many years. There are some Python scripts that only support Python 2, some that only support Python 3, and some that support either. It's a problem if the python command executes Python 3, because Python-2-only scripts will fail to run.

In the name of backwards compatibility, Unix/Linux/etc distributions are required to install Python 2 as python2 and to install Python 3 as python3.

Therefore, Ubuntu is following the requirements laid out by the Python community. The lack of a python command is by design rather than by mistake.

Back to installing npm packages

Let's try installing some native code packages and see what happens. To do so, we set up a test directory, and install packages that require native code installation.

    5  mkdir t
    6  cd t
    7  npm init
    8  node --version
    9  npm install bcrypt --save
   10  python
   11  npm install sqlite3 --save

Installing bcrypt works fine. But installing sqlite3 fails with this:

$ npm install sqlite3 --save

> sqlite3@4.2.0 install /home/ubuntu/t/node_modules/sqlite3
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using needle for node-pre-gyp https download 
node-pre-gyp WARN Pre-built binaries not installable for sqlite3@4.2.0 and node@13.13.0 (node-v79 ABI, glibc) (falling back to source compile with node-gyp) 
node-pre-gyp WARN Hit error Remote end closed socket abruptly. 
node-pre-gyp WARN Pre-built binaries not installable for sqlite3@4.2.0 and node@13.13.0 (node-v79 ABI, glibc) (falling back to source compile with node-gyp) 
node-pre-gyp WARN Hit error bad download 
make: Entering directory '/home/ubuntu/t/node_modules/sqlite3/build'
  ACTION deps_sqlite3_gyp_action_before_build_target_unpack_sqlite_dep Release/obj/gen/sqlite-autoconf-3310100/sqlite3.c
make: Entering directory '/home/ubuntu/t/node_modules/sqlite3/build'
  ACTION deps_sqlite3_gyp_action_before_build_target_unpack_sqlite_dep Release/obj/gen/sqlite-autoconf-3310100/sqlite3.c
/bin/sh: 1: python: not found
make: *** [deps/action_before_build.target.mk:13: Release/obj/gen/sqlite-autoconf-3310100/sqlite3.c] Error 127
make: Leaving directory '/home/ubuntu/t/node_modules/sqlite3/build'

There was more output, but the key is there - python: not found.

For libxslt the installation fails, but for another reason having to do with source incompatibilities.

Some native code packages, like sharp, provide prebuilt binaries and do not require compilation on the target system. In the case of SQLite3, that team does the same, but is apparently not supporting Ubuntu 20.04 with prebuilt binaries. Likewise, the SQLite3 team does not yet support Node.js 14.x with prebuilt binaries.

I cannot think of other native code packages but we have found one native code library that fails to install because the python command does not exist.

The solution - that breaks the Python community recommendations

There is a very simple solution that is alluded to above. Namely:

$ sudo apt-get install python-is-python3

When we ran the python command earlier, the message suggested we might find that command in the python-is-python3 package. So we install that package and now the python command works:

$ python
Python 3.8.2 (default, Mar 13 2020, 10:14:16) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

And we can now successfully install SQLite3:

$ npm install sqlite3 --save

> bcrypt@4.0.1 install /home/ubuntu/t/node_modules/bcrypt
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using needle for node-pre-gyp https download 
node-pre-gyp WARN Pre-built binaries not installable for bcrypt@4.0.1 and node@13.13.0 (node-v79 ABI, glibc) (falling back to source compile with node-gyp) 
node-pre-gyp WARN Hit error Remote end closed socket abruptly. 
make: Entering directory '/home/ubuntu/t/node_modules/bcrypt/build'
  CC(target) Release/obj.target/nothing/../node-addon-api/src/nothing.o
  AR(target) Release/obj.target/../node-addon-api/src/nothing.a
  COPY Release/nothing.a
  CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o
  CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o

Well, "success" is a relative term since there is a zillion warning messages printed because of deprecated this's and that's. But it no longer fails on the missing python command.

But of course any Python script that is only compatible with Python 2 will now fail unless we remember to run the python2 command.

To do this magic, a symbolic link is add to /usr/bin:

$ ls -l /usr/bin/python
lrwxrwxrwx 1 root root 7 Apr 15 03:45 /usr/bin/python -> python3

I found strong recommendations to not do this because obviously Python2 scripts will fail to run.

Installing the python-is-python3 package lets us run a script that expects to find a python command. It sets up a condition that is incompatible with Python community requirements. We're supposed to transition to using python2 for Python 2 scripts, and python3 for Python 3 scripts, and to eschew the python command. But sometimes pragmatism wins out over such clearly stated technical requirements.