; Date: Mon Apr 20 2020
Node.js 14.x was just released, as was Ubuntu 20.04. Testing my application on that combination failed because native code npm packages fail because python is not found. WTF? I followed the instructions and installed build-essential which supposedly brings in every compiler tool we would need. But ... fortunately there is a simple solution.
Remember that when Node.js installs a native-code package, a build tool (node-gyp) is used. That build tool relies on Python. For Node.js 14.x, the team transitioned the requirements such that
node-gyp must be executed with Python3, because the Python team has end-of-life'd Python2.
As we'll see, some of the packages in the npm repository are affected by being unable to install because there is no
python command. By default Ubuntu installs Python 3, but it follows the Python community requirements of installing it as
python3. Therefore a script trying to run
python will fail because there is no
The actual bug here is the script which tries to run
python. The Python community has required for several years now that Python be installed as
python3, and therefore scripts must accommodate either one. Therefore an npm package that fails to install because
python is not found is in error.
I don't know where it's clearly stated the installation requirements to not only install Node.js, but to support installing native code modules. The npm documentation doesn't document the requirements. The document on building Node.js from source talks about Python 2 being eschewed in favor of Python 3, but suggests that Python 2 is used if it is found. The NodeSource page on installing on Linux from their package repository says to do only this:
apt-get install -y build-essential
Further, when you install Node.js on Ubuntu from the Nodesource package repository, you're told this:
## Run `sudo apt-get install -y nodejs` to install Node.js 13.x and npm ## You may also need development tools to build native addons: sudo apt-get install gcc g++ make
Hence, that results in Python 3 being the only Python release installed. If we instead install
build-essential, Python 2 does get installed alongside Python 3.
I have installed - using Multipass - an Ubuntu 20.4 instance. To set up Node.js, I ran these commands:
1 curl -sL https://deb.nodesource.com/setup_13.x | sudo -E bash - 2 sudo apt-get update 3 sudo apt-get upgrade -y 4 sudo apt-get install -y nodejs build-essential
You'll note this is installing Node.js 13.x (13.13 to be exact) but the same result happens if I install Node.js 14.x instead.
build-essential is useful because it installs all compiler tools, including Python. Supposedly.
$ ls /usr/bin/python* /usr/bin/python2 /usr/bin/python2.7 /usr/bin/python3 /usr/bin/python3.8
python3 installed, but no command named
python. Indeed, if I execute python, I get this:
$ python Command 'python' not found, did you mean: command 'python3' from deb python3 command 'python' from deb python-is-python3
There is no python command installed, and a couple suggestions are made. Ubuntu kindly makes suggestions like this to help guide us in which packages to install.
$ python3 Python 3.8.2 (default, Mar 13 2020, 10:14:16) [ ] on linux Type "help", "copyright", "credits" or "license" for more information. >>>
Because Python 3 is installed, we can run it using the
python3 command. Bottom line is that on Ubuntu 20.04 there is not a command named
As we'll see, for some native code packages in the npm repository, because there is no
python command. But.. that's by intention rather than a bug.
But, wait, the Python community wants us to use
This issue will affect any other tool that expects to run the
python command. But it seems that my lack of knowledge about Python has led me astray. I thought that the
python command must be updated to equate to Python 3 because Python 3 is now the default. But looking at the Ubuntu Wiki, and then at
PEP-0394, I see the situation is more nuanced than that.
Namely, the Python community has been in a transition process for many years. There are some Python scripts that only support Python 2, some that only support Python 3, and some that support either. It's a problem if the
python command executes Python 3, because Python-2-only scripts will fail to run.
In the name of backwards compatibility, Unix/Linux/etc distributions are required to install Python 2 as
python2 and to install Python 3 as
Therefore, Ubuntu is following the requirements laid out by the Python community. The lack of a
python command is by design rather than by mistake.
Back to installing npm packages
Let's try installing some native code packages and see what happens. To do so, we set up a test directory, and install packages that require native code installation.
5 mkdir t 6 cd t 7 npm init 8 node --version 9 npm install bcrypt --save 10 python 11 npm install sqlite3 --save
Installing bcrypt works fine. But installing sqlite3 fails with this:
$ npm install sqlite3 --save > email@example.com install /home/ubuntu/t/node_modules/sqlite3 > node-pre-gyp install --fallback-to-build node-pre-gyp WARN Using needle for node-pre-gyp https download node-pre-gyp WARN Pre-built binaries not installable for firstname.lastname@example.org and email@example.com (node-v79 ABI, glibc) (falling back to source compile with node-gyp) node-pre-gyp WARN Hit error Remote end closed socket abruptly. node-pre-gyp WARN Pre-built binaries not installable for firstname.lastname@example.org and email@example.com (node-v79 ABI, glibc) (falling back to source compile with node-gyp) node-pre-gyp WARN Hit error bad download make: Entering directory '/home/ubuntu/t/node_modules/sqlite3/build' ACTION deps_sqlite3_gyp_action_before_build_target_unpack_sqlite_dep Release/obj/gen/sqlite-autoconf-3310100/sqlite3.c make: Entering directory '/home/ubuntu/t/node_modules/sqlite3/build' ACTION deps_sqlite3_gyp_action_before_build_target_unpack_sqlite_dep Release/obj/gen/sqlite-autoconf-3310100/sqlite3.c /bin/sh: 1: python: not found make: *** [deps/action_before_build.target.mk:13: Release/obj/gen/sqlite-autoconf-3310100/sqlite3.c] Error 127 make: Leaving directory '/home/ubuntu/t/node_modules/sqlite3/build'
There was more output, but the key is there - python: not found.
For libxslt the installation fails, but for another reason having to do with source incompatibilities.
Some native code packages, like sharp, provide prebuilt binaries and do not require compilation on the target system. In the case of SQLite3, that team does the same, but is apparently not supporting Ubuntu 20.04 with prebuilt binaries. Likewise, the SQLite3 team does not yet support Node.js 14.x with prebuilt binaries.
I cannot think of other native code packages but we have found one native code library that fails to install because the
python command does not exist.
The solution - that breaks the Python community recommendations
There is a very simple solution that is alluded to above. Namely:
$ sudo apt-get install python-is-python3
When we ran the
python command earlier, the message suggested we might find that command in the
python-is-python3 package. So we install that package and now the
python command works:
$ python Python 3.8.2 (default, Mar 13 2020, 10:14:16) [ ] on linux Type "help", "copyright", "credits" or "license" for more information. >>>
And we can now successfully install SQLite3:
$ npm install sqlite3 --save > firstname.lastname@example.org install /home/ubuntu/t/node_modules/bcrypt > node-pre-gyp install --fallback-to-build node-pre-gyp WARN Using needle for node-pre-gyp https download node-pre-gyp WARN Pre-built binaries not installable for email@example.com and firstname.lastname@example.org (node-v79 ABI, glibc) (falling back to source compile with node-gyp) node-pre-gyp WARN Hit error Remote end closed socket abruptly. make: Entering directory '/home/ubuntu/t/node_modules/bcrypt/build' CC(target) Release/obj.target/nothing/../node-addon-api/src/nothing.o AR(target) Release/obj.target/../node-addon-api/src/nothing.a COPY Release/nothing.a CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o
Well, "success" is a relative term since there is a zillion warning messages printed because of deprecated this's and that's. But it no longer fails on the missing
But of course any Python script that is only compatible with Python 2 will now fail unless we remember to run the
To do this magic, a symbolic link is add to
$ ls -l /usr/bin/python lrwxrwxrwx 1 root root 7 Apr 15 03:45 /usr/bin/python -> python3
I found strong recommendations to not do this because obviously Python2 scripts will fail to run.
python-is-python3 package lets us run a script that expects to find a
python command. It sets up a condition that is incompatible with Python community requirements. We're supposed to transition to using
python2 for Python 2 scripts, and
python3 for Python 3 scripts, and to eschew the
python command. But sometimes pragmatism wins out over such clearly stated technical requirements.