Tags: Amazon Web Services
The AWS platform is an incredibly diverse and mature cloud hosting platform, and the AWS CLI is your gateway to interacting with AWS from the comfort of your laptop. Instead of going click-click-click on the AWS website, you type commands from the command line, and you can even use those commands in scripts for semi-automated control of AWS resources. In this guide we'll go over how to install the AWS CLI tool on your laptop.
In this guide we'll discuss the AWS CLI version 2, which is the latest version at the time of writing. To proceed with this guide, you must already have an AWS account. It is recommended that you create an IAM account for each project, for security.
Setting up the AWS CLI tool is fairly easy, but there are several considerations to ensuring it is correctly set up.
In AWS there are two kinds of accounts: The root account, and an IAM account. The root account is what's created when one signs up for an AWS account. The IAM account can be thought of as a sub-account, and is created after you create the root account. To create a root account, go to the AWS website and sign up. Creating an IAM account is handled in the IAM dashboard, but doing so is fairly involved since you must create several things.
For more information see the
The best practice is to create one or more IAM accounts for each project. This lets you assign specific permissions to each IAM account based on the needs of the project. But the hows and wherefores of doing that is beyond the scope of this particular guide, which is focused on setting up AWS CLI.
Where this intersects with the AWS CLI is the concept of the AWS Profile. Each AWS Profile is associated with either a root account or an IAM account. To make that association, one downloads an Access Keys file containing tokens for authenticating AWS API calls. The AWS CLI tool of course uses the AWS API under the covers, and therefore uses the access tokens for authentication.
When logged in either with a root account or IAM account, navigate to the My Security Credentials dashboard. The easiest way is via the Services dropdown at the top of the screen, because there's a direct link to that dashboard in that dropdown. Once in the dashboard click on the Access Keys tab.
There will be a button marked Create New Access Key. When you click on this, you'll be shown two security tokens, the Access Key ID and the Secret Access Key. These are the access credentials for the account you're currently logged in as. You'll also be able to download a CSV file containing the access keys, which looks like this:
$ cat ~/Downloads/accessKeys.csv
Access key ID,Secret access key
AKIAEKX7AHMBPWLKEU7H,41WEtXEAaMPPfLLE1C5CEQXLA2MSP2LfEVGJKU2A
This file contains the security tokens for your account, and should be carefully stashed somewhere. Don't worry that I'm leaking some credentials, because these are example values (look carefully and see the letters E-X-A-M-P-L-E salted into the tokens), and in any case I would have revoked the credentials before publishing them openly. Anybody who gets ahold of your access credentials has access to your AWS account, and can act on your AWS infrastructure as if they are you. The good news is that if your AWS credentials do leak to the public, they can be revoked at any time.
This gives you the access tokens required to use AWS CLI on your laptop. Before we get into using this tool, we need to install it.
Installing the AWS CLI on Windows, macOS, Linux and Docker
Generally speaking we download and run an application installer from the AWS website. The process is of course a little different for each operating system.
The AWS CLI is itself an open source project at:
https://github.com/aws/aws-cli
While we can install it using the source code, it's cleaner to get the installer. The official instructions are
Once you have the AWS CLI installed, it is useful to
Installing the AWS CLI on Windows using the official installer package
At the time of this writing, you download the AWS CLI installer for Windows from:
If you need to update the AWS CLI, simply download and run the latest installer from that link.
This is an MSI Installer, so once the download finishes navigate to the Downloads folder and double-click on the installer to run it. Go through the installer wizard, as I'm sure you will have done hundreds of times by now. By default, the AWS CLI installs to C:\Program Files\Amazon\AWSCLIV2. That directory will be already setup so the aws command can execute at the command line.
Once installed you should be able to open a command window, or PowerShell window, and run this command:
C:\> aws --version
aws-cli/2.0.23 Python/3.7.4 Windows/10 botocore/2.0.0
Later in this guide we'll go over usage more deeply.
Installing the AWS CLI on macOS using the official installer package
For macOS the AWS team has made this task far more complex than necessary. You can install the AWS CLI either as a global command, or as a local command, and there is allowance for situations where your user ID does not have Administrator permissions. Further, there are additional commands to run for updating the AWS CLI, including an alternative if you do not have sudo permission.
The most likely situation is that you are the sole user of the macOS computer you're using. In that most likely situation your account will have Administrator access, and can install global commands, and has the ability to run sudo to run commands as root. That's the model we'll discuss in this guide.
Yes there are situations of shared macOS systems, and there are cases where corporate Information Technology policies do not allow Administrator access to assigned computers. In such a case you'll need to follow the additional instructions in
Another consideration is that Homebrew has an AWS CLI bundle. That means it's possible to install AWS CLI using brew install awscli, but the official guide warns against this. Namely: Because AWS doesn't maintain third-party repositories, we can’t guarantee that they contain the latest version of the AWS CLI. You Have Been Warned.
To cut to the chase - get the installer here:
This is a PKG file, and I'm sure you will have already installed dozens of such packages. That means, you double-click the PKG file and follow the directions.
By default the tool is installed to /usr/local/aws-cli and a symlink is created as /usr/local/bin/aws. Since /usr/local/bin is probably already in your PATH variable, that means the aws command will automatically be available at your command-line.
It's also possible to install it elsewhere. For example if you are not in an Administrator account, it can be installed to a location in your home directory. If so you'll need to set up your PATH variable to include the directory where it is installed, so that the tool can run from your command-line.
Another option is running the installer at the command-line using command-line options to control where AWS CLI is installed. If you need this, consult the official documentation.
To verify:
$ which aws
/usr/local/bin/aws
$ aws --version
aws-cli/2.0.23 Python/3.7.4 Darwin/18.7.0 botocore/2.0.0
Again, we'll discuss verification more completely later in this guide.
Installing the AWS CLI on Linux using the official installer package
This installation involves downloading and unpacking a ZIP archive, then running an installation script in the archive. This means having the unzip (or equivalent) command installed. Additionally, glibc, groff, and less must be installed. All this is the default for most Linux systems.
While the AWS CLI is available through the package management systems (e.g. apt-get install awscli), the AWS team warns us saying: Because AWS doesn't maintain third-party repositories, we can’t guarantee that they contain the latest version of the AWS CLI. You Have Been Warned.
There are installers available for both Linux x86 64 bit, and Linux ARM. Did that just say you can run the AWS CLI on a Raspberry Pi? Yes that would be accurate, but also consider that AWS offers ARM based EC2 instances, and therefore needs to support the AWS CLI on such instances.
On Linux x86 64 bit:
$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
$ unzip awscliv2.zip
$ sudo ./aws/install
And for Linux ARM:
$ curl "https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip" -o "awscliv2.zip"
$ unzip awscliv2.zip
$ sudo ./aws/install
The first command downloads the ZIP archive. What's shown is using Curl, but of course there are alternatives, for example:
$ wget https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip
Whatever is your preferred download method, retrieve the ZIP file from that URL.
The next step is to unpack the ZIP archive, such as using the unzip command. One wonders why they want Linux users to use a ZIP archive, when the .tar.gz format is more native to Linux than the ZIP format.
The final step is running the installation script. As shown here you need to have sudo access. As I said for the macOS installation, you will usually have sudo access, but of course there are plenty of cases where that's not the case. The
./aws/install script with some command-line options to install to a non-privileged directory.
Once AWS CLI is installed, verify:
$ aws --version
aws-cli/2.0.23 Python/3.7.4 Linux/4.14.133-113.105.amzn2.x86_64 botocore/2.0.0
Later we'll go over using AWS CLI and verify it further.
Using the AWS CLI Docker image instead of a normal installation
Maybe your computer has Docker installed, and you want to skip the complexity of installing AWS CLI on your laptop. The AWS team provides a Docker image which lets you simply run the tool without installing it. While Docker images normally contain server images, they're frequently used for distributing command-line tools.
First, you need a sufficiently advanced version of Docker:
$ docker --version
Docker version 19.03.1
Next, running AWS CLI this way is as simple as:
$ docker run --rm -it amazon/aws-cli command
The first time you do this, it will look like this:
$ docker run --rm -it amazon/aws-cli --version
Unable to find image 'amazon/aws-cli:latest' locally
latest: Pulling from amazon/aws-cli
37373184fe69: Pull complete
12ad2aecf25e: Pull complete
5988150955c7: Pull complete
50942d897f99: Pull complete
3cf2d07cbbca: Pull complete
Digest: sha256:67e7fc5dec6d782cf1a37cbf6da4a7971a9d441646ac04b4a9ffc0af21a60c11
Status: Downloaded newer image for amazon/aws-cli:latest
aws-cli/2.0.38 Python/3.7.3 Linux/4.19.76-linuxkit docker/x86_64.amzn.2
This says to automatically remove the generated container, and to run it interactively at the terminal. We are to list the preferred aws command options after the container name. In this case we put the --version option, as we'd done with previous examples.
In this section we've gone over installing the AWS CLI on Windows, macOS, Linux, and as a Docker container. That's only the first step of successfully setting up the AWS CLI.
Configuring AWS CLI profiles
At the beginning of this guide you set up an AWS account, and possibly an IAM sub-account. You also downloaded access keys for those accounts. These credentials are available in the My Security Credentials dashboard of the AWS Management Console. You access that dashboard via the Services dropdown in the Management Console. But if you followed the directions earlier, you've already done this, and the CSV file will be stashed on your disk somewhere.
In this section we'll be configuring the AWS CLI with these credentials, and set up one or more AWS profiles.
An AWS profile is a named entity containing specific access credentials. The profiles are simply listed in a pair of configuration files. On Unix-like systems (macOS, Linux, etc) these files are here:
$ ls ~/.aws
config credentials
The profile name is simply the name used in these files, and it has no meaning beyond the boundaries of your laptop.
To create a profile, run this command:
$ aws configure --profile root-user
AWS Access Key ID [****************E3GA]: ... ENTER ACCESS KEY
AWS Secret Access Key [****************J9cp]: ... ENTER SECRET KEY
Default region name [us-west-2]:
Default output format [json]:
The profile name being created is on the command-line after the --profile option, in this case we are creating a profile named root-user. In addition to the access keys, we enter an AWS region name, and the desired output format for information printed by the AWS CLI tool.
This means each AWS profile is associated with the account (either root or IAM account) associated with the credentials used for the profile.
The --profile option is used in every AWS CLI command to choose the profile to use. You can avoid specifying that option by setting the environment variable AWS_PROFILE to the desired profile name. There is also a default profile you can create, which is the profile used if no profile is specified via the --profile option or AWS_PROFILE variable.
There is a debate over the best practice .. is it best to have a default profile, or is it best to not have one? By not having a default profile, you must always be explicit about the profile to use, which is theoretically a good thing. My recommendation is to skip the default profile, and to set AWS_PROFILE accordingly for each project.
If you are using Docker to provide the AWS CLI, there is a consideration regarding the credentials file we just discussed. Recall the command format:
$ docker run --rm -it amazon/aws-cli command
The --rm option says to delete the container after running the command. This means the configuration file inside the container will vaporize. What's actually required is to use this command:
$ docker run --rm -it -v ~/.aws:/root/.aws amazon/aws-cli command
This mounts your .aws directory into the correct place inside the container. The configuration files in that directory will persist, and not vaporize when the container vaporizes.
Because of the complexity of that command it's probably best to create a shell alias or a shell script to hide most of that command.
Checking that AWS CLI is correctly configured
Now that you have the AWS CLI installed, and have configured one or more AWS profiles, let's take it for a spin around the block and make sure everything is okay.
First off is getting help:
$ aws help
This tells you a high-level overview of the AWS command itself. One thing this tells you is the structure of AWS commands, namely:
aws [options] <command> <subcommand> [parameters]
The aws command uses the command parameter to select which AWS subsystem to use, and one or more subcommand parameters for the specific API call. The options and parameters provide additional data.
You can get further help like so:
$ aws ec2 help
$ aws ec2 describe-instances help
In other words, appending help to the end of the command shows you the help text. In this case we're showing ec2 as the command and describe-instances as the subcommand.
That verified basic functioning of the AWS command, but did not verify that you correctly entered the access tokens, and that the AWS command can access AWS infrastructure. The simplest verification is this:
$ aws s3 ls
Unable to locate credentials. You can configure credentials by running
"aws configure".
$ aws s3 ls --profile root-user
$ export AWS_PROFILE=root-user
$ aws s3 ls
The aws s3 command accesses the S3 subsystem, which is the AWS Simple Storage Service. FWIW, That's a cloud based file storage system. It's not important at this moment whether you have any files stored in S3, but whether the command executes without error.
The first command demonstrates what happens if you have no default profile, and do not specify a profile. The AWS command tells you to set up a profile.
The next shows using the --profile option, and the third shows using the AWS_PROFILE environment variable.
Another command useful for verifying the connection is:
$ aws iam list-users --profile root-user
{
"Users": [ {
"Path": "/",
"UserName": "root-user",
"UserId": "AIDARNEXAMPLEYM35LE",
"Arn": "arn:aws:iam::....",
"CreateDate": "2020-03-08T02:19:39+00:00",
"PasswordLastUsed": "2020-04-05T15:34:28+00:00"
} ]
}
This lists the users associated with your AWS account. Therefore this makes an API call to AWS, and if it successfully prints something like this your AWS credentials are working correctly.
Summary
In this guide we've learned how to install and configure the AWS CLI tool. With this tool you can more effectively work with AWS infrastructure than by using click-click-click on the AWS Management website. You will instead be able to write scripts that can be committed to source code repositories.
There is a huge world of AWS subsystems to explore now that you've taken this step. Good luck.
