How to install and use s3cmd in Windows 10

There are a ton of reasons why you should be using object storage right now, faster data retrieval, reduction in cost, optimization of resources, are just a few of those reasons. Due to the scale-out nature of object storage, it's less costly to store all of your data with this technology. Amazon S3 is without a doubt the most know service of this kind out there, however there are other options outside besides from Amazon. The object storage service of providers such as Digital Ocean, Linode and Vultr are just a few of the most trustworthy solutions for object storage.

Manipulating buckets in Amazon is quite easy using the AWS CLI, however, when we talk about the manipulation of buckets in the object storage of other providers, AWS CLI is not an option, instead, all of them rely on the s3cmd command line tool. As it seems to be unclear for a lot of people how to use this CLI tool in Windows, I will explain to you how to easily install and use s3cmd in Windows 10.

1. Download s3cmd

S3cmd is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage or DreamHost DreamObjects. It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc.

You can download the latest available version of S3cmd from the official repository at Github here. Go to the releases page here, I downloaded the s3cmd-2.2.0.zip file. Then, extract its content in the following directory:

C:\s3cmd

Of course you can change this directory and place it wherever you want, but I'll use this directory in this article. The binary can be used like this:

python C:\s3cmd\s3cmd

But it will work only if Python is already installed, if it's not, be sure to follow the next step.

2. Install Python 3

As mentioned in the first step, the latest version of s3cmd 2.2.0 requires Python 3 to work properly. You need to install Python 3 in your system, in case you don't have Python 3 installed already, be sure to download the installer from the official Python website here. For Windows, there's already an user friendly installer that registers automatically the python binary to the PATH, so after the installation, you will be able to use python from the command prompt:

python --version

In my case, I installed Python 3.9.8.

Important: you need to install Python 3, otherwise if you run the package using Python 2, you may find the following exception in your terminal:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
ImportError trying to import dateutil.parser. 
Please install the python dateutil module:    
$ sudo apt-get install python-dateutil        
  or
$ sudo yum install python-dateutil
  or
$ pip install python-dateutil
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

3. Proceed with the configuration

In my case, I'm configuring a bucket in the object storage service of Vultr, so the information that I'm going to fill in this step are the required by this service, however it should be similar for the service you decide to use. The binary of s3cmd is the s3cmd file located in the directory where you extracted the downloaded content, just run the script with Python and use the --configure argument to start configuring the Object Storage credentials:

python C:\s3cmd\s3cmd --configure

Once you finish the questionnaire, you will get an output similar to the following one (in the last step, the connection will be tested):

ERROR: Option --preserve is not yet supported on MS Windows platform. Assuming --no-preserve.
ERROR: Option --progress is not yet supported on MS Windows platform. Assuming --no-progress.

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: XXXXXXXXXXXX
Secret Key: XXXXXXXXXXXXX
Default Region [US]: 

Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]: ewr1.vultrobjects.com

Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: %(bucket)s.ewr1.vultrobjects.com

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password:
Path to GPG program [C:\Program Files (x86)\Gpg4win\..\GnuPG\bin\gpg.EXE]: 

When using secure HTTPS protocol all communication with Amazon S3       
servers is protected from 3rd party eavesdropping. This method is       
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]: Yes

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:

New settings:
  Access Key: XXXXXXXXXXXX
  Secret Key: XXXXXXXXXXXX
  Default Region: US
  S3 Endpoint: ewr1.vultrobjects.com
  DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.ewr1.vultrobjects.com
  Encryption password:
  Path to GPG program: C:\Program Files (x86)\Gpg4win\..\GnuPG\bin\gpg.EXE
  Use HTTPS protocol: True
  HTTP Proxy server name:
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Not configured. Never mind.

Save settings? [y/N] y
Configuration saved to 'C:\Users\sdkca\AppData\Roaming\s3cmd.ini'