How to build/install awscli2 on FreeBSD?

[YevheniiPokhvalii]

There are ports for awscli1 for FreeBSD: https://www.freshports.org/devel/awscli
But I cannot find the way to install or build awscli2 on FreeBSD.
Linuxbrew does not have support for FreeBSD either, so it is not possible to get awscli2 from there as well.

Are there any workarounds?

[tim-finnigan]

We don't recommend using third-party packaging/distributions services for installing the AWS CLI. Please refer to this documentation for the official AWS CLI installation instructions: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

[YevheniiPokhvalii]

Well, I could not manage to get it to work on FreeBSD. There is neither FreeBSD support in the official documentation nor in awscliv2 installation scripts.

My question is regarding awscli v2 not awscli v1 because it is possible to install awscli v1 on FreeBSD and it is also available via pip.

[jeheyer]

I'm stuck here too. My plan was just PIP install it, but I get this:

pkg install py39-pip
pip install awscliv2
awscliv2 --install
09:27:26 - awscliv2 - ERROR - FreeBSD amd64 is not supported, use docker version

It's really a bummer. AWS CLI tools should just be a collection of Python scripts, so I'm not sure why they'd stick OS/Platform restrictions on it.

[jeheyer]

Well I see the problem now. Although the bulk of AWS CLI Tools is Python, the entry point is indeed a binary. Fortunately, with some hacking, it is possible to run this via Linux emulation in FreeBSD. I put together a blog post on that:

https://layer77.net/2023/04/07/installing-aws-cli-tools-v2-on-freebsd-12-4/

[dylanneild]

Here is how I got got this working on FreeBSD 14.0-p3. I do not have Linux emulation installed or enabled. This is a pure, native FreeBSD 14 build and install of aws-cli v2.

The following process assumes:

1. You have sudo installed and have privileges to use it.
2. You want aws to be installed globally in the system (/usr/local/bin)

Of course if you just want a local / user install you're free to remove the sudo from step 5 and 6. If you do this, you will also need to alias python3.9 to python as the aws script assumes the Python interpreter is simply python.

1. Make sure you have the pre-requisites installed on FreeBSD:

% sudo pkg install git curl python39 py39-pip py39-sqlite3 cmake gmake rust gcc groff

4. Clone the source tree and checkout the v2 branch:

%  git clone https://github.com/aws/aws-cli.git
%  cd aws-cli
%  git checkout v2

5. Globally install all pre-requisites:

% sudo pip install -r requirements.txt

6. Build and install aws, making sure to specify the gcc compiler for any C/C++ binary compilation.

%  sudo CC=/usr/local/bin/gcc pip install -e .

The build will take some time and will execute various backend dependency builds. This include C and Rust compilation. You'll know it's working if the c compiler process being used during the build is cc1.

Specifying the gcc compiler here is critical, as there is presently a bug in clang 16.0.6 (the default compiler on FreeBSD 14) which breaks compilation of at least one dependency. Note that his bug has been fixed by the FreeBSD team, but hasn't been deployed to package updates as I write this.

By the time you read this guide you may be able to remove gcc from the package dependence above and simply run sudo pip install -e .

For more details, see:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271531

If all goes well, you'll get:

Successfully built awscli
Installing collected packages: wcwidth, urllib3, ruamel.yaml.clib, python-dateutil, pycparser, prompt-toolkit, jmespath, docutils, distro, awscrt, ruamel.yaml, cffi, cryptography, awscli
Successfully installed awscli-2.15.19 awscrt-0.19.19 cffi-1.16.0 cryptography-40.0.1 distro-1.8.0 docutils-0.19 jmespath-1.0.1 prompt-toolkit-3.0.38 pycparser-2.21 python-dateutil-2.8.2 ruamel.yaml-0.17.21 ruamel.yaml.clib-0.2.7 urllib3-1.26.18 wcwidth-0.2.13

You can now access aws as expected:

% aws --version
aws-cli/2.15.19 Python/3.9.18 FreeBSD/14.0-RELEASE-p3 source/amd64 prompt/off

Note that by doing the global package install in step 5, you may be walking on existing py39-xxxxx packages installed on the system, such as py39-six and py39-urllib3. Someone could probably figure out if some of these can be installed via pkg install rather than the global pip install process.

For my setup, I just wanted a working backup server running awscli and didn't need any other python apps.

Hope this helps someone out.

[samos667]

Thanks my savior ! That work perfectly !

# aws --version
aws-cli/2.15.36 Python/3.9.18 FreeBSD/13.1-RELEASE-p9 source/amd64 prompt/off

[tuaris]

Strange, even though the bug is 'Closed FIXED', the error still appears. My python version is 3.11 and clang is 18.1

FreeBSD clang version 18.1.6 (https://github.com/llvm/llvm-project.git llvmorg-18.1.6-0-g1118c2e05e67)
Target: x86_64-unknown-freebsd14.2
Thread model: posix
InstalledDir: /usr/bin

Works fine when you use GCC.

[tuaris]

This is broken on FreeBSD 14.3-RELEASE:

            -- The C compiler identification is GNU 13.3.0
            -- Detecting C compiler ABI info
            -- Detecting C compiler ABI info - done
            -- Check for working C compiler: /usr/local/bin/gcc - skipped
            -- Detecting C compile features
            -- Detecting C compile features - done
            -- Detected CMAKE_SYSTEM_PROCESSOR as amd64
            -- Detected 64-Bit system
            -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
            -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
            -- Check if compiler accepts -pthread
            -- Check if compiler accepts -pthread - yes
            -- Found Threads: TRUE
            -- FOUND AWS-LC CRYPTO cmake config - static
            -- Using libcrypto from the cmake path
            -- CMAKE_AR found: /usr/local/bin/ar
            -- CMAKE_RANLIB found: /usr/local/bin/ranlib
            -- CMAKE_OBJCOPY found: /usr/local/bin/objcopy
            CMake Error at CMakeLists.txt:326 (file):
              file failed to open for reading (No such file or directory):

                /tmp/pip-install-y07hxht6/awscrt_8ee0b6a5c022493fbb3a878270ca93ad/crt/s2n/tests/features/GLOBAL.flags
            Call Stack (most recent call first):
              CMakeLists.txt:367 (feature_probe)


            -- feature S2N_STACKTRACE: FALSE
            -- Configuring incomplete, errors occurred!
            CMake Error at aws-c-common/cmake/AwsPrebuildDependency.cmake:65 (message):
              Configuration failed for dependency project S2N
            Call Stack (most recent call first):
              CMakeLists.txt:77 (aws_prebuild_dependency)


            -- Configuring incomplete, errors occurred!
            > /usr/local/bin/cmake -H/tmp/pip-install-y07hxht6/awscrt_8ee0b6a5c022493fbb3a878270ca93ad/crt -Bbuild/temp.freebsd-14.3-RELEASE-p2-amd64-cpython-311/deps -DCMAKE_INSTALL_PREFIX=build/temp.freebsd-14.3-RELEASE-p2-amd64-cpython-311/deps/install -DCMAKE_BUILD_TYPE=RelWithDebInfo -DAWS_USE_LIBCRYPTO_TO_SUPPORT_ED25519_EVERYWHERE=ON
            Traceback (most recent call last):
              File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
                main()
              File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
                json_out['return_val'] = hook(**hook_input['kwargs'])
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
                return _build_backend().build_wheel(wheel_directory, config_settings,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 435, in build_wheel
                return _build(['bdist_wheel', '--dist-info-dir', str(metadata_directory)])
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 423, in _build
                return self._build_with_temp_dir(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir
                self.run_setup()
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 317, in run_setup
                exec(code, locals())
              File "<string>", line 524, in <module>
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 115, in setup
                return distutils.core.setup(**attrs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 186, in setup
                return run_commands(dist)
                       ^^^^^^^^^^^^^^^^^^
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
                dist.run_commands()
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
                self.run_command(cmd)
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command
                super().run_command(command)
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
                cmd_obj.run()
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/command/bdist_wheel.py", line 370, in run
                self.run_command("build")
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
                self.distribution.run_command(command)
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command
                super().run_command(command)
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
                cmd_obj.run()
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 135, in run
                self.run_command(cmd_name)
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
                self.distribution.run_command(command)
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command
                super().run_command(command)
              File "/tmp/pip-build-env-tpnozjvq/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
                cmd_obj.run()
              File "<string>", line 382, in run
              File "<string>", line 361, in _build_dependencies
              File "<string>", line 306, in _build_dependencies_impl
              File "<string>", line 68, in run_cmd
              File "/usr/local/lib/python3.11/subprocess.py", line 413, in check_call
                raise CalledProcessError(retcode, cmd)
            subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '-H/tmp/pip-install-y07hxht6/awscrt_8ee0b6a5c022493fbb3a878270ca93ad/crt', '-Bbuild/temp.freebsd-14.3-RELEASE-p2-amd64-cpython-311/deps', '-DCMAKE_INSTALL_PREFIX=build/temp.freebsd-14.3-RELEASE-p2-amd64-cpython-311/deps/install', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DAWS_USE_LIBCRYPTO_TO_SUPPORT_ED25519_EVERYWHERE=ON']' returned non-zero exit status 1.
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for awscrt
      Failed to build awscrt
      ERROR: Could not build wheels for awscrt, which is required to install pyproject.toml-based projects

[jeheyer]

I haven't tried the build process by @dylanneild, but FWIW I just upgraded to 14.3 and the Linux emulator solution still works fine & dandy:

# aws --version
2.3.1
AWS CLI v2 command: /root/.awscliv2/binaries/aws
aws-cli/2.30.3 Python/3.13.7 Linux/5.15.0 exe/x86_64.centos.7

Commands for great success:

pkg install py311-pip
pip install awscliv2
ln -s /usr/local/bin/awscliv2 /usr/local/bin/aws
echo "linux_enable=\"YES\"" >> /etc/rc.conf
pkg install linux_base-c7
cd /usr/local/lib/python3.11/site-packages/awscliv2/

Open installers.py with your favorite editor and modify the install_multiplatform() function so that there is match:

def install_multiplatform() -> None:

    if os_platform == "FreeBSD" and arch == "amd64":
        return install_linux_x86_64()

Run awscliv2 --install and you should be good to go.

For anyone curious about the docker solution, seems the docker package isn't setup to run as a daemon:

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.

# service docker start
docker does not exist in /etc/rc.d or the local startup
directories (/usr/local/etc/rc.d), or is not executable

I can't find an easy solution. Really seems the FreeBSD package should just do the emulation option.

[tuaris]

oh, I didn't know there was a working docker package in the ports tree. Interesting...

[jeheyer]

Eh...it's really only partially working. Seems the standard work-around is use podman instead.