CPU specific optimized Python on AWS

Recently there has been a need to more efficiently host our software services on AWS. As of publication this is most readily achievable for general purpose Python code with (AWS Graviton instances)[https://aws.amazon.com/ec2/graviton/]

Starting from a Debian 11 ARM EC2 t4g instance, the following commands are able to create a Python build from source optimized for the CPU architecture running on AWS. Note, LTO is enabled in the build script and it uses a huge amount of RAM, so make sure you have enough RAM or use a swapfile as I demonstrate below since I chose a t4g.small instance.

sudo apt update -y
sudo apt upgrade -y
sudo apt install -y \
    git \
    build-essential \
    gdb \
    lcov \
    libbz2-dev \
    libffi-dev \
    libgdbm-dev \
    liblzma-dev \
    libncurses5-dev \
    libreadline6-dev \
    libsqlite3-dev \
    libssl-dev \
    lzma \
    lzma-dev \
    tk-dev \
    uuid-dev \
    libxml2-dev \
    libxml2 \
    libxslt1-dev \
    libxslt1.1 \
    xvfb \
    zlib1g-dev
apt build-dep python3 -y
curl https://pyenv.run | bash

# If you have less than 8 GB RAM, create a swapfile and enable it
# Don't put the swapfile on EBS.
# It should be on your instance root volume for performance.
sudo fallocate -l 8G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

################################################
# Append the following to the end of ~/.bashrc #
################################################

export PYENV_ROOT="$HOME/.pyenv"
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

# Restart your shell for the changes to take effect.

# Load pyenv-virtualenv automatically by adding
# the following to ~/.bashrc:

eval "$(pyenv virtualenv-init -)"

################################################
#        Restart your shell to continue        #
################################################

CFLAGS="-march=native -mtune=native" CONFIGURE_OPTS="--enable-optimizations --with-lto=full" pyenv install 3.11.0 --verbose
pyenv virtualenv 3.11.0 MY_VIRTUAL_ENVIRONMENT_NAME