Build Python from Source

A good exercise that I feel has great benefits is building Python from the source repo. We will be building Python from CPython. CPython can be found on GitHub at https://github.com/python/cpython. I suggest forking the repo into your GitHub account. This can be done by simply visiting the CPython Github page and pressing ‘Fork’ on the top right. Choose to fork it to your username and it will be created at "https://github.com/<username>/cpython" (<username>is your Github username).

Your environment will need tools installed to compile the source code. There are countless resources are available online so I will not rehash it here. I suggest researching and looking into the tools you will need as you progress through the steps. Hitting breakpoints may allow for learning something new.

I am compiling to a macOS environment. Use a package manager like Homebrew for macOS to get the tools and utilities you need.

CPython source local copy

1. Clone the CPython repo from your GitHub repositories:

   $ git clone git@github.com:<username>/cpython.git
   

2. Configure an upstream remote:

   $ cd cpython
   $ git remote add upstream git@github.com:python/cpython.git
   You can verify that your setup is correct using:
   $ git remote -v
   

After these steps you should have a folder in your environment called cpython. This directory will have source files from the cpython github repo.

Compile > Build > Python

1. macOS install xcode developer tools:

   $ xcode-select --install
   

2. Homebrew install the following, you may need them:

   $ brew install openssl xz gdbm cmake wget pkg-config
   

3. Now configure Python. For this we will configure Python versions >= 3.7:

   $ ./configure --with-pydebug --with-openssl=$(brew --prefix openssl)
   

4. Then make:

   $ make -s -j2
   

Test Python

You should now have a file python.exe in the cpython folder. Let’s test it:

1. In the cpython folder type:

   $ ./python.exe
   

2. Try importing modules. Check the ssl module:

   $ import ssl
   

   If there are no errors it means everything is installed properly. Otherwise, some dependencies were missed during the installation.

3. Exit the REPL. Type exit() or quit() command.

4. In the cpython folder type:

   $ ./python.exe -m test_os
   Raised RLIMIT_NOFILE: 256 -> 1024
   0:00:00 load avg: 2.60 Run tests sequentially
   0:00:00 load avg: 2.60 [1/1] test_os
   == Tests result: SUCCESS ==
   1 test OK.
   Total duration: 3.7 sec
   Tests result: SUCCESS
   

5. Made it this far with no errors, great! If something went wrong retrace your steps and try to debug the issue.

Closing

Congratulations! You have a working, complied build of Python in your environment. Play around with the git repo and see if you can build different versions of Python.

Enjoy! Happy Coding.

REFERENCES

1. Getting Started - Python Developer's Guide
2. Homebrew
3. Build CPython on macOS