Htydjtk

I have a fresh install of Debian 9.8, the current stable release version.

$ lsb_release --description

Description:    Debian GNU/Linux 9.8 (stretch)

This system includes Python 3.5.3:

$ python3 --version

Python 3.5.3

Unfortunately that version doesn't meet my needs. I want a more current version of Python. At the time of writing, the current version is 3.7.2.

Hopefully this will be helpful for a range of versions, but in the interest of limiting the scope to something we can be very precise about, this question is about the above combination of OS and python versions.

To further specify: I want to build from the python git source repository, using the default configuration: we'll run ./configure with no arguments except maybe --prefix to choose a custom install location. I want any needed dependencies to come from the standard Debian apt sources, if possible.

Finally and most important: For any additional software we install as a prerequisite, give some information about it. Don't just say “install this incomprehensible list of apt packages”.

For any dependency available from Debian's stretch repositories, I want the following info:

• Whether the dependency can be considered optional or must be treated as required in order to successfully complete python's build/install steps.




• A link to the project homepage (or next next best thing if there isn't a suitable homepage, e.g. a repo url) that apt package came from.




• What feature/module in the resulting installation does the dependency map to? Sometimes this will be obvious because the names will be similar or identical between project name, package name, and the module it is used to build, but sometimes it won't.

With that kind of information, anyone reading this can have some understanding of what they are installing and why. The task of determining which dependencies their project will need will hopefully be made significantly easier.

Install git

$ sudo apt install git

Install Python build dependencies

Python includes a set of modules that it builds by linking to other popular open source projects. Many of those projects are included in Debian by default as executable programs, but the development files (headers, libraries) necessary to link them into Python are packaged separately and not included in a default Debian install.

Some of these libraries are actually required for the python build/install steps to complete successfully:

| Library                            | Python Module | Dev Package  |

| ---------------------------------- | ------------- | ------------ |

| https://www.zlib.net/              | `zlib`        | `zlib1g-dev` |

| https://www.sourceware.org/libffi/ | `_ctypes`     | `libffi-dev` |

The entries in the 'Dev Package' column are the names of packages containing the development files our python build needs.

Each of these 'dev' packages has a corresponding binary package that is probably already installed. So you're likely not going to be adding new pieces of software to your system by installing these. You're just installing the files necessary to compile new pieces of software (in our case a newer python) so that it can use them. (Also note that these 'dev' packages all have their corresponding binary packages as
dependencies, so installing a dev package will ensure that its binary package is also installed.)

So let's install them:

sudo apt install zlib1g-dev libffi-dev

Next we have the OpenSSL library. Python considers OpenSSL optional, but you probably want it. For example, even using Python's package installation
tools can run into trouble when fetching https urls if SSL/TLS support is missing.

| Library                  | Python Module | Dev Package |

| ------------------------ | ------------- | ----------- |

| https://www.openssl.org/ | `_ssl`        | `libssl-dev |

Let's install it:

sudo apt install libssl-dev

The next set of packages fall more in the 'optional' category:

| Library                                              | Python Module(s)              | Dev Package        |

| ---------------------------------------------------- | ----------------------------- | ------------------ |

| http://www.bzip.org/                                 | `_bz2`                        | `libbz2-dev`       |

| https://www.gnu.org/software/ncurses/                | `_curses` and `_curses_panel` | `libncursesw5-dev` |

| https://www.gnu.org.ua/software/gdbm/                | `_dbm` and `_gdbm`            | `libgdbm-dev`      |

| https://tukaani.org/xz/                              | `_lzma`                       | `liblzma-dev`      |

| https://www.sqlite.org/                              | `_sqlite3`                    | `libsqlite3-dev`   |

| https://www.tcl.tk/software/tcltk/                   | `_tkinter`                    | `tk-dev`           |

| https://github.com/karelzak/util-linux               | `_uuid`                       | `uuid-dev`         |

| https://tiswww.case.edu/php/chet/readline/rltop.html | `readline`                    | `libreadline-dev`  |

Python can build and install without these, and your applications might not need them. On the other hand, if you install them, the 'make' step shouldn't report any modules it was unable to build, and the binaries are already included in a default Debian install.

The command to copy-paste if you'd like to go ahead and install all of the above 'optional' packages is:

sudo apt install libbz2-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev tk-dev uuid-dev libreadline-dev

Build and install python from source

1. clone the python git repository.

$ git clone git@github.com:python/cpython.git

Cloning into 'cpython'...

[...]

Resolving deltas: 100% (592215/592215), done.

2. Checkout the version you want to install (in this case, 3.7.2) using the git tag name.

$ cd cpython/

cpython$ git checkout v3.7.2

Note: checking out 'v3.7.2'.

[...]

HEAD is now at 9a3ffc0492... 3.7.2final

cpython$

3. Choose an install prefix. That is, the path the compiled and linked project will be installed into. In this case I'll use $HOME/python/v3.7.2. I'm including the version number since I'll want to install other versions in the future, and keep them separate. Run the configure script with this path as the --prefix argument. (If you don't provide a --prefix argument, it will default to /usr/local).

cpython$ ./configure --prefix=$HOME/python/v3.7.2

[...]

checking for inflateCopy in -lz... yes

[...]

checking for openssl/ssl.h in /usr... yes

[...]

The configure script will check a bunch of things, many of which are unnecessary. The ones shown in the above output, though, can be considered necessary.

4. Build python by running make. If we included the minimum set of dependencies necessary to build and install python, the output will tell you near the end what modules were not built:

cpython$ make

[...]

Python build finished successfully!

The necessary bits to build these optional modules were not found:

_bz2                  _curses               _curses_panel      

_dbm                  _gdbm                 _lzma              

_sqlite3              _tkinter              _uuid              

readline                                                       

To find the necessary bits, look in setup.py in detect_modules() for the module's name.

The make output describes this as a list of 'optional' modules but again, at least for this particular version of python, it probably won't build and install successfully without the zlib, and _ctypes modules.

5. Install python by running make install. We've already provided the install location back in the configure step. If your install location is a root-owned directory (e.g. /usr/local), prefix this command with sudo. Since in this case I'm installing to a user-owned directory, I don't want to do that.

cpython$ make install

Creating directory /home/python/v3.7.2/bin

Creating directory /home/python/v3.7.2/lib

[...]

6. You should now have an executable you can run from the bin directory underneath the install prefix named python3. (Note that this installer doesn't put anything at bin/python; just bin/python3.)

$ $HOME/python/v3.7.2/bin/python3

Python 3.7.2 (v3.7.2:9a3ffc0492, Mar 10 2019, 19:35:56) 

[GCC 6.3.0 20170516] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> quit()

$