LinkedIn Sourceforge Twitter

Vincent's Blog

Pleasure in the job puts perfection in the work (Aristote)

Upgrade python packages after an OpenBSD upgrade

Posted on 2020-11-08 16:09:00 from Vincent in OpenBSD

You have just upgraded your OpenBSD machine to the brand new release, but some of your applications are no more working because they do not have the required python package.

This is a very common problem between python package you have installed via pkg_add and the one you have installed via "pip install". The problem is that both are installed on your machine in /usr/local/lib/python3.8/site-packages.

I propose you a methodology to identify which one were installed with which tool


Introduction

In general I install my specific packages for python via pkg_add. This way, they are following the standard process of OpenBSD after each upgrade. In general this is the most convenient way to upgrade python software running on an OpenBSD machine.

I'm a heavy python user, and in some cases, I need specific python packages. Either it does not exist in the OpenBSD ports tree, either it's not the correct version.

Most efficient way to use PIP

pip is the Package Installer for Python. I prefer to use it in my own local folders. In such way it does not interfere with Python packages provide by OpenBSD.

~/.local/lib/python3.8/site-packages

To do that, I just run pip with my own user. Not as root, not via doas.

In most cases, this works great. If this is not enough, I rarely use venv. Venv is ideal when you need to build shared libraries. This way you can have different version running on your DEV machine.

But for server's applications or daemon, we must store our python codes on standard folders:

/usr/local/lib/python3.8/site-packages

And in such cases troubles start.

Typical problems

When the only solution is to store the package in the /usr/local folders, I have problems like:

  • the version required is higher than the proposed version by OpenBSD's ports
  • the package required does not exist in the OpenBSD's ports tree

Thus, when the OpenBSD upgrade is coming (every 6 months), it becomes difficult to remember which package is managed by which package's manager.

Solution

To avoid such waste of time after every upgrade, I've developed a small script which allow me to classify each of those packages. Such script only works on OpenBSD.

The script generates an output like this:

obsd:~ $ ./my_own_py_pkgs.sh  
Python version: 3.8
checking /home/vi/.local/lib/python3.8/site-packages:
  Specific element: /home/vi/.local/lib/python3.8/site-packages/fapws4
checking /usr/local/lib/python3.8/site-packages:
  Specific element: /usr/local/lib/python3.8/site-packages/Beaker-1.11.0-py3.8.egg-info
  Specific element: /usr/local/lib/python3.8/site-packages/PyAudio-0.2.11-py3.8.egg-info
  Specific element: /usr/local/lib/python3.8/site-packages/SpeechRecognition-3.8.1.dist-info
  Specific element: /usr/local/lib/python3.8/site-packages/distro-1.5.0.dist-info
  Specific element: /usr/local/lib/python3.8/site-packages/jedi-0.17.2.dist-info
  Specific element: /usr/local/lib/python3.8/site-packages/msgpack-1.0.0-py3.8.egg-info
  Specific element: /usr/local/lib/python3.8/site-packages/parso-0.7.1.dist-info
  Specific element: /usr/local/lib/python3.8/site-packages/speech_recognition

So, I know that those Python packages will not have been managed by pkg_add.
For sure, python package located in my home folder are specific. But, via this script I thus know all python packages I must upgrade my self.

Script

The script I've developed is quite small. I short, it checks if folders presented are known by one of the OpenBSD's package via the pkg_info tool.

You can download the script here.

There is no specific requirements. It works only on OpenBSD.



2, 1
displayed: 479



What is the last letter of the word Python?