About other languages

PID has been designed to manage many compiled languages that can be used together with C/C++: Fortran, CUDA and ASSEMBLER (this later being by default provided with your C compiler). Using these languages is automatically supported and does not require specific actions if adequate toolchains are installed in your workstation.

PID workspace provides information about available languages:

pid cd
pid configure

Gives you the output:

...
[PID] INFO: using default profile, based on host native development environment.
[PID] INFO : ASM language available with gcc toolchain.
[PID] INFO : C language available with gcc toolchain (version 9.3.0).
[PID] INFO : C++ language available with gcc toolchain (version 9.3.0).
[PID] INFO : Python language available (version 2.7). To use python modules installed in workspace please set the PYTHONPATH to =/home/robin/soft/PID/pid-workspace/install/x86_64_linux_stdc++11/__python2.7__
[PID] INFO : CUDA language available (version 11.0). Building for architecture 6.1.
[PID] INFO : Fortran language available with GNU toolchain (version 5.5.0).

[PID] INFO : Target platform in use is x86_64_linux_stdc++11:
 + processor family = x86 (optimizations: SSE, SSE2, SSE3, SSSE3, SSE4_1, POPCNT, SSE4_2, FP16, FMA3, AVX, AVX2, AVX_512F, AVX512_SKX)
 + binary architecture= 64
 + operating system=linux (ubuntu 16.04, apt packaging)
 + C++ ABI= stdc++11

Here we know that CUDA and Fortran code can be built. This means that your source folders can contain files with .f (or .f77) and .cu extensions that will be automatically compiled as respectively Fortran and CUDA code. There is no much more to learn, except you must know how to interface those specific code with C code (same as in any other build system), but it is not in the scope of PID tutorial.

About Python

PID also provides a convenient way to deal with Python language. Even if this language can be compiled, in PID Python code is considered as interpretable script only.

For a given local workspace there is only one version of python that can be used at a time. This can be configured using dedicated python_interpreter environment. The default Python version in use in a workspace is defined by the default python interpreter in your workstation (if any, otherwise python is not available).

In previous output of pid-workspace configuration we know that:

  • python language is available, so we can define python code in packages.
  • the python version in use is 2.7.
  • for using python packages defined within the workspace you need to add to your PYTHONPATH environment variable the given path (here: <pid-workspace>/install/python2.7)

First thing to understand when using python with PID is that all python packages generated by PID packages are installed in a unique folder of the workspace that depends on the python version. In the current example it is <pid-workspace>/install/python2.7 since we use version 2.7 of python.

PID packages allows to generate python packages from two types of components:

  • Python components directly defined by the user into a PID package.
  • Module components defining python wrappers. Python wrappers are specific python packages that are used to wrap code written in other languages, and widely used to interface Python with C/C++ code.

Following sections explain how to do define such components.

Defining Python Components

Let’s reuse my-first-package defined in previous tutorials. To define a normal python package (a package that contains pure python scripts) called python_pack simply create the folder share/script/python_pack in your package source. In this folder, create a file __init__.py (this file can be left empty it is just used by python to identify the folder as a python package). Then simply put you python scripts in this folder as usual when writting python code.

Then edit the file share/CMakeLists.txt:

PID_Component(python_pack PYTHON DIRECTORY python_pack)

Finally build your package:

pid cd my-first-package
pid build

No compilation of the python code takes place, the build command only installs it in <pid-workspace>/install/python2.7.

about dependencies between python components

As python components dependencies are managed directly into the python language at runtime there is no need to define explicit dependencies between python components using PID API.

But now let’s suppose another package, called another-package, is defining a new python package another_python_pack:

PID_Component(another_python_pack PYTHON DIRECTORY another_python_pack)

And let’s suppose another_python_pack is using functionalities defined in python_pack. There is an implicit dependency between both python components. It is recommanded to define a package dependency between another-package and my-first-package: this way anytime another-package is deployed in a local workspace then my-first-package will be deployed too.

The final benefit is that whenever using another_python_pack then python will be able to find also python_pack since both will have been installed by their respective package build process.

Defining Python wrappers

Python wrappers are python packages that wraps C/C++ code.

To define a wrapper you need to first define a module library in PID.

Edit the file src/CMakeLists.txt of my-first-package to add a new library:

PID_Component(python_wrapper MODULE DIRECTORY python_wrapper_module)

Then in src folder create a folder named python_wrapper_module. Inside you can put your C/C++ code as usual.

A MODULE library is a library without an interface (i.e. without any public header file). A module library can have dependencies just like any other component in PID (they just never export them since they have no interface). It is used to define pure dynamically loadable libraries (for instance typically used to define plugins for an application). By default a module is simply a shared object (.so in linux).

To transform a module library into a python wrapper:

  • add a __init__.py file into its source folder. This file defines the symbols exported to python by the wrapper. There are specific rules to export symbols defined by the C/C++ code into python.
  • then write your C++ file so that it can be used in python. There are various way to write python wrappers, you can directly use the python library or use dedicated third party libraries. For instance boost-python from boost project or pybind11 project are convenient ways to write python wrappers in C++ and corresponding wrappers boost and pybind11 are already available for both in PID.

Finally build the package:

pid cd my-first-package
pid build

This builds and installs the C/C++ python wrapper. If you look into <pid-workspace>/install/python2.7 you will se that there is a new python package installed named python_wrapper (also a debug version is installed). Check its content:

ls <pid-workspace>/install/python2.7/python_wrapper

You see the __init__.py file and some symbolic links to all binaries (shared objects) implementing the C/C++ wrapper. These links are:

  • the link to the shared object define by python_wrapper.
  • the links to all direct or undirect runtime dependencies of python_wrapper.

This way all runtime depenencies will be resolve anytime python is using your wrapper, without the need to configure environment variables such as LD_LIBRARY_PATH.

about dependencies between python components and python wrappers

Dependencies between python wrappers follow the same rule as for python components. And this is also the same rule if you want to define dependencies between python components and Python wrappers. To be short:

  • there is no need to define an explicit dependency between any kind of python components in PID.
  • Always remember to declare package dependencies between packages that define these components, to automate the deployment process and get everything needed and installed when you run python code.

Conclusion

That’s it you know how to define any kind of python packages. Probably if you are not familiar with python programming you should follow this tutorial. Also if you want some indications about the best libraries for writting python wrappers we suggest for instance this site.