Using environments and profiles
In a workspace an user can define profiles. A profile defines a specific configuration of a host that is described using a set of environments. Environments
are used to configure the build system, for instance to change the compilers in use.
Basically all available profiles can be found in the file <pid-workspace>/environments/profiles_list.cmake
, that looks something like:
There is always a profile named default
that matches the default settings of your host patform, and that is automatically detected. The default
profile always has the host
environment as base one. Base environment (the first in the list ENVIRONMENTS
) defines default settings of the host, typically default compiler toolchains. The host
environment is specific, always used as base environment, and describes the host platform detected settings. The default profile can also have additional environments, in this example pkg-config
and f2c
. Additional environments (the following in the list) are typically used to define plugins and alternative toolchains.
A workspace can have any number of profiles and any profile can use the host
environment (in the example my_test
) or a user defined environment (e.g. use_clang
profile has clang_toolchain
as base environment).
For now an important thing to remember is that default
is a reserved keyword for profiles and host
a reserved keyword for environments.
Finally there is always one and only one profile considered as current profile: this is the one currenlty used for building packages.
Now let’s learn how to use profiles and environments.
Step 1: Using the default profile
Let’s start from a clean situation:
If you defined no profile, the current profile is the host default
one and this is the only one defined:
This profile will be evaluated any time you configure the workspace:
According to this profile, PID automatically defines a target platform and print its specifications in output, something like:
The name and base environment for profile currently in use is given: default
profile with default settings coming from host
detected settings.
The target platform description is sum up in its name: x86_64_linux_stdc++11
. There are also other informations, like possible optimizations for processor or OS distribution.
There is nothing more to do to use this default profile.
Step 2: Creating and using a new profile
2.1 create a profile based on host detected setting
We now learn how to create a new profile using the default host
environment. This is achieved by doing:
This first create the environment and make it automatically the default environment. The output is now:
Except for the name of the profile, nothing changed. This is perfectly normal since we use the host default settings. So finally using this profile you will build for same target platform. The file <pid-workspace>/environments/profiles_list.cmake
, now looks something like:
A line has been added to the list of available profiles to define my-profile
and this later is specified as current one.
Explanation:
Once the command profiles
is used it first load description from profiles_list.cmake
then perform operations specified using the cmd
argument (in this example: creating a new profile and set it as current). Finally it launches the workspace configuration that automatically uses the current profile to define the build system and target platform.
So now anytime you configure the workspace, for instance :
The output will be the same as previously: the current profile does not change until you modify the file profiles_list.cmake
or use the profiles
command.
Important Note: The profile will not be completely reevaluated if you do not change current profile or explicitly ask for it. So if you changed the default host settings (for instance changing the default c++ toolchain), they will not be applied until you reevaluate a profile that uses the host
environment.
2.2 create a profile based on a specific environment
Now we will create and use yet another profile, but this time we want to explicitly use another environment than default one.
We first need to know which environments are available:
Output should look something like:
We see that the environment clang_toolchain
exists, lets’ get some more info on it, also using the info
command:
Output gives:
We have minimal information about it, we now who to contact to get more information about this environment and we now that we can target a specific version of clang
using the version
constraint.
Let’s suppose that we want to use the clang toolchain with version 3.8 or more for building C/C++ code. We want so to define the corresponding profile that we call use-clang
:
If everything works as expected (if the clang_toolchain
environment has generated a configuration of the workspace that allows to build code using the clang tool), the output looks something like:
We can see that the current profile has changed, it is now use-clang
. The ASM/C/C++ toolchain has changed, this is printed with those lines:
But the target platform is still the same, this is normal since we just changed the c++ compiler (or version of this compiler), not the target platform itself (e.g. we do not crosscompile for instance).
And profiles_list.cmake
looks like:
2.3 Understanding problems
If the profile creation failed for any reason, you should see error messages in the output. The most straitforward reason is probably that the clang_toolchain
cannot be evaluated with the constraints you gave. It is probably simpler now to directly check what is the problem with this environment. Before any other action we need to see if clang_toolchain
can be deployed locally:
If any problem occurs during deployment you will see an error message in the output. The cause of the problem has been found : you probably do not have rights to clone the repository. Contact the authors of the environment to get help.
Otherwise, the environment can be found in <pid-workspace>/environments/clang_toolchain
and the problem comes from the evaluation of the environment itself. You need to test it:
Then if there is an error in output, you know that the cause is that the environment cannot find a solution to generate a valid configuration for using clang
tool. For instance because: it cannot find clang or install it in your host system, or it has no solution according to the constraints you gave (here that the version
must be geater or equal than 3.8).
Anyway resolving such problems requires to understand how environments are written but this is reserved to advanced users. Otherwise you should ask the authors how you could deal with the problem.
2.4 Understanding resulting target platform
When using the use-clang
profile you may have noticed that the target platform generated contains a lot of information while you only changed the default compiler. More precisely those lines must be explained:
Indeed you defined absolutely nothing about either of Python
, CUDA
or Fortran
languages, the only languages you affect by using the clang_toolchain
are C
, C++
and ASM
languages.
This is normal as PID tries to automatically detect toolchains for all supported languages if they are not predefined at environment level. This is of course only true when no crosscompilation takes place.
Step 3: Management of available profiles
Now we have created profiles we can change the current one in use, remove some of them, etc. All these operations are achieved using the profiles
command of the workspace.
3.1: listing available profiles
To know profiles that are available:
Gives the output:
3.2: removing a profile
Let’s suppose we do not need my-profile
:
Gives same output as previously:
and to verify that it has been deleted:
Gives the output:
and the file profiles_list.cmake
reflects this change:
3.2: reset to default profile
Now if we want to use again the default profile:
Gives the output:
3.4: using use-clang profile
And finally to use of the already existing use-clang
profile:
Gives the output:
Well you now know how to manage profiles from a global point of view.
Step 4: Refining profiles description with target platform constraints
4.1 : Apply constraints to target platform
Environments like gcc_toolchain
or clang_toolchain
may use paremeters to define the exact build tools and versions to use. They may for instance adapt the build tools used for a given target platform, so there is a set of default arguments to the configure command used to (partly) specify the target platform:
platform
: this is the full specification of a target platform. For instance:
proc_type
: specifies a constraint on the processor type (e.g.x86
,arm
).proc_arch
: specifies a constraint on the processor data bus size (e.g.32
,64
).os
: specifies a constraint on the operating system (e.g.linux
,macosx
).abi
: specify a constraint on C++ abi in use (stdc++
orstdc++11
can be alternative versions when usinggcc
, in addition you can specifyc++
if you useclang
toolchain).distribution
: specify a constraint on OS distribution (e.g. ubuntu).distrib_version
: specify a constraint on version of OS distribution (e.g. ubuntu). Used together withdistribution
.
Depending on the constraints passed as arguments the gcc_toolchain
environment will try to find a solution.
For instance when working on a x86_64
host platform changing the abi is performed this way:
It should produce a result like :
The current platform has changed (now it is x86_64_linux_stdc++
) and everytime a package is built it is installed in the corresponding isolated install tree for this platform. Be warned that such a change can have undesired side effect due for instance to OS libraries being not built with dual ABI support.
The corresponding line in profiles_list.cmake
is:
On a 64 bits platform, changing the processor binary architecture to 32 bits is performed this way:
It should produce a result like :
And the corresponding line in profiles_list.cmake
is:
One interesting information here is that we cannot build CUDA
code. This is because the cuda toolkit installed in system does not support multi arch.
Remarks:
-
For now changing those environment does not lead to a cross compilation since the linux operating system nows how to manage multi arch and the standard library
stdc++
is built with dual ABI support. -
At the time this tutorial has been written, only
gcc_toolchain
implements the managemement of these contraints. Butclang_toolchain
for instance could be improved to do it as well.
4.2: cross compiling
Environments can be used to define a cross-compilation process by simply targeting another platform: it takes place if you change processor type or operating system. For instance if we want to target a bare metal 32 bits arm system:
Here notice that the full platform specification arm_32_stdc++
does not define an OS, which is normal in this situation because bare metal systems have no OS installed.
Of course it requires that the environment defines the build tools required to target the given platform. For instance gcc_toolchain
environment defines how to install adequate version of gcc on ubuntu. Anytime we need to manage new host distributions the gcc_toolchain
has to be updated accordingly to manage the situation.
When crosscompiling to OS equipped systems we may define additional information. To do this there are specific arguments to the configure
command:
sysroot
: allow user to define a custom sysroot folder for its target platform. Environment should always define a default sysroot but user may want to define a new one.staging
: allow user to define a custom staging folder for its target platform. It is only usefull if you usedsysroot
argument to a path that is read only.
So let’s imagine we want to build for a raspberry pi, we can do:
It configures the workspace with the gcc toolchain able to build for a raspberry pi target platform. gcc_toolchain
provides a default sysroot but user may want to change it by doing:
4.3: reusing complete target platform specifications
Even if possible to define the target platform directly into profiles definition, it is most of time more convenient to define an environment that fully specifies the target platform constraints as well as the available tools. This way people can reuse the whole description without boring about constraint to define at profile level. This is particularly true when targetting a specific computer, for instance the one that controls a machine, that is most of time defined once and for all (or at least for a long time).
To define such an environment follow this tutorial.
Then to define a profile you simply use this environment the simple way:
When you want to cross-compile for a well-known target, this is the same. For instance for targetting a raspberry-pi card
Step 5: profiles description with additional environments
There is still one last aspect to understand: a profile can have environments in addition to its base one. These environments can be used for customizing the workspace configuration.
Of course it is always possible, as explained in previous section, to create an environment that already depends on all other environments you need but it is quite difficult to anticipate the use of all possible environments and all possible combinations.
This is particularly true for environment defining plugin for PID and particularly those used to manage third party tools that are not directly involved in the build process. This is for instance the case of the pkg-config
environment, that is used to generate configurations files in order to use libraries from the outside of the workspace.
Let’s add the pkg-config
environment to default default environment:
If default
profile is not the current one you should see no output, and this is normal: the profile has not been reevaluated.
Simply reset to the default profile:
Now it is evaluated, the output should look something like:
The additional environments in use are explicitly listed and their usage information is printed. Now everytime you build a native package the plugin will generate and install configuration files for pkg-config
tool.
In the file profiles_list.cmake
you can find this line:
Then you can add more environments, there is no limit:
This time the workspace should automatically reconfigure and print something like:
Now you can use the f2c
generator in your packages. Notice that adding this environment to the current profile is not required if a package requires it (using check_PID_Environment
). Indeed there is an automatic environment deployment and evaluation process that can take place from packages description. But this environment is not automatically added to current profile so each time the package is reconfigured the f2c
environment will be reevaluated which can become a bit annoying. So for a purpose of time saving it is preferable to directly put it in the profiles(s) you use if you need it.
Again there is not real limit to what you can add, for instance you can choose to add an alternative C/C++ compiler:
The workspace automatically reconfigures and prints something like:
As you can see, the default C/C++ compiler did not change, it is still gcc 9.3. But now a package can require the use of clang
compiler using check_PID_Environment
call. This is not obviously a good solution to require the use of a specific compiler, but sometimes if your code uses some compiler specific features there is no other solution. Same remark applies than for the f2c
environment: your package can install it if required but it is always better to have it added to your profile.
That’s it for now, you know everything about environments usage and profiles management.