Tutorials for environments
It is sometimes required or preferred to use specific build tools. The reason may be for instance to enforce specific compiler usage, in particular needed to crosscompile to a specific target platform (e.g. targeting a raspberry pi card). In PID customizing the build environment used at workspace level is made with environments
. An environment
is a deployment unit defining how to configure the development tools of the host environment.
Generally we use environments to achieve different tasks:
-
managing a toolchain. For instance
gcc_toolchain
environment defines how to configure the build process when using the GNU C/C++ compiler ;nvcc_toolchain
environment defines how to configure the build process when using the NVIDIA CUDA compiler, etc. -
defining plugins in order to use development tools not managed by default. For instance
pkg-config
environment defines how to generate configuration files for thepkg-config
tool, which can be usefull to provide a way to use packages deployed into a workspace from a third party project. -
describing a specific platform instance, this is the description of the complete build environment for a given target platform. They are used to agregate a set of toolchains and plugins with specific constraints (for instance minimum version required).
Following sections explain how to define new environments.
Warning
Writing environment may be a complex task, and is very hard to generalize as it is completey bound to the specificities of the host, of the compiler in use, of the target platform, etc.
That is why writing new environments should be reserved to advanced users.
Step 1: create the environment repository and project
First of all we need to create the project for the environment and its online repository.
1.1 Create online repository
Go into your git hosting server (gitlab for LIRMM) and create a new repository with same name as environment projects.
In this example we create a repository named gcc_toolset
, that will finally be more or less a copy of gcc_toolchain
. We name it this way to avoid conflicts with the already existing gcc_toolchain
environment.
Then copy the SSH address of this git repository.
1.2 Create the environment project
cd <pid-workspace>
pid create environment=gcc_toolset url=<url previously copied>
cd <pid-workspace>/environments/gcc_toolset
The environment project should have been created and put into folder <pid-workspace>/environments
. In the following sections we use <gcc_toolset>
as a short name for the path to <pid-workspace>/environments/gcc_toolset
.
Step 2: describe the content of the environment
Now first thing to do is to define adequate meta information of the environment. Edit the CMakeListst.txt
file in <gcc_toolset>
and paste the code:
cmake_minimum_required(VERSION 3.15.7)
set(WORKSPACE_DIR ${CMAKE_SOURCE_DIR}/../.. CACHE PATH "root of the PID workspace directory")
list(APPEND CMAKE_MODULE_PATH ${WORKSPACE_DIR}/share/cmake/system) # using generic scripts/modules of the workspace
include(Environment_Definition NO_POLICY_SCOPE)
project(gcc_toolchain ASM C CXX)
PID_Environment(
AUTHOR Robin Passama
INSTITUTION CNRS/LIRMM
MAIL robin.passama@lirmm.fr
YEAR 2019
LICENSE CeCILL-C
ADDRESS git@gite.lirmm.fr:pid/environments/gcc_toolchain.git
PUBLIC_ADDRESS https://gite.lirmm.fr/pid/environments/gcc_toolchain.git
CONTRIBUTION_SPACE pid
DESCRIPTION "using GNU gcc toolchain to build C/C++ code of projects"
)
PID_Environment_Constraints(IN_BINARY version # give the version of the desired gcc toolset in use
OPTIONAL exact
CHECK check_gcc.cmake) # check script for current configuration
#for now only define a solution for ubuntu distributions
PID_Environment_Solution(HOST CONFIGURE configure_generic_gcc.cmake)
PID_Environment_Solution(OS linux DISTRIBUTION ubuntu CONFIGURE ubuntu/configure_gcc.cmake)
build_PID_Environment()
Explanations:
include(Environment_Definition NO_POLICY_SCOPE)
is used to import the API for writing environments.-
PID_Environment
command (equivalent todeclare_PID_Environment
) transforms the current CMake project into a PID environment. Arguments to provide are more or less the same as for native packages:AUTHOR
,INSTITUTION
,MAIL
refer to the contact author of the environment project.ADDRESS
andPUBLIC_ADDRESS
have the same meaning than for native packages, they are used to configure official git remote.YEAR
,LICENSE
,CONTRIBUTION_SPACE
andDESCRIPTION
have the same meaning than for native packages.
-
PID_Environment_Constraints
defines environment specific constraints that can be applied to evaluate current environment. TheCHECK
keyword defines the CMake script file used to check if host system configuration matches constraints. In the example, the scriptcheck_gcc.cmake
that lies in projectsrc
folder. Basically it checks ifgcc
toolchain is the current toolchain. If constraints are defined it then also checks thatgcc
toolchain matches these constraints. In this exampleversion
andexact
constraints are defined, as usual for toolchain definition environments.exact
is optional (defined using theOPTIONAL
keyword) so it can be used or not.version
is also optional but mandatory in banaries (defined using theIN_BINARY
keyword). This means that any package that explicitly requires this environment will have its version information included in its description.
-
PID_Environment_Solution
defines a solution used to deploy the toolchain when host does not macth target requirements. In this example we have only two solutions: one applying when host matches target ; the second applying when the host is a linux ubuntu system. Solutions are evaluated sequentially so if first one succeeded, the second will not be evaluated.CONFIGURE
argument defines the script file used to generate the adequate configuration.- there are filters to to those solutions dependening on host system features:
OS
andDISTRIBUTION
are used to specify for which type of host system the solution apply. if using theHOST
kerword, it means that any host can be used.
build_PID_Environment
configures the environment and create its build commands.
Step 3: Writing the check script
The first thing to do is to edit the script src/check_gcc.cmake
. This script simply checks if host is already configured with the toolchain defined by gcc_toolset
, eventually taking into account constraints defined in root CMakeLists.txt
. Edit this file and paste the code:
# check if host already matches the constraints
#host must have a GNU compiler !!
if(NOT CMAKE_CXX_COMPILER_ID STREQUAL "GNU"
OR NOT CMAKE_C_COMPILER_ID STREQUAL "GNU"
OR NOT CMAKE_ASM_COMPILER_ID STREQUAL "GNU")
return_Environment_Check(FALSE)
endif()
#check C compiler version
if(NOT CMAKE_C_COMPILER_VERSION)
check_Program_Version(IS_GCC_OK gcc_toolchain_version "${gcc_toolchain_exact}" "gcc -v" "^gcc[ \t]+version[ \t]+([^ \t]+)[ \t]*.*$")
else()
check_Environment_Version(IS_GCC_OK gcc_toolchain_version "${gcc_toolchain_exact}" "${CMAKE_C_COMPILER_VERSION}")
endif()
if(NOT IS_GCC_OK)
return_Environment_Check(FALSE)
endif()
#check CXX compiler version
if(NOT CMAKE_CXX_COMPILER_VERSION)
check_Program_Version(IS_GPP_OK gcc_toolchain_version "${gcc_toolchain_exact}" "g++ -v" "^gcc[ \t]+version[ \t]+([^ \t]+)[ \t]*.*$")
else()
check_Environment_Version(IS_GPP_OK gcc_toolchain_version "${gcc_toolchain_exact}" "${CMAKE_CXX_COMPILER_VERSION}")
endif()
if(NOT IS_GPP_OK)
return_Environment_Check(FALSE)
endif()
return_Environment_Check(TRUE)
The check script basically tests if the default host ASM/C/C++ compiler are GNU compilers. If not then return_Environment_Check(FALSE)
exits this script on error.
The script should also takes into account defined constraints, in this example the version related constraints. The variable gcc_toolchain_version
is correctly valued according to value given to the version
constraint or is not defined if no version
constraint has been specified. So it is simple to handle constraints in CMake script by simly checking value of variables with pattern <environment>_<constraint>
. Furthermore, the environment API provides a set of functions that can be used to ease the writing of the script. For instance, check_Program_Version
and check_Environment_Version
are provided to test the common arguments version
and exact
, respectively on any program capable of outputing its version number or using a variable provided by CMake.
Finally if all tests succeeded the script should exit with return_Environment_Check(TRUE)
.
Step 4: Writing the configure scripts
Last part consists in writing configuration scripts for each solution.
4.1: generic configuration script
For the first solution, the script is src/configure_generic_gcc.cmake
and looks like:
# check if host matches the target platform
host_Match_Target_Platform(MATCHING)
if(NOT MATCHING)
return_Environment_Configured(FALSE)
endif()
evaluate_Host_Platform(EVAL_RESULT)
if(EVAL_RESULT)
# thats it for checks => host matches all requirements of this solution
# simply set the adequate variables
configure_Environment_Tool(LANGUAGE C CURRENT)
configure_Environment_Tool(LANGUAGE CXX CURRENT)
configure_Environment_Tool(LANGUAGE ASM CURRENT)
set_Environment_Constraints(VARIABLES version
VALUES ${CMAKE_C_COMPILER_VERSION})
return_Environment_Configured(TRUE)
endif()
return_Environment_Configured(FALSE)
The role of the configuration script is to define a configuration of the host platform in order to allow the use of a given set of tools, in this example the compiler and other associated tools that constitute the gcc_toolchain
. The configuration itself is performed using the configure_Environment_Tool
that is used to memorize configuration choices. Since we want to define a set of tools related to a given set of languages (C
, C+
+ and ASM
) we need to provide a valid configuration for these languages using the LANGUAGE
argument.
Since this script is supposed to be executed on any host platform, first step consists in testing if host matches target platform (using function host_Match_Target_Platform
) otherwise solution evaluation fails. Indeed, there is few chances to write a generic host configruation script that can automatically manage cross compilation features.
Then evaluate_Host_Platform
is simply used to automatically call the check script defined at previous step. If evaluation succeeded the script memorize corresponding settings for C, C++ and ASM languages using the configure_Environment_Tool
function. The CURRENT
argument is here used to tell to memorize current CMake configuration of the environment project itself (i.e. everything related to the compiler toolchain in use for corresponding languages). It also memorizes the value of constraints that will be published in binary description of packages (REQUIRED
and IN_BINARY
constraints like version
) using set_Environment_Constraints
.
Finally, the script returns evaluation success or failure using return_Environment_Configured
function.
4.2: configuration script for an ubuntu host
The second solution is used to deploy specific version of the toolchain on an ubuntu host system, and is described in src/ubuntu/configure_gcc.cmake
file:
# if this script executes code is built on a ubuntu system
# build the pool of available versions depending on the target specification
get_Environment_Target_Platform(DISTRIBUTION target_distrib DISTRIB_VERSION target_distrib_version
TYPE target_proc ARCH target_bits OS target_os ABI target_abi)
get_Environment_Host_Platform(DISTRIB_VERSION host_distrib_version
TYPE proc_host ARCH bits_host ABI host_abi)
if(target_os STREQUAL linux)
if(target_distrib STREQUAL ubuntu)#target is ubuntu and host is ubuntu as well (by definition if this script is called)
if(host_distrib_version STREQUAL target_distrib_version)
if(proc_host STREQUAL target_proc AND bits_host EQUAL target_bits)#same processor architecture => the problem may simply come from the version of the compiler in use
#I know the procedure to install gcc whatever the processor specification are (and I know binaries will be compatibles)
install_System_Packages(APT gcc g++)#getting last version of gcc/g++
evaluate_Host_Platform(EVAL_RESULT)#evaluate again the host (only check that version constraint is satisfied)
if(EVAL_RESULT)
#only ABI may be not compliant
get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})
if(ABI_FLAGS)#an ABI constraint is explicilty specified => need to force it!
configure_Environment_Tool(LANGUAGE CXX CURRENT FLAGS ${ABI_FLAGS})
endif()
configure_Environment_Tool(LANGUAGE C CURRENT)
configure_Environment_Tool(LANGUAGE ASM CURRENT)
set_Environment_Constraints(VARIABLES version
VALUES ${CMAKE_C_COMPILER_VERSION})
return_Environment_Configured(TRUE)#that is OK gcc and g++ have just been updated and are now OK
endif()#no solution found with OS installers -> using alternative
elseif(proc_host STREQUAL target_proc AND bits_host EQUAL 64 AND target_bits EQUAL 32)
# host is a 64 bits system and target is a 32 bits => on ubuntu I can generate 32 compatible code
# I know the procedure to install gcc whatever the processor specification are since same processor type
install_System_Packages(APT gcc g++ gcc-multilib g++-multilib)#getting last version of gcc/g++ AND multi arch support for gcc
evaluate_Host_Platform(EVAL_RESULT)#evaluate again the host (only check that version constraint is satisfied)
if(EVAL_RESULT)
get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})
configure_Environment_Tool(LANGUAGE ASM FLAGS -m32)
configure_Environment_Tool(LANGUAGE C FLAGS -m32)
configure_Environment_Tool(LANGUAGE CXX FLAGS -m32 ${ABI_FLAGS})
configure_Environment_Tool(SYSTEM LIBRARY_DIRS /lib32 /usr/lib32)
set_Environment_Constraints(VARIABLES version
VALUES ${CMAKE_C_COMPILER_VERSION})
return_Environment_Configured(TRUE)#that is OK gcc and g++ have just been updated and are now OK
endif()#no solution found with OS installers -> using alternative
endif()
endif()
elseif(target_distrib STREQUAL raspbian
AND target_proc STREQUAL arm
AND target_bits EQUAL 32)#targetting a raspbian v3+ system
if( proc_host STREQUAL x86
AND bits_host EQUAL 64)#we have built a cross compiler for this situation !!
set(POSSIBLE_VERSIONS 5.4.0 8.3.0)
set(version_selected)
if(gcc_toolset_version) #there is a constraint on version
if(gcc_toolset_exact)
list(FIND POSSIBLE_VERSIONS ${gcc_toolset_version} INDEX)
if(INDEX EQUAL -1)
return_Environment_Configured(FALSE)#required gcc version is not provided by the environment
endif()
else()
foreach(version IN LISTS POSSIBLE_VERSIONS)
if(gcc_toolset_version VERSION_LESS version)
#an adequate version (>= required version) has been found
if(version VERSION_LESS version_selected)
set(version_selected ${version})#take the version that is the closest to the required one
endif()
endif()
endforeach()
if(NOT version_selected)#no compatible version found
return_Environment_Configured(FALSE)#compatible gcc version is not provided by the environment
endif()
endif()
else()#otherwise any version possible => take the biggest one
foreach(version IN LISTS POSSIBLE_VERSIONS)
if(version VERSION_GREATER version_selected)
set(version_selected ${version})
endif()
endforeach()
endif()
if(version_selected VERSION_EQUAL 5.4.0)
set(PATH_TO_ROOT ${CMAKE_SOURCE_DIR}/src/ubuntu/raspi-x86_64/armv8-rpi3-linux-gnueabihf-5.4.0)
elseif(version_selected VERSION_EQUAL 8.3.0)
set(PATH_TO_ROOT ${CMAKE_SOURCE_DIR}/src/ubuntu/raspi-x86_64/armv8-rpi3-linux-gnueabihf-8.3.0)
else()#PROBLEM => error
message("[PID] INTERNAL ERROR: no version selected for raspberry pi v3 cross compiler => aborting")
return_Environment_Configured(FALSE)#compatible gcc version is not provided by the environment
endif()
#C compiler
set(PATH_TO_GCC ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-gcc)
set(PATH_TO_GCC_AR ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-ar)
set(PATH_TO_GCC_RANLIB ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-ranlib)
set(PATH_TO_GPP ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-g++)
configure_Environment_Tool(LANGUAGE C COMPILER ${PATH_TO_GCC} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
#c++ compiler (always explicitly setting the C++ ABI)
get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})
configure_Environment_Tool(LANGUAGE CXX COMPILER ${PATH_TO_GPP} FLAGS ${ABI_FLAGS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})#Note same ar and ranlib than for gcc
#assembler
set(PATH_TO_AS ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-as)
configure_Environment_Tool(LANGUAGE ASM COMPILER ${PATH_TO_AS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})#Note same ar and ranlib than for gcc
# system tools
set(PATH_TO_LINKER ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-ld)
set(PATH_TO_NM ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-nm)
set(PATH_TO_OBJCOPY ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-objcopy)
set(PATH_TO_OBJDUMP ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-objdump)
configure_Environment_Tool(SYSTEM LINKER ${PATH_TO_LINKER} NM ${PATH_TO_NM} OBJCOPY ${PATH_TO_OBJCOPY} OBJDUMP ${PATH_TO_OBJDUMP})
#sysroot !!
set(PATH_TO_SYSROOT ${PATH_TO_ROOT}/armv8-rpi3-linux-gnueabihf/sysroot)
configure_Environment_Tool(SYSTEM SYSROOT ${PATH_TO_SYSROOT})
# TODO See if necessary
# SET(CMAKE_FIND_ROOT_PATH $ENV{HOME}/x-tools/armv8-rpi3-linux-gnueabihf)
set_Environment_Constraints(VARIABLES version
VALUES ${version_selected})
return_Environment_Configured(TRUE)
endif()
#TODO directly put the compiler for proc x86 64 bits
#else add other OS/distrib when supported
endif()
elseif(target_os STREQUAL Generic)#no target OS => we compile for bare metal system
if(target_proc STREQUAL "arm") # compile for ARM microcontrollers
if(target_bits EQUAL 32)#armv7 for pic microcontrollers
install_System_Packages(APT gcc-arm-none-eabi)#getting last version of gcc/g++ for that target
find_program(PATH_TO_GCC arm-none-eabi-gcc)
if(PATH_TO_GCC-NOTFOUND)
return_Environment_Configured(FALSE)#not found after install => problem
endif()
find_program(PATH_TO_GPP arm-none-eabi-g++)
if(PATH_TO_GPP-NOTFOUND)
return_Environment_Configured(FALSE)#not found after install => problem
endif()
find_program(PATH_TO_AS arm-none-eabi-as)
if(PATH_TO_AS-NOTFOUND)
return_Environment_Configured(FALSE)#not found after install => problem
endif()
find_program(PATH_TO_GCC_AR arm-none-eabi-gcc-ar)
if(PATH_TO_GCC_AR-NOTFOUND)
return_Environment_Configured(FALSE)#not found after install => problem
endif()
find_program(PATH_TO_GCC_RANLIB arm-none-eabi-gcc-ranlib)
if(PATH_TO_GCC_RANLIB-NOTFOUND)
return_Environment_Configured(FALSE)#not found after install => problem
endif()
find_program(PATH_TO_LINKER arm-none-eabi-ld)
if(PATH_TO_LINKER-NOTFOUND)
return_Environment_Configured(FALSE)#not found after install => problem
endif()
find_program(PATH_TO_NM arm-none-eabi-nm)
if(PATH_TO_NM-NOTFOUND)
return_Environment_Configured(FALSE)#not found after install => problem
endif()
find_program(PATH_TO_OBJCOPY arm-none-eabi-objcopy)
if(PATH_TO_OBJCOPY-NOTFOUND)
return_Environment_Configured(FALSE)#not found after install => problem
endif()
find_program(PATH_TO_OBJDUMP arm-none-eabi-objdump)
if(PATH_TO_OBJDUMP-NOTFOUND)
return_Environment_Configured(FALSE)#not found after install => problem
endif()
#now set the compiler
get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})#computing compiler flags to get the adequate target c++ ABI
configure_Environment_Tool(LANGUAGE C COMPILER ${PATH_TO_GCC} TOOLCHAIN_ID "GNU" AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
configure_Environment_Tool(LANGUAGE CXX COMPILER ${PATH_TO_GPP} TOOLCHAIN_ID "GNU" FLAGS ${ABI_FLAGS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
configure_Environment_Tool(LANGUAGE ASM COMPILER ${PATH_TO_AS} TOOLCHAIN_ID "GNU" AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
configure_Environment_Tool(SYSTEM LINKER ${PATH_TO_LINKER} NM ${PATH_TO_NM} OBJCOPY ${PATH_TO_OBJCOPY} OBJDUMP ${PATH_TO_OBJDUMP})
get_Configured_Environment_Tool(LANGUAGE C COMPILER c_compiler)
check_Program_Version(RES_VERSION gcc_toolchain_version "${gcc_toolchain_exact}" "${c_compiler} -v" "^gcc[ \t]+version[ \t]+([^ \t]+)[ \t]+.*$")
if(RES_VERSION)
set_Environment_Constraints(VARIABLES version
VALUES ${RES_VERSION})
return_Environment_Configured(TRUE)
endif()
endif()
endif()
endif()
#TODO add more specific compiler that depends on hardware
return_Environment_Configured(FALSE)
Explanations:
As you may see the script is a bit complex (and has even been truncated from its original version for sake of readbility). The goal of this script is to configure the build environment to use a gcc toolchain when the host is an ubuntu distribution.
Writing a configuration script consists in comparing host and target platforms and, depending on their differences, apply adequate configuration actions. get_Environment_Target_Platform
and get_Environment_Host_Platform
are used to get informations about those platforms that can then be compared so the whole script file should be organized around a hierarchy of if/elseif
tests that define all cases that are managed.
For instance if both target and host platforms have same OS, distribution and processor related properties then it means that host either has a different abi OR a different version. If problem comes from the required version, the script can simply use install_System_Packages
to install or update system packages so that last version managed by the distribution can be installed. Then evaluate_Host_Platform
redo an evaluation of the current host default environment and call again the check file to see if after update the host current configuration now complies with given constraints (i.e. version
constraint). Is yes, the c++ compiler ABI can still be different between host and target, so we call get_Environment_Target_ABI_Flags
to know what flags must be passed to the C++ compiler in order to use OLD or NEW ABI of libstdc++
. Those flags will be set as default in workspace configuration when the configure_Environment_Tool(LANGUAGE CXX FLAGS ${ABI_FLAGS})
is used. The call to return_Environment_Configured(TRUE)
simply tells that the configuration process is finished AND successful.
The following part of the script defines how to configure gcc
to target for instance a raspberry Pi card. In this situation only version 5.4 of gcc/g++ is available AND only for an ubuntu 16.04 OS because we built only this version on this platform. We suppose that the complete toolchain version lies in the folder src/ubuntu/raspi-x86_64
. The configuration here simply consists in targetting adequate tools using the configure_Environment_Tool
function:
...
configure_Environment_Tool(LANGUAGE C COMPILER ${PATH_TO_GCC} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
#c++ compiler (always explicitly setting the C++ ABI)
get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})
configure_Environment_Tool(LANGUAGE CXX COMPILER ${PATH_TO_GPP} FLAGS ${ABI_FLAGS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
#assembler
set(PATH_TO_AS ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-as)
configure_Environment_Tool(LANGUAGE ASM COMPILER ${PATH_TO_AS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
We can so specify all tools (compiler but also ranlib tools and so on) for one or more languages supported by PID. FOr instance, the gcc_toolchain
is used to configure the 3 base languages (ASM
, C
and C++
).
An environment may also change system related settings when a crosscompilation is required:
...
configure_Environment_Tool(SYSTEM LINKER ${PATH_TO_LINKER} NM ${PATH_TO_NM} OBJCOPY ${PATH_TO_OBJCOPY} OBJDUMP ${PATH_TO_OBJDUMP})
set(PATH_TO_SYSROOT ${PATH_TO_ROOT}/armv8-rpi3-linux-gnueabihf/sysroot)
configure_Environment_Tool(SYSTEM SYSROOT ${PATH_TO_SYSROOT})
To do this we use the SYSTEM
keyword instead of LANGUAGE
to say that we configrue system level settings. We use the LINKER
keyword to set path to the systel linker.
Also a sysroot being provided by this gcc toolchain we also set the sysroot using the SYSROOT
keyword.
Remark: an important aspect in this part of the script is that you can also directly provide binaries for a given toolchain version in the source tree of the environment. So you do not have to use system packages only to provide a toolchain. Eventually you could also get a complete toolchain by downloading its source and building it, using host and target platforms specifications to know what to do.
Step 5: Configuring the environment project
Once a description has been defined you can test you environment by doing:
pid cd gcc_toolset
pid build
You should see something like:
[PID] INFO : environment gcc_toolchain has been evaluated.
[PID] description of environment gcc_toolchain solution
- configured languages:
+ C:
* compiler : /usr/bin/cc (id=GNU)
* archiver : /usr/bin/gcc-ar-7
* static libraries creator : /usr/bin/gcc-ranlib-7
+ CXX:
* compiler : /usr/bin/c++ (id=GNU)
* compilation flags : "-D_GLIBCXX_USE_CXX11_ABI=1"
* archiver : /usr/bin/gcc-ar-7
* static libraries creator : /usr/bin/gcc-ranlib-7
+ ASM:
* compiler : /usr/bin/cc (id=GNU)
* archiver : /usr/bin/gcc-ar
* static libraries creator : /usr/bin/gcc-ranlib
The build command of environment has no side effect outside of the environment itself. It is used to test environment. Since there is not target platform constraint specified the environment will simply check if gcc is the default compiler on host.
To test if environment constraints are working as expected you may do:
pid configure environment=gcc_toolset version=7.0
If your host station default compiler toolchain is currently gcc version 7.0 or more, the output will be the same as previously. Now if you do:
pid configure environment=gcc_toolset version=8.2
Output should notify a failure:
[PID] CRITICAL ERROR: cannot configure host with environment gcc_toolchain.
No valid solution found.
Step 5: Referencing the environment
Once your environment is ready to be used you should reference it in the workspace:
pid cd gcc_toolset
pid referencing
Then as for native packages commit the reference file that has just been generated and propose a merge request to the official pid workspace. This way other people can use the environment you defined.
Step 6: Using environments to describe a platform instance
Now we learned how to define new environment in order to manage new toolchains like gcc
, clang
or nvcc
we have to learn how to combine to create more “complete” description of the host and target platforms, what we call platform instance.
A platform instance is a more or less complete description of an host system configured to build for a specific target system. In the following sections we decide to create an environment named test_env
that will completely specify a platform instance in order to reflect the exact configuration of a given host system.
6.1 Create online repository and environment project
Create a new repository with name test_env
into your git hosting server (gitlab for LIRMM), copy its address then:
pid cd
pid create environment=test_env url=<url previously copied>
pid cd test_env
The environment project should have been created and put into folder <pid-workspace>/environments
in subfolder test_env
.
6.2 Describing environment
Edit the root CMakeLists.txt file of test_env
and paste the code:
cmake_minimum_required(VERSION 3.15.7)
set(WORKSPACE_DIR ${CMAKE_SOURCE_DIR}/../.. CACHE PATH "root of the PID workspace directory")
list(APPEND CMAKE_MODULE_PATH ${WORKSPACE_DIR}/cmake) # using generic scripts/modules of the workspace
include(Environment_Definition NO_POLICY_SCOPE)
project(test_env C CXX ASM)
PID_Environment(
AUTHOR Your Name
MAIL your.mail@company.something
YEAR 2019
LICENSE CeCILL-C
ADDRESS <SSH url>
DESCRIPTION "used environment for the test_env host"
)
#give constraints about target platform in use
PID_Environment_Platform(
PLATFORM x86_64_linux_stdc++11
DISTRIBUTION ubuntu
DISTRIB_VERSION 16.04
)
# here no need to check host specific capabilities (no constraint definition), in the end the check will be performed by dependencies
PID_Environment_Dependencies(
gcc_toolchain[version=7.4]
nvcc_toolchain[version=10.0:exact=true]
python_interpreter[version=3.5]
gfortran_toolchain[version=5.5]
)
build_PID_Environment()
Description is made the usual way with same commands as for first example, but arguments changed .
-
PID_Environment_Platform
is now used to define a complete platform instance specification by using togetherPLATFORM
,DISTRIBUTION
andDISTRIB_VERSION
specification. Constraints on target platform are propagated to the configuration scripts and to dependencies. -
PID_Environment_Dependencies
defines the dependencies of environment:gcc_toolchain
,gfortran_toolchain
,python_interpreter
andnvcc_toolchain
. The evaluation of the dependencies will be performed with constraints on target platform specified inPID_Environment_Platform
and additional specific constraints, here the version of toolchains. So for instance the version 7.4 or greater of gcc toolchain is required, which in turn will result in setting the variablegcc_toolchain_version
when evaluating thegcc_toolchain
environment. So in this example thegcc_toolchain
will be evaluated so that it generates a host configuration capable of building with gcc compiler toolchains for ax86_64_linux_stdc++11
target platform with anubuntu 16.04
distribution. Same logic applies for all dependencies.
Remark: there is no solution definition because configurations of build process by test_env
will be finally completely performed by the different dependencies it uses. Adding a configure script is feasible and woud have been useful for instance to configure other build tools than those managed by dependencies (dependencies are always checked first).
So finally environments can also be used to define a complete target platform configuration, allowing to share and reuse sets of build toolchains.
6.3 Use the environment
To test it simply do :
pid cd test_env
pid build
If your host system matches the target platform and all build tools have been found the generation of host configuration will be successful. The output should look like:
...
[PID] INFO : environment robin_laptop_env has been evaluated.
[PID] description of environment robin_laptop_env solution
- configured languages:
+ C:
* compiler : /usr/bin/cc (id=GNU)
* archiver : /usr/bin/gcc-ar-7
* static libraries creator : /usr/bin/gcc-ranlib-7
+ CXX:
* compiler : /usr/bin/c++ (id=GNU)
* compilation flags : "-D_GLIBCXX_USE_CXX11_ABI=1"
* archiver : /usr/bin/gcc-ar-7
* static libraries creator : /usr/bin/gcc-ranlib-7
+ ASM:
* compiler : /usr/bin/cc (id=GNU)
* archiver : /usr/bin/gcc-ar
* static libraries creator : /usr/bin/gcc-ranlib
+ CUDA:
* compiler : /usr/local/cuda-10.0/bin/nvcc (id=NVIDIA)
* compilation flags : "-gencode arch=compute_61,code=sm_61 -D_FORCE_INLINES"
* standard library : /usr/local/cuda-10.0/lib64/libcudart.so
* host compiler : /usr/bin/cc
+ Python:
* standard library include dirs : /usr/include/python3.5m
* standard library : /usr/lib/x86_64-linux-gnu/libpython3.5m.so
* interpreter : /usr/bin/python3
+ Fortran:
* compiler : /usr/bin/f95 (id=GNU)
* archiver : /usr/bin/gcc-ar-5
* static libraries creator : /usr/bin/gcc-ranlib-5
But if your host platform is different then the configuration process may lead either :
- to an error, if no solution exists to configure your host for generating code for the target platform. This means that at least one of the dependencies cannot be resolved either due to its specific constraint (e.g. version requirements) or because there is no solution for targetting target platform on current host. If you face this issue the best solution is to test the faulty dependencies one by one and try to provide a new solution in this dependency or create a new one (see first part of tutorial).
- to the use of specific toolchains versions if all dependencies propose a solution for target platform constraints.
6.4 Some advices
Environments are used to configure the workspace for a host system to be capable of building for a target platform using a set of compilers toolchains. It can be used to integrate new toolchain, but this part is mostly for experienced users.
For less advanced users one good usage is to define a specific environement that reflects the desired configuration ot its workstation. This way he/she can easily configure any workstation in use and even share this configuration with other users.
This can also be used by machine maintainers to reflect the configuration of a machine (typically the embedded computer of a robot). In this way they can define a complete description of their machines’ configuration that can be used during CI process to automate the building of binaries for their machines.
Finally, for dev ops this is a solution to enforce reproductible build.
Now you know how to generate configurations for customizing host platform compiler toolchains, let’s see how to create a plugin using environments.