It is sometimes required or preferred to use specific build tools. The reason may be for instance to enforce specific compiler usage, in particular needed to crosscompile to a specific target platform (e.g. targeting a raspberry pi card). In PID customizing the build environment used at workspace level is made with environments. An environment is a deployment unit defining how to configure the development tools of the host environment.

Generally we use environments to achieve different tasks:

  • managing a toolchain. For instance gcc_toolchain environment defines how to configure the build process when using the GNU C/C++ compiler ; nvcc_toolchain environment defines how to configure the build process when using the NVIDIA CUDA compiler, etc.

  • defining plugins in order to use development tools not managed by default. For instance pkg-config environment defines how to generate configuration files for the pkg-config tool, which can be usefull to provide a way to use packages deployed into a workspace from a third party project.

  • describing a specific platform instance, this is the description of the complete build environment for a given target platform. They are used to agregate a set of toolchains and plugins with specific constraints (for instance minimum version required).

Following sections explain how to define new environments.

Warning

Writing environment may be a complex task, and is very hard to generalize as it is completey bound to the specificities of the host, of the compiler in use, of the target platform, etc.

That is why writing new environments should be reserved to advanced users.

Step 1: create the environment repository and project

First of all we need to create the project for the environment and its online repository.

1.1 Create online repository

Go into your git hosting server (gitlab for LIRMM) and create a new repository with same name as environment projects.

In this example we create a repository named gcc_toolset, that will finally be more or less a copy of gcc_toolchain. We name it this way to avoid conflicts with the already existing gcc_toolchain environment.

Then copy the SSH address of this git repository.

1.2 Create the environment project

cd <pid-workspace>
pid create environment=gcc_toolset url=<url previously copied>
cd <pid-workspace>/environments/gcc_toolset

The environment project should have been created and put into folder <pid-workspace>/environments. In the following sections we use <gcc_toolset> as a short name for the path to <pid-workspace>/environments/gcc_toolset.

Step 2: describe the content of the environment

Now first thing to do is to define adequate meta information of the environment. Edit the CMakeListst.txt file in <gcc_toolset> and paste the code:

CMAKE_MINIMUM_REQUIRED(VERSION 3.0.2)
set(WORKSPACE_DIR ${CMAKE_SOURCE_DIR}/../.. CACHE PATH "root of the PID workspace directory")
list(APPEND CMAKE_MODULE_PATH ${WORKSPACE_DIR}/share/cmake/system) # using generic scripts/modules of the workspace
include(Environment_Definition NO_POLICY_SCOPE)

project(gcc_toolchain ASM C CXX)

PID_Environment(
      AUTHOR 		        Robin Passama
			INSTITUTION				CNRS/LIRMM
			MAIL							robin.passama@lirmm.fr
			YEAR 							2019
			LICENSE 					CeCILL-C
			ADDRESS						git@gite.lirmm.fr:pid/environments/gcc_toolchain.git
			PUBLIC_ADDRESS		https://gite.lirmm.fr/pid/environments/gcc_toolchain.git
      CONTRIBUTION_SPACE pid
			DESCRIPTION 			"using GNU gcc toolchain to build C/C++ code of projects"
		)

PID_Environment_Constraints(IN_BINARY version # give the version of the desired gcc toolset in use
                            OPTIONAL  exact
                            CHECK check_gcc.cmake) # check script for current configuration
#for now only define a solution for ubuntu distributions
PID_Environment_Solution(HOST CONFIGURE configure_generic_gcc.cmake)
PID_Environment_Solution(OS linux DISTRIBUTION ubuntu CONFIGURE ubuntu/configure_gcc.cmake)
build_PID_Environment()

Explanations:

  • include(Environment_Definition NO_POLICY_SCOPE) is used to import the API for writing environments.
  • PID_Environment command (equivalent to declare_PID_Environment) transforms the current CMake project into a PID environment. Arguments to provide are more or less the same as for native packages:

    • AUTHOR, INSTITUTION, MAIL refer to the contact author of the environment project.
    • ADDRESS and PUBLIC_ADDRESS have the same meaning than for native packages, they are used to configure official git remote.
    • YEAR, LICENSE, CONTRIBUTION_SPACE and DESCRIPTION have the same meaning than for native packages.
  • PID_Environment_Constraints defines environment specific constraints that can be applied to evaluate current environment. The CHECK keyword defines the CMake script file used to check if host system configuration matches constraints. In the example, the script check_gcc.cmake that lies in project src folder. Basically it checks if gcc toolchain is the current toolchain. If constraints are defined it then also checks that gcc toolchain matches these constraints. In this example version and exact constraints are defined, as usual for toolchain definition environments.

    • exact is optional (defined using the OPTIONAL keyword) so it can be used or not.
    • version is also optional but mandatory in banaries (defined using the IN_BINARY keyword). This means that any package that explicitly requires this environment will have its version information included in its description.
  • PID_Environment_Solution defines a solution used to deploy the toolchain when host does not macth target requirements. In this example we have only two solutions: one applying when host matches target ; the second applying when the host is a linux ubuntu system. Solutions are evaluated sequentially so if first one succeeded, the second will not be evaluated.

    • CONFIGURE argument defines the script file used to generate the adequate configuration.
    • there are filters to to those solutions dependening on host system features: OS and DISTRIBUTION are used to specify for which type of host system the solution apply. if using the HOST kerword, it means that any host can be used.
  • build_PID_Environment configures the environment and create its build commands.

Step 3: Writing the check script

The first thing to do is to edit the script src/check_gcc.cmake. This script simply checks if host is already configured with the toolchain defined by gcc_toolset, eventually taking into account constraints defined in root CMakeLists.txt. Edit this file and paste the code:

# check if host already matches the constraints
#host must have a GNU compiler !!
if(NOT CMAKE_CXX_COMPILER_ID STREQUAL "GNU"
   OR NOT CMAKE_C_COMPILER_ID STREQUAL "GNU"
   OR NOT CMAKE_ASM_COMPILER_ID STREQUAL "GNU")
  return_Environment_Check(FALSE)
endif()

#check C compiler version
if(NOT CMAKE_C_COMPILER_VERSION)
  check_Program_Version(IS_GCC_OK gcc_toolchain_version "${gcc_toolchain_exact}" "gcc -v" "^gcc[ \t]+version[ \t]+([^ \t]+)[ \t]*.*$")
else()
  check_Environment_Version(IS_GCC_OK gcc_toolchain_version "${gcc_toolchain_exact}" "${CMAKE_C_COMPILER_VERSION}")
endif()
if(NOT IS_GCC_OK)
  return_Environment_Check(FALSE)
endif()

#check CXX compiler version
if(NOT CMAKE_CXX_COMPILER_VERSION)
  check_Program_Version(IS_GPP_OK gcc_toolchain_version "${gcc_toolchain_exact}" "g++ -v" "^gcc[ \t]+version[ \t]+([^ \t]+)[ \t]*.*$")
else()
  check_Environment_Version(IS_GPP_OK gcc_toolchain_version "${gcc_toolchain_exact}" "${CMAKE_CXX_COMPILER_VERSION}")
endif()
if(NOT IS_GPP_OK)
  return_Environment_Check(FALSE)
endif()

return_Environment_Check(TRUE)

The check script basically tests if the default host ASM/C/C++ compiler are GNU compilers. If not then return_Environment_Check(FALSE) exits this script on error.

The script should also takes into account defined constraints, in this example the version related constraints. The variable gcc_toolchain_version is correctly valued according to value given to the version constraint or is not defined if no version constraint has been specified. So it is simple to handle constraints in CMake script by simly checking value of variables with pattern <environment>_<constraint>. Furthermore, the environment API provides a set of functions that can be used to ease the writing of the script. For instance, check_Program_Version and check_Environment_Version are provided to test the common arguments version and exact, respectively on any program capable of outputing its version number or using a variable provided by CMake.

Finally if all tests succeeded the script should exit with return_Environment_Check(TRUE).

Step 4: Writing the configure scripts

Last part consists in writing configuration scripts for each solution.

4.1: generic configuration script

For the first solution, the script is src/configure_generic_gcc.cmake and looks like:

# check if host matches the target platform
host_Match_Target_Platform(MATCHING)
if(NOT MATCHING)
  return_Environment_Configured(FALSE)
endif()

evaluate_Host_Platform(EVAL_RESULT)
if(EVAL_RESULT)
  # thats it for checks => host matches all requirements of this solution
  # simply set the adequate variables
  configure_Environment_Tool(LANGUAGE C CURRENT)
  configure_Environment_Tool(LANGUAGE CXX CURRENT)
  configure_Environment_Tool(LANGUAGE ASM CURRENT)
  set_Environment_Constraints(VARIABLES version
                              VALUES     ${CMAKE_C_COMPILER_VERSION})
  return_Environment_Configured(TRUE)
endif()

return_Environment_Configured(FALSE)

The role of the configuration script is to define a configuration of the host platform in order to allow the use of a given set of tools, in this example the compiler and other associated tools that constitute the gcc_toolchain. The configuration itself is performed using the configure_Environment_Tool that is used to memorize configuration choices. Since we want to define a set of tools related to a given set of languages (C, C++ and ASM) we need to provide a valid configuration for these languages using the LANGUAGE argument.

Since this script is supposed to be executed on any host platform, first step consists in testing if host matches target platform (using function host_Match_Target_Platform) otherwise solution evaluation fails. Indeed, there is few chances to write a generic host configruation script that can automatically manage cross compilation features.

Then evaluate_Host_Platform is simply used to automatically call the check script defined at previous step. If evaluation succeeded the script memorize corresponding settings for C, C++ and ASM languages using the configure_Environment_Tool function. The CURRENT argument is here used to tell to memorize current CMake configuration of the environment project itself (i.e. everything related to the compiler toolchain in use for corresponding languages). It also memorizes the value of constraints that will be published in binary description of packages (REQUIRED and IN_BINARY constraints like version) using set_Environment_Constraints.

Finally, the script returns evaluation success or failure using return_Environment_Configured function.

4.2: configuration script for an ubuntu host

The second solution is used to deploy specific version of the toolchain on an ubuntu host system, and is described in src/ubuntu/configure_gcc.cmake file:

# if this script executes code is built on a ubuntu system
# build the pool of available versions depending on the target specification
get_Environment_Target_Platform(DISTRIBUTION target_distrib DISTRIB_VERSION target_distrib_version
TYPE target_proc ARCH target_bits OS target_os ABI target_abi)
get_Environment_Host_Platform(DISTRIB_VERSION host_distrib_version
TYPE proc_host ARCH bits_host ABI host_abi)


if(target_os STREQUAL linux)
  if(target_distrib STREQUAL ubuntu)#target is ubuntu and host is ubuntu as well (by definition if this script is called)
    if(host_distrib_version STREQUAL target_distrib_version)
      if(proc_host STREQUAL target_proc AND bits_host EQUAL target_bits)#same processor architecture => the problem may simply come from the version of the compiler in use
        #I know the procedure to install gcc whatever the processor specification are (and I know binaries will be compatibles)
        install_System_Packages(APT gcc g++)#getting last version of gcc/g++
        evaluate_Host_Platform(EVAL_RESULT)#evaluate again the host (only check that version constraint is satisfied)
        if(EVAL_RESULT)
          #only ABI may be not compliant
          get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})
          if(ABI_FLAGS)#an ABI constraint is explicilty specified => need to force it!
            configure_Environment_Tool(LANGUAGE CXX CURRENT FLAGS ${ABI_FLAGS})
          endif()
          configure_Environment_Tool(LANGUAGE C CURRENT)
          configure_Environment_Tool(LANGUAGE ASM CURRENT)
          set_Environment_Constraints(VARIABLES version
                                      VALUES     ${CMAKE_C_COMPILER_VERSION})
          return_Environment_Configured(TRUE)#that is OK gcc and g++ have just been updated and are now OK
        endif()#no solution found with OS installers -> using alternative
      elseif(proc_host STREQUAL target_proc AND bits_host EQUAL 64 AND target_bits EQUAL 32)
        # host is a 64 bits system and target is a 32 bits => on ubuntu I can generate 32 compatible code
        # I know the procedure to install gcc whatever the processor specification are since same processor type
        install_System_Packages(APT gcc g++ gcc-multilib g++-multilib)#getting last version of gcc/g++ AND multi arch support for gcc
        evaluate_Host_Platform(EVAL_RESULT)#evaluate again the host (only check that version constraint is satisfied)
        if(EVAL_RESULT)
          get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})
          configure_Environment_Tool(LANGUAGE ASM FLAGS -m32)
          configure_Environment_Tool(LANGUAGE C FLAGS -m32)
          configure_Environment_Tool(LANGUAGE CXX FLAGS -m32 ${ABI_FLAGS})
          configure_Environment_Tool(SYSTEM LIBRARY_DIRS /lib32 /usr/lib32)
          set_Environment_Constraints(VARIABLES version
                                      VALUES     ${CMAKE_C_COMPILER_VERSION})
          return_Environment_Configured(TRUE)#that is OK gcc and g++ have just been updated and are now OK
        endif()#no solution found with OS installers -> using alternative
      endif()
    endif()
  elseif(target_distrib STREQUAL raspbian
        AND target_proc STREQUAL arm
        AND target_bits EQUAL 32)#targetting a raspbian v3+ system

    if( proc_host STREQUAL x86
        AND bits_host EQUAL 64)#we have built a cross compiler for this situation !!

        set(POSSIBLE_VERSIONS 5.4.0 8.3.0)
        set(version_selected)
        if(gcc_toolset_version) #there is a constraint on version
          if(gcc_toolset_exact)
            list(FIND POSSIBLE_VERSIONS ${gcc_toolset_version} INDEX)
            if(INDEX EQUAL -1)
              return_Environment_Configured(FALSE)#required gcc version is not provided by the environment
            endif()
          else()
            foreach(version IN LISTS POSSIBLE_VERSIONS)
              if(gcc_toolset_version VERSION_LESS version)
                #an adequate version (>= required version) has been found
                if(version VERSION_LESS version_selected)
                  set(version_selected ${version})#take the version that is the closest to the required one
                endif()
              endif()
            endforeach()
            if(NOT version_selected)#no compatible version found
              return_Environment_Configured(FALSE)#compatible gcc version is not provided by the environment
            endif()
          endif()
        else()#otherwise any version possible => take the biggest one
          foreach(version IN LISTS POSSIBLE_VERSIONS)
            if(version VERSION_GREATER version_selected)
              set(version_selected ${version})
            endif()
          endforeach()
        endif()
        if(version_selected VERSION_EQUAL 5.4.0)
          set(PATH_TO_ROOT ${CMAKE_SOURCE_DIR}/src/ubuntu/raspi-x86_64/armv8-rpi3-linux-gnueabihf-5.4.0)
        elseif(version_selected VERSION_EQUAL 8.3.0)
          set(PATH_TO_ROOT ${CMAKE_SOURCE_DIR}/src/ubuntu/raspi-x86_64/armv8-rpi3-linux-gnueabihf-8.3.0)
        else()#PROBLEM => error
          message("[PID] INTERNAL ERROR: no version selected for raspberry pi v3 cross compiler => aborting")
          return_Environment_Configured(FALSE)#compatible gcc version is not provided by the environment
        endif()
        #C compiler
        set(PATH_TO_GCC ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-gcc)
        set(PATH_TO_GCC_AR ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-ar)
        set(PATH_TO_GCC_RANLIB ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-ranlib)
        set(PATH_TO_GPP ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-g++)
        configure_Environment_Tool(LANGUAGE C COMPILER ${PATH_TO_GCC} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
        #c++ compiler (always explicitly setting the C++ ABI)
        get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})
        configure_Environment_Tool(LANGUAGE CXX COMPILER ${PATH_TO_GPP} FLAGS ${ABI_FLAGS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})#Note same ar and ranlib than for gcc
        #assembler
        set(PATH_TO_AS ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-as)
        configure_Environment_Tool(LANGUAGE ASM COMPILER ${PATH_TO_AS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})#Note same ar and ranlib than for gcc

        # system tools
        set(PATH_TO_LINKER ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-ld)
        set(PATH_TO_NM ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-nm)
        set(PATH_TO_OBJCOPY ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-objcopy)
        set(PATH_TO_OBJDUMP ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-objdump)
        configure_Environment_Tool(SYSTEM LINKER ${PATH_TO_LINKER} NM ${PATH_TO_NM} OBJCOPY ${PATH_TO_OBJCOPY} OBJDUMP ${PATH_TO_OBJDUMP})

        #sysroot !!
        set(PATH_TO_SYSROOT ${PATH_TO_ROOT}/armv8-rpi3-linux-gnueabihf/sysroot)
        configure_Environment_Tool(SYSTEM SYSROOT ${PATH_TO_SYSROOT})

        # TODO See if necessary
        # SET(CMAKE_FIND_ROOT_PATH $ENV{HOME}/x-tools/armv8-rpi3-linux-gnueabihf)
        set_Environment_Constraints(VARIABLES version
                                    VALUES     ${version_selected})
        return_Environment_Configured(TRUE)
    endif()
    #TODO directly put the compiler for proc x86 64 bits
  #else add other OS/distrib when supported
  endif()
elseif(target_os STREQUAL Generic)#no target OS => we compile for bare metal system
  if(target_proc STREQUAL "arm") # compile for ARM microcontrollers
    if(target_bits EQUAL 32)#armv7 for pic microcontrollers
      install_System_Packages(APT gcc-arm-none-eabi)#getting last version of gcc/g++ for that target
      find_program(PATH_TO_GCC arm-none-eabi-gcc)
      if(PATH_TO_GCC-NOTFOUND)
        return_Environment_Configured(FALSE)#not found after install => problem
      endif()
      find_program(PATH_TO_GPP arm-none-eabi-g++)
      if(PATH_TO_GPP-NOTFOUND)
        return_Environment_Configured(FALSE)#not found after install => problem
      endif()
      find_program(PATH_TO_AS arm-none-eabi-as)
      if(PATH_TO_AS-NOTFOUND)
        return_Environment_Configured(FALSE)#not found after install => problem
      endif()
      find_program(PATH_TO_GCC_AR arm-none-eabi-gcc-ar)
      if(PATH_TO_GCC_AR-NOTFOUND)
        return_Environment_Configured(FALSE)#not found after install => problem
      endif()
      find_program(PATH_TO_GCC_RANLIB arm-none-eabi-gcc-ranlib)
      if(PATH_TO_GCC_RANLIB-NOTFOUND)
        return_Environment_Configured(FALSE)#not found after install => problem
      endif()
      find_program(PATH_TO_LINKER arm-none-eabi-ld)
      if(PATH_TO_LINKER-NOTFOUND)
        return_Environment_Configured(FALSE)#not found after install => problem
      endif()
      find_program(PATH_TO_NM arm-none-eabi-nm)
      if(PATH_TO_NM-NOTFOUND)
        return_Environment_Configured(FALSE)#not found after install => problem
      endif()
      find_program(PATH_TO_OBJCOPY arm-none-eabi-objcopy)
      if(PATH_TO_OBJCOPY-NOTFOUND)
        return_Environment_Configured(FALSE)#not found after install => problem
      endif()
      find_program(PATH_TO_OBJDUMP arm-none-eabi-objdump)
      if(PATH_TO_OBJDUMP-NOTFOUND)
        return_Environment_Configured(FALSE)#not found after install => problem
      endif()
      #now set the compiler
      get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})#computing compiler flags to get the adequate target c++ ABI
      configure_Environment_Tool(LANGUAGE C COMPILER ${PATH_TO_GCC}  TOOLCHAIN_ID "GNU" AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
      configure_Environment_Tool(LANGUAGE CXX COMPILER ${PATH_TO_GPP} TOOLCHAIN_ID "GNU" FLAGS ${ABI_FLAGS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
      configure_Environment_Tool(LANGUAGE ASM COMPILER ${PATH_TO_AS} TOOLCHAIN_ID "GNU" AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
      configure_Environment_Tool(SYSTEM LINKER ${PATH_TO_LINKER} NM ${PATH_TO_NM} OBJCOPY ${PATH_TO_OBJCOPY} OBJDUMP ${PATH_TO_OBJDUMP})
      get_Configured_Environment_Tool(LANGUAGE C COMPILER c_compiler)
      check_Program_Version(RES_VERSION gcc_toolchain_version "${gcc_toolchain_exact}" "${c_compiler} -v" "^gcc[ \t]+version[ \t]+([^ \t]+)[ \t]+.*$")
      if(RES_VERSION)
        set_Environment_Constraints(VARIABLES version
                                    VALUES     ${RES_VERSION})
        return_Environment_Configured(TRUE)
      endif()
    endif()
  endif()
endif()

#TODO add more specific compiler that depends on hardware
return_Environment_Configured(FALSE)

Explanations:

As you may see the script is a bit complex (and has even been truncated from its original version for sake of readbility). The goal of this script is to configure the build environment to use a gcc toolchain when the host is an ubuntu distribution.

Writing a configuration script consists in comparing host and target platforms and, depending on their differences, apply adequate configuration actions. get_Environment_Target_Platform and get_Environment_Host_Platform are used to get informations about those platforms that can then be compared so the whole script file should be organized around a hierarchy of if/elseif tests that define all cases that are managed.

For instance if both target and host platforms have same OS, distribution and processor related properties then it means that host either has a different abi OR a different version. If problem comes from the required version, the script can simply use install_System_Packages to install or update system packages so that last version managed by the distribution can be installed. Then evaluate_Host_Platform redo an evaluation of the current host default environment and call again the check file to see if after update the host current configuration now complies with given constraints (i.e. version constraint). Is yes, the c++ compiler ABI can still be different between host and target, so we call get_Environment_Target_ABI_Flags to know what flags must be passed to the C++ compiler in order to use OLD or NEW ABI of libstdc++. Those flags will be set as default in workspace configuration when the configure_Environment_Tool(LANGUAGE CXX FLAGS ${ABI_FLAGS}) is used. The call to return_Environment_Configured(TRUE) simply tells that the configuration process is finished AND successful.

The following part of the script defines how to configure gcc to target for instance a raspberry Pi card. In this situation only version 5.4 of gcc/g++ is available AND only for an ubuntu 16.04 OS because we built only this version on this platform. We suppose that the complete toolchain version lies in the folder src/ubuntu/raspi-x86_64. The configuration here simply consists in targetting adequate tools using the configure_Environment_Tool function:

...
configure_Environment_Tool(LANGUAGE C COMPILER ${PATH_TO_GCC} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
#c++ compiler (always explicitly setting the C++ ABI)
get_Environment_Target_ABI_Flags(ABI_FLAGS ${target_abi})
configure_Environment_Tool(LANGUAGE CXX COMPILER ${PATH_TO_GPP} FLAGS ${ABI_FLAGS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})
#assembler
set(PATH_TO_AS ${PATH_TO_ROOT}/bin/armv8-rpi3-linux-gnueabihf-as)
configure_Environment_Tool(LANGUAGE ASM COMPILER ${PATH_TO_AS} AR ${PATH_TO_GCC_AR} RANLIB ${PATH_TO_GCC_RANLIB})

We can so specify all tools (compiler but also ranlib tools and so on) for one or more languages supported by PID. FOr instance, the gcc_toolchain is used to configure the 3 base languages (ASM, C and C++).

An environment may also change system related settings when a crosscompilation is required:

...
configure_Environment_Tool(SYSTEM LINKER ${PATH_TO_LINKER} NM ${PATH_TO_NM} OBJCOPY ${PATH_TO_OBJCOPY} OBJDUMP ${PATH_TO_OBJDUMP})
set(PATH_TO_SYSROOT ${PATH_TO_ROOT}/armv8-rpi3-linux-gnueabihf/sysroot)
configure_Environment_Tool(SYSTEM SYSROOT ${PATH_TO_SYSROOT})

To do this we use the SYSTEM keyword instead of LANGUAGE to say that we configrue system level settings. We use the LINKER keyword to set path to the systel linker.

Also a sysroot being provided by this gcc toolchain we also set the sysroot using the SYSROOT keyword.

Remark: an important aspect in this part of the script is that you can also directly provide binaries for a given toolchain version in the source tree of the environment. So you do not have to use system packages only to provide a toolchain. Eventually you could also get a complete toolchain by downloading its source and building it, using host and target platforms specifications to know what to do.

Step 5: Configuring the environment project

Once a description has been defined you can test you environment by doing:

pid cd gcc_toolset
pid build

You should see something like:

[PID] INFO : environment gcc_toolchain has been evaluated.
[PID] description of environment gcc_toolchain solution
- configured languages:
  + C:
    * compiler : /usr/bin/cc (id=GNU)
    * archiver : /usr/bin/gcc-ar-7
    * static libraries creator : /usr/bin/gcc-ranlib-7

  + CXX:
    * compiler : /usr/bin/c++ (id=GNU)
    * compilation flags : "-D_GLIBCXX_USE_CXX11_ABI=1"
    * archiver : /usr/bin/gcc-ar-7
    * static libraries creator : /usr/bin/gcc-ranlib-7

  + ASM:
    * compiler : /usr/bin/cc (id=GNU)
    * archiver : /usr/bin/gcc-ar
    * static libraries creator : /usr/bin/gcc-ranlib

The build command of environment has no side effect outside of the environment itself. It is used to test environment. Since there is not target platform constraint specified the environment will simply check if gcc is the default compiler on host.

To test if environment constraints are working as expected you may do:

pid configure environment=gcc_toolset version=7.0

If your host station default compiler toolchain is currently gcc version 7.0 or more, the output will be the same as previously. Now if you do:

pid configure environment=gcc_toolset version=8.2

Output should notify a failure:

[PID] CRITICAL ERROR: cannot configure host with environment gcc_toolchain.
 No valid solution found.

Step 5: Referencing the environment

Once your environment is ready to be used you should reference it in the workspace:

pid cd gcc_toolset
pid referencing

Then as for native packages commit the reference file that has just been generated and propose a merge request to the official pid workspace. This way other people can use the environment you defined.

Step 6: Using environments to describe a platform instance

Now we learned how to define new environment in order to manage new toolchains like gcc, clang or nvcc we have to learn how to combine to create more “complete” description of the host and target platforms, what we call platform instance.

A platform instance is a more or less complete description of an host system configured to build for a specific target system. In the following sections we decide to create an environment named test_env that will completely specify a platform instance in order to reflect the exact configuration of a given host system.

6.1 Create online repository and environment project

Create a new repository with name test_env into your git hosting server (gitlab for LIRMM), copy its address then:

pid cd
pid create environment=test_env url=<url previously copied>
pid cd test_env

The environment project should have been created and put into folder <pid-workspace>/environments in subfolder test_env.

6.2 Describing environment

Edit the root CMakeLists.txt file of test_env and paste the code:

CMAKE_MINIMUM_REQUIRED(VERSION 3.0.2)
set(WORKSPACE_DIR ${CMAKE_SOURCE_DIR}/../.. CACHE PATH "root of the PID workspace directory")
list(APPEND CMAKE_MODULE_PATH ${WORKSPACE_DIR}/cmake) # using generic scripts/modules of the workspace
include(Environment_Definition NO_POLICY_SCOPE)

project(test_env C CXX ASM)

PID_Environment(
    AUTHOR      Your Name
    MAIL        your.mail@company.something
    YEAR        2019
    LICENSE     CeCILL-C
    ADDRESS     <SSH url>
    DESCRIPTION "used environment for the test_env host"
)

#give constraints about target platform in use
PID_Environment_Platform(
  PLATFORM x86_64_linux_stdc++11
  DISTRIBUTION ubuntu
  DISTRIB_VERSION 16.04
)

# here no need to check host specific capabilities (no constraint definition), in the end the check will be performed by dependencies
PID_Environment_Dependencies(
  gcc_toolchain[version=7.4]
  nvcc_toolchain[version=10.0:exact=true]
  python_interpreter[version=3.5]
  gfortran_toolchain[version=5.5]
)

build_PID_Environment()

Description is made the usual way with same commands as for first example, but arguments changed .

  • PID_Environment_Platform is now used to define a complete platform instance specification by using together PLATFORM, DISTRIBUTION and DISTRIB_VERSION specification. Constraints on target platform are propagated to the configuration scripts and to dependencies.

  • PID_Environment_Dependencies defines the dependencies of environment: gcc_toolchain, gfortran_toolchain, python_interpreter and nvcc_toolchain. The evaluation of the dependencies will be performed with constraints on target platform specified in PID_Environment_Platform and additional specific constraints, here the version of toolchains. So for instance the version 7.4 or greater of gcc toolchain is required, which in turn will result in setting the variable gcc_toolchain_version when evaluating the gcc_toolchain environment. So in this example the gcc_toolchain will be evaluated so that it generates a host configuration capable of building with gcc compiler toolchains for a x86_64_linux_stdc++11 target platform with an ubuntu 16.04 distribution. Same logic applies for all dependencies.

Remark: there is no solution definition because configurations of build process by test_env will be finally completely performed by the different dependencies it uses. Adding a configure script is feasible and woud have been useful for instance to configure other build tools than those managed by dependencies (dependencies are always checked first).

So finally environments can also be used to define a complete target platform configuration, allowing to share and reuse sets of build toolchains.

6.3 Use the environment

To test it simply do :

pid cd test_env
pid build

If your host system matches the target platform and all build tools have been found the generation of host configuration will be successful. The output should look like:

...
[PID] INFO : environment robin_laptop_env has been evaluated.
[PID] description of environment robin_laptop_env solution
- configured languages:
  + C:
    * compiler : /usr/bin/cc (id=GNU)
    * archiver : /usr/bin/gcc-ar-7
    * static libraries creator : /usr/bin/gcc-ranlib-7

  + CXX:
    * compiler : /usr/bin/c++ (id=GNU)
    * compilation flags : "-D_GLIBCXX_USE_CXX11_ABI=1"
    * archiver : /usr/bin/gcc-ar-7
    * static libraries creator : /usr/bin/gcc-ranlib-7

  + ASM:
    * compiler : /usr/bin/cc (id=GNU)
    * archiver : /usr/bin/gcc-ar
    * static libraries creator : /usr/bin/gcc-ranlib

  + CUDA:
    * compiler : /usr/local/cuda-10.0/bin/nvcc (id=NVIDIA)
    * compilation flags : "-gencode arch=compute_61,code=sm_61 -D_FORCE_INLINES"
    * standard library : /usr/local/cuda-10.0/lib64/libcudart.so
    * host compiler : /usr/bin/cc

  + Python:
    * standard library include dirs : /usr/include/python3.5m
    * standard library : /usr/lib/x86_64-linux-gnu/libpython3.5m.so
    * interpreter : /usr/bin/python3

  + Fortran:
    * compiler : /usr/bin/f95 (id=GNU)
    * archiver : /usr/bin/gcc-ar-5
    * static libraries creator : /usr/bin/gcc-ranlib-5

But if your host platform is different then the configuration process may lead either :

  • to an error, if no solution exists to configure your host for generating code for the target platform. This means that at least one of the dependencies cannot be resolved either due to its specific constraint (e.g. version requirements) or because there is no solution for targetting target platform on current host. If you face this issue the best solution is to test the faulty dependencies one by one and try to provide a new solution in this dependency or create a new one (see first part of tutorial).
  • to the use of specific toolchains versions if all dependencies propose a solution for target platform constraints.

6.4 Some advices

Environments are used to configure the workspace for a host system to be capable of building for a target platform using a set of compilers toolchains. It can be used to integrate new toolchain, but this part is mostly for experienced users.

For less advanced users one good usage is to define a specific environement that reflects the desired configuration ot its workstation. This way he/she can easily configure any workstation in use and even share this configuration with other users.

This can also be used by machine maintainers to reflect the configuration of a machine (typically the embedded computer of a robot). In this way they can define a complete description of their machines’ configuration that can be used during CI process to automate the building of binaries for their machines.

Finally, for dev ops this is a solution to enforce reproductible build.


Now you know how to generate configurations for customizing host platform compiler toolchains, let’s see how to create a plugin using environments.