This part provides a detailed description on the usage of PID methodology’s core concepts and tools used to deal with wrappers.

Wrapper definition

When developing a PID wrapper one needs to handle information such as:

  • meta-information : who is involved in its development ? what is its general purpose ? where to find this package on the network ? what is the license of the code ? etc.
  • build-related information : what are the components provided by the wrapper and how to compile/link them from source code ? How components are tested ? what is the version of the package ? what are its functional dependencies ?

The whole package development is organized and described with CMake, and the CMakeLists.txt files used contain the whole meta-information and build-related information used by the wrapper. Each package contains several CMakeLists.txt files that are generally used to define components :

  • the root CMakeLists.txt file is used to define meta-information and dependencies of the wrapper.
  • the CMakeLists.txt files contained in each version subfolder of the src folder defines the components available for each version of the external package that is known into PID. They also reference a cmake deploy script that will be used to download, build and install the given version of the external project.

In the context of PID, components are software artefacts generated and installed by a package or a wrapper. Each component may require other components from another external package and so may explicitly define dependencies between wrappers. The primary role of PID is to manage these dependencies.

General description

The wrapper general description takes place in the root CMakeLists.txt. This file contains the description of meta-information of the wrapper.

Let’s suppose we define a wrapper for yaml-cpp, its root CMakeLists.txt could look like:

cmake_minimum_required(VERSION 3.0.2) #minimum version of CMake to use PID
set(WORKSPACE_DIR ${CMAKE_SOURCE_DIR}/../.. CACHE PATH "root of the packages workspace directory")
list(APPEND CMAKE_MODULE_PATH ${WORKSPACE_DIR}/share/cmake/system) #registereing PID CMake scripts
include(Wrapper_Definition NO_POLICY_SCOPE) # use the PID API
project(yaml-cpp)

PID_Wrapper(
	AUTHOR      Robin Passama
	MAIL        passama@lirmm.fr
	INSTITUTION	CNRS / LIRMM: Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier, www.lirmm.fr
	ADDRESS git@gite.lirmm.fr:pid/yaml-cpp.git
	PUBLIC_ADDRESS https://gite.lirmm.fr/pid/yaml-cpp.git
	YEAR 		2018
	LICENSE 	MIT
	DESCRIPTION 	"a YAML parser library for C++"
)
define_PID_Wrapper_Original_Project_Info(
	AUTHORS "yaml-cpp authors"
	LICENSES "MIT License"
	URL https://code.google.com/p/yaml-cpp/)

Explanations:

  • Exactly as in any CMake project, the CMake project is defined with the PROJECT keyword. The CMake project’s name must be the same as the name of the git repository and must be the same as the project root folder name. The previous lines must be let unchanged (initialization of the PID specific cmake API).

  • Then comes the PID_Wrapper (equivalent to declare_PID_Wrapper) macro, which is mandatory. This macro defines general meta-information on the wrapper, and should not change a lot during wrapper life cycle:

    • the main author (AUTHOR keyword) and its institution (INSTITUTION optional keyword), as well as his/her email (MAIL keyword), considered as the maintainer of the wrapper .
    • the YEAR field helps defining the wrapper’s life cycle range.
    • the LICENSE field is used to specify the license that applies to the wrapper code. This license must be defined in the workspace in the form of a license file.
    • the ADDRESS field is used to specify the address of the official GIT repository of the wrapper.
    • the PUBLIC_ADDRESS field is used to specify the public address of the same official GIT repository of the wrapper, public meaning that the clone can be done without any identification.
    • the DESCRIPTION field must be filled with a short description of the wrapper usage/utility.
  • Then the user fill meta-information about original external project by using define_PID_Wrapper_Original_Project_Info:

    • the AUTHORS keyword help defining who are the authors of the original project, in order to clearly identify them.
    • the LICENSES keywork is used to list licenses applying to the original project code.
    • the URL keyword document where to find online documentation about original project.

Managed versions description

Then you have to create version subfolders in the src folder of the wrapper. Each of these folders contains the the deployment procedure of the external project and the description of the content resulting from its build process. The whole version is described in a CMakeLists.txt that looks like this:

#declaring a new known version
PID_Wrapper_Version(VERSION 0.5.1 DEPLOY deploy.cmake
                    SONAME 0.5 #define the extension name to use for shared objects
										)
#now describe the content
PID_Wrapper_Configuration(CONFIGURATION posix) #depends only on posix
PID_Wrapper_Dependency(boost)#any version of boost can be used as we use only headers

#component for shared library version
PID_Wrapper_Component(libyaml INCLUDES include SHARED_LINKS ${posix_LINK_OPTIONS} lib/libyaml-cpp
											EXPORT boost/boost-headers)

#component for static library version
PID_Wrapper_Component(libyaml-st INCLUDES include SHARED_LINKS ${posix_LINK_OPTIONS} STATIC_LINKS lib/libyaml-cpp.a
																EXPORT boost/boost-headers)

Explanations:

  • PID_Wrapper_Version (equivalent signature is add_PID_Wrapper_Known_Version) is used to declare the new version (using VERSION keyword) and general information about it:
    • The VERSION must be the same as the name of the containing folder.
    • The DEPLOY is mandatory used to define the path to the cmake script file that implements the deploy procedure. The path is relative to the current version folder.
    • The SONAME may be used to define which version number is used in SONAME of shared objects generated by the external project.
  • Then platform configurations constraints are defined using PID_Wrapper_Configuration (equivalent to declare_PID_Wrapper_Platform_Configuration), exactly the same way as for native packages.
  • Then dependencies to other external packages wrappers are defined using PID_Wrapper_Dependency (equivalent to declare_PID_Wrapper_External_Dependency), the same way as for native packages.
  • Then components generated by the external project are described using PID_Wrapper_Component:
    • COMPONENT keyword defines the PID name of the component
    • INCLUDES keyword lists the include folders (if any) containing public headers of the component.
      • SHARED_LINKS and/or STATIC_LINKS list the binaries and linker options of a library. Many binaries can be listed. All path used are either absolute (starting with a /) or relative to the external package version install folder.
      • dependencies for these components are defined with DEPEND or EXPORT keywords. In this example yaml-cpp requires boost-headers component provided by the boost wrapper.

Then you need at least to define a deployment script, as explained in PID_Wrapper_Version. These deployment script are really scpecific to the considered external project being wrapped in PID. Here is an example of such a script for yaml-cpp wrapper version 0.5.1:

# all platform related variables are passed to the script
# TARGET_BUILD_DIR is provided by default (where the build of the external package takes place)
# TARGET_INSTALL_DIR is also provided by default (where the external package is installed after build)

if(NOT EXISTS ${TARGET_BUILD_DIR}/yaml_0.5.1.tar.gz)
  file(DOWNLOAD https://github.com/jbeder/yaml-cpp/archive/release-0.5.1.tar.gz
  ${TARGET_BUILD_DIR}/yaml_0.5.1.tar.gz SHOW_PROGRESS)
endif()

if(NOT EXISTS ${TARGET_BUILD_DIR}/yaml_0.5.1.tar.gz)
  message("[PID] ERROR : during deployment of yaml-cpp version 0.5.1, cannot download the archive.")
  set(ERROR_IN_SCRIPT TRUE)
  return()
endif()

if(EXISTS ${TARGET_BUILD_DIR}/yaml-cpp-release-0.5.1)
  file(REMOVE_RECURSE ${TARGET_BUILD_DIR}/yaml-cpp-release-0.5.1)
endif()

execute_process(
  COMMAND ${CMAKE_COMMAND} -E tar xf yaml_0.5.1.tar.gz
  WORKING_DIRECTORY ${TARGET_BUILD_DIR}
)

if(NOT EXISTS ${TARGET_BUILD_DIR}/yaml-cpp-release-0.5.1 OR NOT IS_DIRECTORY ${TARGET_BUILD_DIR}/yaml-cpp-release-0.5.1)
  message("[PID] ERROR : during deployment of yaml-cpp version 0.5.1, cannot extract the archive.")
  set(ERROR_IN_SCRIPT TRUE)
  return()
endif()

#specific work to do
set(YAML_BUILD_DIR ${TARGET_BUILD_DIR}/yaml-cpp-release-0.5.1/build)
file(MAKE_DIRECTORY ${YAML_BUILD_DIR})#create the build dir

get_External_Dependencies_Info(INCLUDES all_includes)

#first pass build the shared library
execute_process(COMMAND ${CMAKE_COMMAND} -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=${TARGET_INSTALL_DIR} -DBoost_INCLUDE_DIR=${all_includes} ..
  WORKING_DIRECTORY ${YAML_BUILD_DIR})
execute_process(COMMAND ${CMAKE_MAKE_PROGRAM} WORKING_DIRECTORY ${YAML_BUILD_DIR})#build
execute_process(COMMAND ${CMAKE_MAKE_PROGRAM} install WORKING_DIRECTORY ${YAML_BUILD_DIR})#install
#second pass build the static library
execute_process(COMMAND ${CMAKE_COMMAND} -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX=${TARGET_INSTALL_DIR} -DBoost_INCLUDE_DIR=${all_includes} ..
  WORKING_DIRECTORY ${YAML_BUILD_DIR})
execute_process(COMMAND ${CMAKE_MAKE_PROGRAM} WORKING_DIRECTORY ${YAML_BUILD_DIR})#build
execute_process(COMMAND ${CMAKE_MAKE_PROGRAM} install WORKING_DIRECTORY ${YAML_BUILD_DIR})#install


if(NOT EXISTS ${TARGET_INSTALL_DIR}/lib OR NOT EXISTS ${TARGET_INSTALL_DIR}/include)
  message("[PID] ERROR : during deployment of yaml-cpp version 0.5.1, cannot install yaml-cpp in worskpace.")
  set(ERROR_IN_SCRIPT TRUE)
endif()

Explanations:

The build process consists in first downloading the archive containing source files (using file(DOWNLOAD ...) cmake command). It then extracts the source file from the archive using execute_process(COMMAND ${CMAKE_COMMAND} -E tar xf ...) command. and finally launch the build process of the external project:

  • execute_process(COMMAND ${CMAKE_COMMAND} ... configure the external project (that is a cmake project).
  • execute_process(COMMAND ${CMAKE_MAKE_PROGRAM} ... compiles the project.
  • execute_process(COMMAND ${CMAKE_MAKE_PROGRAM} install installs the project into the workspace (due to the use of -DCMAKE_INSTALL_PREFIX=${TARGET_INSTALL_DIR} argument at configuration time)

Anytime the deploy script fails, it has to set the variable ERROR_IN_SCRIPT to TRUE.

The yaml-cpp wrapper needs boost to be compiled, that is why we use the get_External_Dependencies_Info to get all require include folder of boost wrapper (we know that we only need headers of boost that is why we only require the include folders).

For other external package wrapper one needs to follow at best the same procedure.

Wrapper build process control

PID wrappers provide a set of cache variables that are used to control their build process. The configuration of these CMake cache variables is basically made using ccmake .. command in the build directory of the wrapper or by using other Cmake configuration GUI. Contrarily to native packages, wrapper do not provide default options, but they can define user specific options using dedicated API in the root CMakeLists.txt:

define_PID_Wrapper_User_Option(
	OPTION BUILD_WITH_CUDA_SUPPORT
	TYPE BOOL
	DEFAULT OFF
	DESCRIPTION "set to ON to enable CUDA support during build")

Explanations:

The wrapper defines an option called BUILD_WITH_CUDA_SUPPORT that is a boolean that default to off. This user option can be consulted in deploy script of versions in order to control the build process, this way:

... # cmake code used to download/get the external project source files
get_User_Option_Info(OPTION BUILD_WITH_CUDA_SUPPORT RESULT using_cuda) #using_cuda variable is set with the value of the option

if(using_cuda)
... # build using CUDA
else()
... # build without CUDA
endif()

Understanding the result

From the complete build/install process an “installed version” of the wrapper is generated and put into the adequate folder of the workspace. The install process is managed by PID system so developer should not worry about how it takes place.

The figure provides an example of the workspace’s external folder containing yaml-cpp installed external packages wrappers for version 0.5.1 for a given platform (x86_64_linux_abi11). There can be many versions of the same wrapper installed in the workspace for the same platform.

Each binary package version folder content is specific to the external project, but there is always a cmake use file (a file generated and used by PID to get information about the installed wrapper) is the share folder.

Generally speaking users should never change the content of any of those folders “by hand”, otherwise it could cause troubles.