Basic concepts

The workspace is the place where developpers work with deployment units (either those they develop or those they simply use).

Deployment units

Deployment units is the generic term to denote what can be put into workspace by users. Each deployment unit respects unity and deployability:

  • unity denotes the fact that the whole content of the deployment unit is bound to the deployment unit itself and cannot be considered independently from the unit. In other words there is a unique lifecycle for all elements of the deployment unit.

  • deployability denotes the fact that the unit can be referenced into workspace (and so can be found online) and can be put into a user local workspace.

There are currenlty four types of deployment units:

  • packages are used to develop, reference, publish and deploy C/C++ projects. Concrete result of packages are binaries like libraries and executable.
  • wrappers are used to build and deploy external C/C++ projects (not directly developped within the PID environment) so that these later can be used as if they were native packages.
  • frameworks are used to document and reference a group of related packages. Concrete result of frameworks are static websites providing online documentation and online binaries repository for its packages’s binaries.
  • environments are used to configure the build environment in a workspace.

Platforms

Platforms are description of target execution environments for packages. A workspace may have many platforms defined but a workspace defines only one current platform at a given moment of packages development lifecycle.

Platforms is the basic concept used to manage compatiblity between packages binaries.

A platform is described with following information:

  • the target processor familly, for instance the most common architecture familly is x86.
  • the target processor architecture, for instance 32 or 64 bits are the most common values.
  • the target operating system, for instance linux or macosx.
  • the target ABI of the c++ compiler in use. Up to now, possible ABIs are CXX (the former ABI of C/C++) and CXX11 (the new ABI for C/C++).

Binaries generated for a given platform will not be compatible with another one generated for another platform. This rule is in fact true most of time but not every time. We decided to force incompatiblity between binaries generated for different platforms to avoid falling into a very complex decision process to know if they are compatible or not…

Environments

Environments are used to configure the workstation of users in order to provide new toolchains for the current platform (e.g. using clang toolchain instead of GNU toolchain) or to provide some cross compilation means (e.g. building packages for a arm 32 bits processor on a host with a x86 64 bits processor).

The current environment in use in the workspace defines the target platform for any build and deployment process. Practically, all binary packages built or deployed will be placed into the corresponding platform folders:

  • <pid-workspace>/install/<platform> folder for native packages.
  • <pid-workspace>/external/<platform> folder for external packages.

A workspace has a default current environment that is automatically deduced from current user’s workstation environment. For instance on an Ubuntu OS this is a GNU GCC toochain by default.

This current environment may be changed by the user on demand, based on available environment description (there are a few by default, but user can define new ones).

Environments are used to configure the build process of all packages contained in the workspace. The toolchain defined and also other configuration information, will be used by package to know how to build binaries.

When the user defines a current environment (or when it is deduced from user’s workstation configuration) the corresponding current platform is defined. Users have to notice that a change of the current environment may not change the current platform simply because different toolchains may be used to generate code for same execution platform.

Workspace structure

Technically, a workspace is a local folder hierarchy in users’ workstations. In this folder hierarchy packages are developed, installed and referenced given a predetermined pattern : they are put into specific subfolders. As compared to classical do as you wish organization, the workspace way of doing as several advantages:

  • there is a reference path in the deploy process in such a way that classifying and finding software artefacts is made easy.
  • there is no need to use system dependencies intensively for developed packages. As far as possible, every dependency should be satisfied by workspace content, except for system dependencies that cannot reasonnably be packaged into workspace.
  • developers can handle many version of a package in the same build/execution environment, without risking to break OS dependencies. Furthermore they can used multiple versions of the same package in the same time.

CMake project

A workspace is a specific CMake project. It consists in a root folder (preferably named pid-workspace) with the following internal structure:

  • the .git folder contains version control related information, managed by the git tool.
  • the CMakeLists.txt file is the workspace configuration file from which workspace specific commands will be generated.
  • the Use_PID.cmake file is cmake script to include when using a PID workspace from an external project.
  • the install folder contains packages’ binaries installed by the user.
  • the packages folder contains all the packages developed by an author. Each sub-folder of packages is a local repository of a given package.
  • the external folder contains all the external packages used by the installed package (source or binary).
  • the wrappers folder contains all the wrapper developed/used. Each sub-folder of wrappers is a local repository of a given external project wrapper.
  • the environments folder contains the definitions of specific development environments that can be used by the workspace to configure package build process.
  • the configurations folder contains the definitions of platform configurations.
  • the sites folder contains the definitions of static sites used to deploy framework and packages documentation and binaries.
  • the .gitignore file is used to exclude install, packages and external folders content from version control. This is mandatory since these folders content is supposed to be purely local in a user workstation.
  • the share folder contains important files used to manage packages.
    • The docs sub-folder contains documentation about PID
    • the patterns sub-folder contains pattern files and folder used to generate most of general information on the packages and frameworks
    • the cmake folder contains all scripts and data required to describe, build, install, deploy and configure packages:
      • the system sub-folder contains cmake modules implementing the core PID functionnalities
      • the find sub-folder contains cmake find modules for commonly used external packages
      • the references sub-folder contains cmake modules containing meta-information about available packages
      • the licenses contains cmake modules containing available license description
      • the plugins contains PID plugins description. Each subfolder defines a plugin that may be used to customize PID behavior.
      • the platforms contains cmake scripts used to deduce target platform currenlty used in the workspace.
  • the pid folder is a kind of build directory for the workspace. This is where the user configures the workspace and launch workspace specific commands.

Git repository

The workspace is also a git repository. It so exists under different state in users workstations and online. The worskpace lying in a user workstation simply reflects the user own development environment while online workspace are used to share information between users (well like any online git repositories).

There are different workspaces we quickly classify them here:

  • the official workspace: It contains an up to date official PID API on the master branch. It also contains work in progress on the PID API (unstable) on the development branch. Finally it contains up-to-date references to all known packages. By official packages we mean those that have been provided to the community by developpers.

  • private workspaces: these are git project forked from the official workspace project. As everyone has not write access to the official repository, the best way to do is to fork the official workspace using the gitlab dedicated functionnality (use the fork button). This allows anyone (even groups of people) to have its own workspace that can be update to save information about packages. Then when developers want to publish these packages (when they want all people to know about them) they simply have to merge their workspace’s master branch with the development branch of the official workspace by creating a merge request using gitlab dedicated functionality.

A workspace contains all the required information to find and use packages. This information must be updated from time to time when a package is significantly modified or added/removed. The meta-data contained in the workspace is just a set of files, more precisely cmake modules files (see cmake). A commit in the workspace consist in updating the content of some specific files/folders of the workspace.

There are different cmake modules files one can found in the workspace:

  • find modules are cmake scripts that follow the CMake standard way of finding dependencies. Each PID package (either native or external) should have a corresponding find file registered in the workspace so that anyone would be able to find them if they are installed locally.
  • reference modules are cmake scripts used to identify all available packages (git repositories and binary archives repositories) and frameworks (repositories) and keep track of information used to retrieve them. These modules are specific to PID.
  • license modules are simple cmake scripts used to provide available licenses description.
  • configuration modules are cmake script used to check that the target platform supports a given configuration and configure the projects adequately.
  • environment modules are cmake script used to used to describe environments.

Each time a new package is created it is referenced using a reference module into the workspace local git repository of the user. Then this later can commit his modifications and push them (or ask for a merge request) to the official workspace repository. Once done, anyone can know the existence of this new package and can retrieve its hosting address. This does not mean that this person can use binary version or repository of this package, since this is bound to his access rights to the servers (at least one can know which person to call to get the rights).

Remark: The official repository of the workspace can be forked into private repositories, exactly the same way as explained in git workflow, to provide a common workspace for a team developing one or more packages at the same time. Private workspaces will be updated with new reference and find modules for these new packages. When package are ready to be more widely used, a merge request coming from private repository will precedes the final merge in the official repository that will lead to the publishing of these packages for everyone.

Installed packages

The install folder contains installed binary version(s) of any package defined natively with PID (we call them native packages). Each of its direct sub directories is a folder representing a given platform. For instance there may have a x86_64_linux_abi11 subfolder and also a x86_32_linux_abi98 subfolder if the packages have been built for these two platforms. Then each of these subfolders directly contains a folder describing all packages for which there is a binary version. These (subsub-)folders simply take the name of the package and themlselves contain:

  • as many folders as there are concurrent binary versions of the package installed (for the current platform). The name of the version folder reflects the installed package version it contains (e.g. : 1.3.5 ; 1.2.0, etc.).
  • an installers folder that contains all installable binary archives with each archive matching a specific version in a specific build mode (release, debug).

Each version folder is organized according to the following structure:

  • the bin folder contains all executables provided by the package, except tests, in both debug and release modes. In other words it contains the result of the compilation of its corresponding package repository apps folder.
  • the include folder contains the exported interfaces of libraries. Basically, its direct sub-folders are libraries’ root folder hierarchically organized the same way as in the package repository include folder.
  • the lib folder contains libraries provided by the package in both debug and release modes: it contains the result of the compilation of its corresponding package repository src folder.
  • the .rpath folder contains a subfolder for each installed runtime component (shared or module libraries, executables). Each of these subfolders contains the symlinks resolving the binary relocation.
  • the share folder contains documents and scripts that are useful to the package users:
    • the Use<Package><Version>.cmake file is a specific cmake module used to identify components and dependencies defined by the installed package version. This file is generated automatically by native PID packages.
    • doc sub-folder contains API documentation generated by doxygen
    • cmake sub-folder contains cmake modules files that are required to use the package, like find scripts used to configure external packages
    • resources sub-folder contains installed elements contained in the corresponding resources subfolder of the package repository share folder
    • ci sub-folder contains bash scripts used for by the continuous integration system.
    • script sub-folder contains a subfolder for each installed python module. Each of these subfolders contains the symlinks resolving the binary relocation specifically required by python.
  • license.txt file describes the license that applies to the software. This is a copy of the license file in package repository.

The install folder may also contain some folders for one or more python version. If no python version is found on your host platform then no such folder will appear. Each python version folder has the form python<major>.<minor> and contains a set of symlinks pointing to python modules lying in the install folder of some packages (those that define python modules). The main utility of such folder is to simplify the boring task of setting the PYTHONPATH for individual packages.

For external packages the organization inside external folder is quite the same: each of its direct sub directories is a folder representing a given platform that itself contains folders containing external packages, that itself contains as many folders as there are concurrent binary versions of the package installed. Each version folder in turn follows at best the pattern explained previously for native packages.