Basic concepts

The workspace is the place where developpers work with deployment units (either those they develop or those they simply use).

Deployment units

Deployment units is the generic term to denote what can be put into workspace by users. Each deployment unit respects unity and deployability:

  • unity denotes the fact that the whole content of the deployment unit is bound to the deployment unit itself and cannot be considered independently from the unit. In other words there is a unique lifecycle for all elements of the deployment unit.

  • deployability denotes the fact that the unit can be referenced into workspace (and so can be found online) and can be put into a user local workspace.

There are currenlty four types of deployment units:

  • packages are used to develop, publish and deploy C/C++ projects. Concrete result of packages are binaries like libraries and executable.
  • wrappers are used to build and deploy external C/C++ projects (not directly developped within the PID methodology) so that these later can be used as if they were native packages.
  • frameworks are used to document and reference a group of related packages. Concrete result of a framework is a static website providing online documentation and online binaries repository for its packages’s binaries.
  • environments are used to configure the build environment in a workspace.

Contribution spaces

Contribution spaces are marketplaces where user can share their contributions. This is the place where knowledge on available deployment units is stored.

A workspace can contain one or more contribution spaces, whose content is used to deploy and configure packages, wrappers, frameworks and environments. To know more about contribution space, please have a look at this page.

Platforms

Platforms are description of target execution environments for packages. A workspace can have many platforms defined but a workspace defines only one current platform at a given moment of packages development lifecycle.

Platforms is the basic concept used to manage compatiblity between packages binaries.

A platform is described with following information:

  • the target processor familly, for instance the most common architecture familly is x86 but you can also use the name arm to target arm processors.
  • the target processor architecture, for instance 32 or 64 bits are the most common values.
  • the target operating system, for instance linux or macos.
  • the target C++ ABI. This reflects the ABI of the c++ standard library in use. Most of time there is only one ABI type per standard library, for instance c++ standard library has one ABI. The stdc++ has dual ABI support whose usage can be selected using a specific compilation flag. So possible values of ABI for this library is stdc++11 for using the new ABI or stdc++ for using former ABI.

Binaries generated for a given platform will not be compatible with another one generated for another platform. This rule is in fact true most of time but not every time. We decided to force incompatiblity between binaries generated for different platforms to avoid falling into a very complex decision process to know if they are compatible or not…

Technically, a platform is a specific folder into install folder of the workspace, for instance a folder named x86_64_linux_stdc++11. Practically, all binary packages built or deployed for this platform will be placed into the corresponding folder: <pid-workspace>/install/x86_64_linux_stdc++11.

Profiles

Profiles are used to configure the workspace with a set of tools. A profile defines default /alternative toolchains for supported languages and/or additional behaviors for using tools that are not supported by default in PID. By default, PID generates a default profile based on current detected features of the user’s workstation and this profile is the current profile. Profile are used to memorize various possible configurations of your workspace, but there is only one profile, called current profile that is used at a time. What is important to notice is that the current profile undirectly specifies for which target platform the code will be built.

For instance, first time you configure your workspace:

cd <pid-workspace>
pid configure

You should see an output looking like:

...
[PID] INFO: using default profile, based on host native development environment.
[PID] INFO : ASM language available with gcc toolchain.
[PID] INFO : C language available with gcc toolchain (version 9.3.0).
[PID] INFO : C++ language available with gcc toolchain (version 9.3.0).
[PID] INFO : Python language available (version 2.7). To use python modules installed in workspace please set the PYTHONPATH to =/home/robin/soft/PID/pid-workspace/install/x86_64_linux_stdc++11/__python2.7__
[PID] INFO : CUDA language available (version 11.0). Building for architecture 6.1.
[PID] INFO : Fortran language available with GNU toolchain (version 5.5.0).

[PID] INFO : Target platform in use is x86_64_linux_stdc++11:
 + processor family = x86 (optimizations: SSE, SSE2, SSE3, SSSE3, SSE4_1, POPCNT, SSE4_2, FP16, FMA3, AVX, AVX2, AVX_512F, AVX512_SKX)
 + binary architecture= 64
 + operating system=linux (ubuntu 16.04, apt packaging)
 + C++ ABI= stdc++11

The first line indicates which profile you are currenlty using, at first configuration it is the default profile. From this current profile, PID evaluates the target platform you are building for. Here the target platform is x86_64_linux_stdc++11. PID also provides some more precise informations about target platform like possible optimizations for the target processor or distribution of the target operating system, if it can detect them.

A profile is described using environments. Environments are used to configure the user’s workstation in order to provide new language toolchains (e.g. using clang toolchain), to provide some cross compilation means (e.g. building packages for an arm 32 bits processor on a host with a x86 64 bits processor) or even to provide new behaviors implementing the usage of additional tools (kind of plugins to augment PID capabilities). Anytime a profile is set to become the current one, the workspace is reconfigured with information coming from environments this profile uses. This information will be automatically transmitted to packages so that, finally, toolchains and other configuration information will be used by packages and wrappers to build binaries: this process ensures that they will generate code for same target platform.

Users have to notice that a change of the current environment may not change the target platform simply because different toolchains may be used to generate code for same execution platform.

Workspace structure

Technically, a workspace is a local folder hierarchy in users’ workstations. In this folder hierarchy packages are developed, installed and referenced given a predetermined pattern : they are put into specific subfolders. As compared to a classical do as you wish organization, the workspace way of doing as several advantages:

  • there is a reference path in the deploy process in such a way that classifying and finding software artefacts is made easy.
  • there is no need to use system dependencies intensively. As far as possible, every dependency should be satisfied by workspace content, except for system dependencies that cannot reasonnably be packaged into workspace.
  • developers can handle many version of a package in the same build/execution environment, without risking to break OS dependencies. Furthermore they can used multiple versions of the same package (but not for same build).

CMake project

A workspace is a specific CMake project. It consists in a root folder (named pid-workspace) with the following internal structure:

  • the .git folder contains version control related information, managed by the git tool.
  • the .gitignore file is used to exclude install, packages, wrappers, environments, sites and contribution folders content from version control. This is mandatory since these folders content is supposed to be purely local in a user workstation.
  • the CMakeLists.txt file is the workspace configuration file from which workspace specific commands will be generated.
  • the Use_PID.cmake file is cmake script to include when using a PID workspace from an external project.
  • the install folder contains packages’ binaries installed by the user.
  • the packages folder contains all the packages developed by an author. Each sub-folder of packages is a local repository of a given package.
  • the wrappers folder contains all the wrapper developed/used. Each sub-folder of wrappers is a local repository of a given external project wrapper.
  • the environments folder contains the definitions of specific development environments that can be used by the workspace to configure packages build process. Il also contains the file profiles_list.cmake that contains the description of all profiles currenlty defined in workspace.
  • the sites folder contains the definitions of static sites used to deploy frameworks (in frameworks subfolder) and packages (in packages subfolder) documentation and binaries.
  • the contributions folder contains the contribution spaces in use. It also contain the file contribution_spaces_list.cmake that define all available contribution spaces and their priorities.
  • the files README.md, CONTRIBUTING.md and license.txt are used for pid-workspace project documentation.
  • the cmake folder contains PID system APIs.
    • The .docs sub-folder contains documentation about PID APIs
    • the api sub-folder contains cmake modules implementing the core PID functionnalities.
    • the commands sub-folder contains cmake scripts implementing PID commands.
    • the patterns sub-folder contains pattern files and folder used to generate most of general information on the packages and frameworks.
    • the platforms contains cmake scripts used to deduce target platform currenlty used in the workspace.
  • the build folder is the build directory for the workspace that contains all workspace generated file used for global configuration of projects.

Git repository

The workspace is also a git repository. It exists under different state in users workstations and online. The worskpace lying in a user workstation simply reflects the user own development environment while online workspace is used to provide PID system (well like any online git repositories).

There are different workspaces we quickly classify them here:

  • the official workspace: It contains an up to date official PID APIs and command on the master branch. It also contains work in progress on the PID API (unstable) on the integration branch.

  • private workspaces: these are git project forked from the official workspace project. As everyone has not write access to the official repository, the best way to do is to fork the official workspace using git hosting service dedicated functionnality (use the fork functionaity in github and gitlab). This allows anyone, even groups of people, to have their own pid-workspace in which they can update the PID system and propose their changes to the official workspace by creating a merge request of the git hosting service they use.

  • local workspace: this is a clone of official or a private workspace of a user workstation. This is finally a folder where the user develops its C/C++ projects.

Source projects

All source projects are deployed into specific folders of the local workspace: they do not exist in the workspace repository.

  • All deployment units’ source projects are directly deployed in: packages, wrappers, environments, sites/packages and sites/frameworks depending on their nature. But each deployment units’ source project has a distinct lifecycle than the pid-workspace project.

Each direct subfolder of packages contains a git repository for a given native package, each direct subfolder of wrappers is a repository for a given external package wrapper, and so on.

  • Contribution spaces projects are directly cloned into the contributions folder. They also have a distinct lifecycle than the pid-workspace project.

Installed packages

The install folder contains installed binary version(s) of packages (either externa or native packages. Each of its direct sub directories is a folder representing a given target platform. For instance there may have a x86_64_linux_stdc++11 subfolder and also a x86_32_linux_stdc++ subfolder if the packages have been built for these two platforms. Then each of these subfolders directly contains:

  • a __system__ folder that contains scripts used to check target platform configuration.
  • other folders specifying target platform related features. Those folders follow the pattern __name__ where name is a specific feature of the platform. For instance if target platform supports Python 2.7 you will find a folder __python2.7__ that itself contains symlinks to all python packages defined in binary packages. It contains a set of symlinks pointing to python modules defined by some binary packages. The main utility of such folder is to simplify the boring task of setting the PYTHONPATH for individual packages. Some environment may places their generated artefact into a specific folder, for instance the pkg-config environment generates configuration files into __pkg-config__ folder.

  • as many folders as there are binary packages available for the target platform. Such a folder simply takes the name of the corresponding package and itself contains:

    • as many folders as they are installed versions of the binary package. The name of the version folder reflects the installed package version (e.g. : 1.3.5, 0.2.0, etc.).
    • an installers folder that contains all installable binary archives with each archive matching a specific version in a specific build mode (release, debug).

For native packages each version folder is organized according to the following structure:

  • the bin folder contains all executables provided by the package, except tests, in both debug and release modes. In other words it contains the result of the compilation of its corresponding package repository apps folder.
  • the include folder contains the exported interfaces of libraries. Basically, its direct sub-folders are libraries’ root folder hierarchically organized the same way as in the package repository include folder.
  • the lib folder contains libraries provided by the package in both debug and release modes: it contains the result of the compilation of its corresponding package repository src folder.
  • the .rpath folder contains a subfolder for each installed runtime component (shared or module libraries, executables). Each of these subfolders contains the symlinks resolving the binary relocation.
  • the share folder contains documents and scripts that are useful to the package users:
    • the Use<Package><Version>.cmake file is a specific cmake module used to identify components and dependencies defined by the installed package version. This file is generated automatically by native PID packages.
    • doc sub-folder contains API documentation generated by doxygen
    • cmake sub-folder contains cmake modules files that are required to use the package, like find scripts used to configure external packages
    • resources sub-folder contains installed elements contained in the corresponding resources subfolder of the package repository share folder
    • ci sub-folder contains bash scripts used for by the continuous integration system.
    • script sub-folder contains a subfolder for each installed python module. Each of these subfolders contains the symlinks resolving the binary relocation specifically required by python.
  • license.txt file describes the license that applies to the software. This is a copy of the license file in package repository.

Organization inside external packages is mostly the same than for native packages, except :

  • there is no .rpath folder.
  • the share folder is organized according to what is proposed by the external project, which may vary a bit from one external package to another. There is always a Use<Package><Version>.cmake file generated by PID.