workspace is the place where developpers work with
deployment units (either those they develop or those they simply use).
Deployment units is the generic term to denote
what can be put into workspace by users. Each deployment unit respects
unitydenotes the fact that the whole content of the deployment unit is bound to the deployment unit itself and cannot be considered independently from the unit. In other words there is a
unique lifecyclefor all elements of the deployment unit.
deployabilitydenotes the fact that the unit can be referenced into workspace (and so can be found online) and can be put into a user local workspace.
There are currenlty four types of deployment units:
packagesare used to develop, reference, publish and deploy C/C++ projects. Concrete result of packages are
binarieslike libraries and executable.
wrappersare used to build and deploy external C/C++ projects (not directly developped within the PID environment) so that these later can be used as if they were native packages.
frameworksare used to document and reference a group of related packages. Concrete result of frameworks are static websites providing online documentation and online binaries repository for its packages’s binaries.
environmentsare used to configure the build environment in a workspace.
Platforms are description of target execution environments for packages. A workspace may have many platforms defined but a workspace defines only one current platform at a given moment of packages development lifecycle.
Platforms is the basic concept used to manage compatiblity between packages binaries.
A platform is described with following information:
- the target processor familly, for instance the most common architecture familly is
- the target processor architecture, for instance 32 or 64 bits are the most common values.
- the target operating system, for instance linux or macosx.
- the target ABI of the c++ compiler in use. Up to now, possible ABIs are
CXX(the former ABI of C/C++) and
CXX11(the new ABI for C/C++).
Binaries generated for a given platform will not be compatible with another one generated for another platform. This rule is in fact true most of time but not every time. We decided to force incompatiblity between binaries generated for different platforms to avoid falling into a very complex decision process to know if they are compatible or not…
Environments are used to configure the workstation of users in order to provide new toolchains for the current platform (e.g. using clang toolchain instead of GNU toolchain) or to provide some cross compilation means (e.g. building packages for a arm 32 bits processor on a host with a x86 64 bits processor).
The current environment in use in the workspace defines the target platform for any build and deployment process. Practically, all binary packages built or deployed will be placed into the corresponding platform folders:
<pid-workspace>/install/<platform>folder for native packages.
<pid-workspace>/external/<platform>folder for external packages.
A workspace has a default current environment that is automatically deduced from current user’s workstation environment. For instance on an Ubuntu OS this is a GNU GCC toochain by default.
This current environment may be changed by the user on demand, based on available environment description (there are a few by default, but user can define new ones).
Environments are used to configure the build process of all packages contained in the workspace. The toolchain defined and also other configuration information, will be used by package to know how to build binaries.
When the user defines a current environment (or when it is deduced from user’s workstation configuration) the corresponding current platform is defined. Users have to notice that a change of the current environment may not change the current platform simply because different toolchains may be used to generate code for same execution platform.
Technically, a workspace is a local folder hierarchy in users’ workstations. In this folder hierarchy packages are developed, installed and referenced given a predetermined pattern : they are put into specific subfolders. As compared to classical do as you wish organization, the workspace way of doing as several advantages:
- there is a reference path in the deploy process in such a way that classifying and finding software artefacts is made easy.
- there is no need to use system dependencies intensively for developed packages. As far as possible, every dependency should be satisfied by workspace content, except for system dependencies that cannot reasonnably be packaged into workspace.
- developers can handle many version of a package in the same build/execution environment, without risking to break OS dependencies. Furthermore they can used multiple versions of the same package in the same time.
A workspace is a specific CMake project. It consists in a root folder (preferably named
pid-workspace) with the following internal structure:
.gitfolder contains version control related information, managed by the git tool.
CMakeLists.txtfile is the workspace configuration file from which workspace specific commands will be generated.
Use_PID.cmakefile is cmake script to include when using a PID workspace from an external project.
installfolder contains packages’ binaries installed by the user.
packagesfolder contains all the packages developed by an author. Each sub-folder of
packagesis a local repository of a given package.
externalfolder contains all the external packages used by the installed package (source or binary).
wrappersfolder contains all the wrapper developed/used. Each sub-folder of
wrappersis a local repository of a given external project wrapper.
environmentsfolder contains the definitions of specific development environments that can be used by the workspace to configure package build process.
configurationsfolder contains the definitions of platform configurations.
sitesfolder contains the definitions of static sites used to deploy framework and packages documentation and binaries.
.gitignorefile is used to exclude
externalfolders content from version control. This is mandatory since these folders content is supposed to be purely local in a user workstation.
sharefolder contains important files used to manage packages.
docssub-folder contains documentation about PID
patternssub-folder contains pattern files and folder used to generate most of general information on the packages and frameworks
cmakefolder contains all scripts and data required to describe, build, install, deploy and configure packages:
systemsub-folder contains cmake modules implementing the core PID functionnalities
findsub-folder contains cmake find modules for commonly used external packages
referencessub-folder contains cmake modules containing meta-information about available packages
licensescontains cmake modules containing available license description
pluginscontains PID plugins description. Each subfolder defines a plugin that may be used to customize PID behavior.
platformscontains cmake scripts used to deduce target platform currenlty used in the workspace.
pidfolder is a kind of build directory for the workspace. This is where the user configures the workspace and launch workspace specific commands.
The workspace is also a git repository. It so exists under different state in users workstations and online. The worskpace lying in a user workstation simply reflects the user own development environment while online workspace are used to share information between users (well like any online git repositories).
There are different workspaces we quickly classify them here:
official workspace: It contains an up to date official PID API on the master branch. It also contains work in progress on the PID API (unstable) on the development branch. Finally it contains up-to-date references to all known packages. By official packages we mean those that have been provided to the community by developpers.
private workspaces: these are git project forked from the official workspace project. As everyone has not write access to the official repository, the best way to do is to fork the official workspace using the gitlab dedicated functionnality (use the fork button). This allows anyone (even groups of people) to have its own workspace that can be update to save information about packages. Then when developers want to publish these packages (when they want all people to know about them) they simply have to merge their workspace’s master branch with the development branch of the official workspace by creating a merge request using gitlab dedicated functionality.
A workspace contains all the required information to find and use packages. This information must be updated from time to time when a package is significantly modified or added/removed. The meta-data contained in the workspace is just a set of files, more precisely cmake modules files (see cmake). A commit in the workspace consist in updating the content of some specific files/folders of the workspace.
There are different cmake modules files one can found in the workspace:
- find modules are cmake scripts that follow the CMake standard way of finding dependencies. Each PID package (either native or external) should have a corresponding find file registered in the workspace so that anyone would be able to find them if they are installed locally.
- reference modules are cmake scripts used to identify all available packages (git repositories and binary archives repositories) and frameworks (repositories) and keep track of information used to retrieve them. These modules are specific to PID.
- license modules are simple cmake scripts used to provide available licenses description.
- configuration modules are cmake script used to check that the target platform supports a given configuration and configure the projects adequately.
- environment modules are cmake script used to used to describe
Each time a new package is created it is referenced using a reference module into the workspace local git repository of the user. Then this later can commit his modifications and push them (or ask for a merge request) to the
official workspace repository. Once done, anyone can know the existence of this new package and can retrieve its hosting address. This does not mean that this person can use binary version or repository of this package, since this is bound to his access rights to the servers (at least one can know which person to call to get the rights).
Remark: The official repository of the workspace can be forked into private repositories, exactly the same way as explained in git workflow, to provide a common workspace for a team developing one or more packages at the same time. Private workspaces will be updated with new reference and find modules for these new packages. When package are ready to be more widely used, a merge request coming from private repository will precedes the final merge in the official repository that will lead to the publishing of these packages for everyone.
install folder contains installed binary version(s) of any package defined natively with PID (we call them native packages). Each of its direct sub directories is a folder representing a given
platform. For instance there may have a
x86_64_linux_abi11 subfolder and also a
x86_32_linux_abi98 subfolder if the packages have been built for these two platforms. Then each of these subfolders directly contains a folder describing all packages for which there is a binary version. These (subsub-)folders simply take the name of the package and themlselves contain:
- as many folders as there are concurrent binary versions of the package installed (for the current platform). The name of the version folder reflects the installed package version it contains (e.g. : 1.3.5 ; 1.2.0, etc.).
installersfolder that contains all installable binary archives with each archive matching a specific version in a specific build mode (release, debug).
Each version folder is organized according to the following structure:
binfolder contains all executables provided by the package, except tests, in both debug and release modes. In other words it contains the result of the compilation of its corresponding package repository
includefolder contains the exported interfaces of libraries. Basically, its direct sub-folders are libraries’ root folder hierarchically organized the same way as in the package repository
libfolder contains libraries provided by the package in both debug and release modes: it contains the result of the compilation of its corresponding package repository
.rpathfolder contains a subfolder for each installed runtime component (shared or module libraries, executables). Each of these subfolders contains the symlinks resolving the binary relocation.
sharefolder contains documents and scripts that are useful to the package users:
Use<Package><Version>.cmakefile is a specific cmake module used to identify components and dependencies defined by the installed package version. This file is generated automatically by native PID packages.
docsub-folder contains API documentation generated by doxygen
cmakesub-folder contains cmake modules files that are required to use the package, like find scripts used to configure external packages
resourcessub-folder contains installed elements contained in the corresponding
resourcessubfolder of the package repository
cisub-folder contains bash scripts used for by the continuous integration system.
scriptsub-folder contains a subfolder for each installed python module. Each of these subfolders contains the symlinks resolving the binary relocation specifically required by python.
license.txtfile describes the license that applies to the software. This is a copy of the license file in package repository.
install folder may also contain some folders for one or more python version. If no python version is found on your host platform then no such folder will appear. Each python version folder has the form
python<major>.<minor> and contains a set of symlinks pointing to python modules lying in the install folder of some packages (those that define python modules). The main utility of such folder is to simplify the boring task of setting the
PYTHONPATH for individual packages.
For external packages the organization inside
external folder is quite the same: each of its direct sub directories is a folder representing a given platform that itself contains folders containing external packages, that itself contains as many folders as there are concurrent binary versions of the package installed. Each version folder in turn follows at best the pattern explained previously for native packages.