During our coding work, when we develop a project, we most of time need to use third party software. In open source world we so need to deploy, configure and build various projects. The problems you then immediately face are all related to heterogeneity:
- heterogeneity in filesystem organization.
- heterogeneity in project description. They have specific configuration, build and install procedures. Hopefully the
CMaketool is near to be a standard and it has contributed a lot to the standardization of project description but there is still a lot of hand written code for specifying configuration of complex projects.
- heterogeneity in the management of dependencies.
CMakebrought a better level of specification of dependencies (e.g. using
find_package) but the process of their resolution is still project dependent and hard to write for inexperienced users.
This heterogeneity leads to confusion and confusion leads to bugs…
Paradoxically, anyone that already worked on complex projects has noticed that large part of projects build process are really close to each other, which means we reinvent the wheel everytime.
A common build and deployment process, integrating all best pratices and automating most of the procedures would be a great plus. PID as been made for that : standardizing the C/C++ projects development process in order to limit their complexity and to automate the process as far as possible. PID is a methodology that helps improving the overall quality of the code and applications. By improving the quality we aim at:
- simplifying the understanding and the modification of code produced by others.
- simplifying the reuse of software components we develop.
- pooling software developments between development teams.
The remaining sections of this page explain the fundamentals of PID : tools used and core concepts.
PID stands for Packages Integral Development. The term “integral” is used to denote the fact that PID:
- aims to deal with all aspects of C/C++ projects development (i.e. completeness)
- proposes a process to integrate many parts (the packages) in order to get a final system (i.e. an integrative approach).
The following subsections explain fundamentals that need to be understood to start working with the PID methodology. As definition of concepts and process is intrinsically bound to the concepts and processes involved in the software tools used, the next section first presents these tools. Then we define core concepts of the PID methodology.
For each development task in a package, a tool is used to achieve this task. To ease the portability of code, only cross platforms tools are used:
- git is used for the concurrent version control and patching of the code. It is also the tool used to deliver and retrieve the source code. To know git please have a look here. git-lfs, an extension of git,is used to manage the versionning of large binary files.
- cmake is used to manage the build process, but also deployment and tests. To know more about CMake please refer to this wiki.
Other tools used, like the compiler/linker (e.g. gcc) used with cmake, or the ssh agent used with git, or even development environment (e.g. Xcode, Eclipse) interfaced with cmake, git are supposed to be native (i.e. specific to the OS used) and so adapted by the user depending on the platform he/she uses and his/her preferences.
Furthermore there are other tools that are mainly used during continuous integration process like:
- doxygen, used to generate HTML pages for source code’s API documentation.
- cppcheck, used to perfom static checks on the source code.
- clang-format, used to perfom source code formatting.
- gcov/lcov, used to generate tests coverage reports.
- jekyll, used to generate static sites of packages.
- gitlab, used to control overall CI process.
The package is the working unit for code development: it is in the same time a CMake C/C++ project and a git repository. It basically contains:
- the functional source code of the project (header and source files, configuration files, scripts).
- git version control information.
- CMake projects files used to : build the source code and the documentation, run tests, generate configuration files, install its resulting binaries and generate an installable relocatable archive.
A package provides some functional code that can be reused called components and depends on other packages either for its compilation (e.g. archive libraries, header files) or for its runtime use (dynamic libraries, executables, script-like files).
A package has two main different forms :
- a source form: it is a git repository that contains the package’s source code. It is something alive that is continuously evolving along development process.
- a binary form: it is a collection of software artifacts (headers, configuration files, executables, libraries, documentation, etc.) deployed in a given place of a file system. A binary package can be embedded into an archive in order to be easily retrieved and deployed on a file system.
In all this web site, we use the term package to say source form package. If we speak about the binary form of the package we use the term binary package.
A quite complete description of a package content is provided here.
The wrapper is the working unit for importing external projects into PID: it is in the same time a CMake project and a git repository. It basically contains:
- git version control information.
- CMake projects files used to : download, build and install the external project in workspace, generate relocatable binary archives.
A wrapper provides description of components generated by the external project as well as the procedure (written in CMake) used to deploy it. We so say that wrappers generate external packages. In PID, an external package is a binary package installed in workspace that has not been generated by a package but by a wrapper.
Description of wrappers content is provided here.
A framework represents a collection of packages (and so indirectly a collection of components). Frameworks are use to classify and organize projects that have a common purpose.
From technical point of view frameworks are used to unify documentation of these package and also provide a way to reference binary packages versions that have been released.
Description of frameworks content is provided here.
An environment represents a specific configuration of the host system.
From technical point of view environments are used to provide configurations of development tools used in host system. With PID environments configurations of host build tools become reusable and composable.
A description of a environments content is provided here.
The workspace is the place where developers work with packages, wrappers, frameworks and environments (either those they develop or those they simply use). Technically, workspaces are :
- local folder hierarchies where deployment units (packages/wrappers/frameworks/environments) are created, developed, installed, and referenced.
- a git repository that contains the implementation of PID APIs and commands.
As compared to classical “do as you wish” installation, the “workspace way of doing” as several advantages:
- there is a reference path in the deploy process in such a way that classifying and finding software artefacts is made easy.
- there is no more need to use system dependencies intensively for developed packages.
- developers can handle many versions of a package in the same build/execution environment, without risking to break OS dependencies.
A description of a workspace’s content is provided here.
A contribution space is a specific entity that is used to memorize a set of contributions provided by users. This is particularly usefull to memorize available packages, wrappers, frameworks and environments that users can deploy into their local workspace.
Technically a contribution space is a git repository containing specific CMake files managed by PID to reference any kind of user content that can be deployed.
Basically, using PID consists in:
- using commands with specific arguments in order to control the development process (for instance building a package).
- writing CMake code using PID APIs to document and process content of packages/wrappers/frameworks/environments in order to get resulting software artefacts like executables, libraries and documentation.
- writing C/C++ code.
A good start for working with PID is this tutorial for beginners.
A more detailed explanation on how to use PID is provided here. This let you understand the way to use the PID CMake API in order to define a package and then how to control its build process and finally explains where to find resulting executable code.