Build workflows

This page is an overview of the workflows devised to build the software stack (compile, package, version, document, etc.).

Azure pipelines for streamflow forecasting

Building installers

Not counting third party established libraries such as Boost, a dozen git repositories under our control are used as inputs to the pipelines that package the software stack. Maintaining versions within each of these repositories may be influenced by various factors including unrelated projects, not necessarily solely by these installers. We are using the commit hashsums of this dozen of repositories to record and specify which version of each repositories is used as input to the build pipelines (see Fig. 1)

flowchart TD
  A[code freeze each components see Fig. 2] --> D[/git hashsums in sf-stack repo/] 
  D --> E[Linux build]
  E --> H[/.deb, python wheels \nand R tarballs/] 
  D --> F[MacOSX build]
  H-- R tarballs --> G[Win build]

  G -->I[/.dll, R binary \ninstallers for windows/] 
  F -->J[/.dylib, homebrew packages/] 

  H --> K[Fetch artifacts]
  I --> K
  J --> K
  K --> L[copy to shared drive]

  click D "https://bitbucket.csiro.au/projects/SF/repos/sf-stack/browse" _blank
  click K "https://bitbucket.csiro.au/projects/SF/repos/cruise-control/browse/doc/build_pipelines.md" _blank

Figure 1: Package build workflow.

The source code for Azure Devops pipelines are also as of 2023-01 from github, for information purposes only. Their public availability may change in the future though.

Versioning

This is a (partly populated) placeholder section to document the rationale for versioning installer bundles, and components at a final granularity.

There may be a versioning, semantic or otherwise, used for the bundles of installers produced for various platforms (Fig. 1). The major product is still what used to be refered to as SWIFT2, but there are also other important, related but distinct, products (FoGSS, CHyPP) also.

flowchart TD
  A[Testing branch] --> B{testing}
  B -->|Yes| C[git tag]
  B -->|No| A
  C -->D[update CMakeLists.txt versions] 
  D --> E[update debian changelog version]
  E --> F[update R package versions in\nDESCRIPTION and R doxygen]
  E --> G[Update python _version.py file]
  F --> H[roxygenise R packages]
  H --> I[commit new or updated .Rd files]
  I --> J[commit all repo changes]
  G --> J
  J --> K[tag repo]
  K --> L[git push]

Figure 2: Versioning components.

Channels for software distribution

Some software such as the refcount python package are available via conda-forge and pypi. We cannot use these mechanism for closed source packages, but we may consider in the future distribution via:

“private” conda channel
“private” pypi repository
“private” debian package repository
“private” homebrew package repository (MacOSX)