Investigating Debian source packages repository SCM plugin idea
A bit of context first: we're
in the process of migrating/consolidating the various CIs of several software
development teams into a single, modern Jenkins instance. One of the projects
to migrate is a Debian-based product with the following (highly simplified)
Build source code repositories,
publish source packages on Debian repository
Build Debian source package, publish binary
packages on Debian repository
Create product artifacts from Debian binary
The existing CI is
performing step 2 with an old, customized Open Build System (OBS) instance and we're
currently evaluating our options for moving forward, with a strong emphasis on
simplifying the whole process. One of the options considered would be to use
Jenkins for building these Debian source packages so that we’d have only one
automation server to worry about. However, we have ~60 Debian source packages
across multiple branches to build, so creating ad-hoc jobs for each wouldn’t scale.
Therefore, I have emitted the idea of using the Debian source packages of a
Debian repository as a SCM provider and let the MultiBranch plugin + external
Jenkinsfile take care of business.
Which brings me to the point (and the choice of the developers mailing list instead of the users for this post):
as far as I can tell, it appears this SCM plugin does not exist and no one
seems to have considered this approach before. Am I correct in this assessment?
Regarding the SCM plugin itself,
it would expose for a given distribution in a Debian repository each Debian
source package as a SCMHead, masking source packages whose build dependencies are not met. To properly handle the Debian package lifecycle,
SCMRevision would encode not only the source package version, but also the latest binary package versions of its immediate build dependencies for a given
architecture as well. We
are not concerned so far about repository GPG signing, we're fine with basic SCM polling for this particular task and we'll probably make the source package masking and binary dependency versions tracking optional to provide some flexibility.
I did give it a fair amount of
thought and note scribbling beyond this very succinct abstract and I believe this to be a workable design, as the normal Jenkins workflow combined with the
SCMHead/SCMRevision tricks as described above should be enough to iteratively keep
the binary packages of a Debian repository for a given architecture/distribution pair up to date. The feedback from
colleagues so far is positive once they wrapped their heads around it, but I haven’t briefed our local Debian expert
about this yet since he’s currently on vacation. Have I missed something
To be clear, we’re not looking
to recreate something as performant and feature-complete as Open Build System
or buildd that can handle the entire Debian repository without breaking a
sweat. We merely want to build our own private collection of Debian source
packages, preferably on the same automation server as the rest of our CI. The
only missing piece would be this hypothetical SCM plugin, which we’re currently
evaluating whether we should go ahead and write it, the rest being standard Jenkins setup/configuration stuff.
In short :
Does this Debian source packages repository SCM
plugin sounds like a good idea on a technical level?
Has it been done before and have I somehow missed it?
Have I missed something obvious that would
prevent this from working?