Bill of Materials

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Bill of Materials

Carlos Sanchez
Hi there, 

I have filed a new JEP to address the concept of Bill of Materials (BoM).


The BoM idea came up after different conversations with Tyler/KK/Oleg and several more people - thanks for the feedback! - talking about how can we define a Jenkins package ("classic" jenkins, essentials, custom) to ensure that latest master of core and plugins can be easily used and tested, as well as the too typical use case of PRs in progress across multiple modules.

The main goal is to increase velocity by providing developers automatic artifacts and the result of tests against those artifacts before these combinations of PRs are merged. In order to create some safety nets that encourage contributions and facilitate reviews.

Please share your feedback

Thanks!

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CALHFn6PNnYTpF-0mnr0_pee9yw0tYaX%3D%2B6akgBNp%3DXnMgToOrQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bill of Materials

R. Tyler Croy
(replies inline)

On Fri, 20 Apr 2018, Carlos Sanchez wrote:

> Hi there,
>
> I have filed a new JEP to address the concept of Bill of Materials (BoM).
>
> *https://github.com/jenkinsci/jep/pull/92
> <https://github.com/jenkinsci/jep/pull/92>*
>
> The BoM idea came up after different conversations with Tyler/KK/Oleg and
> several more people - thanks for the feedback! - talking about how can we
> define a Jenkins package ("classic" jenkins, essentials, custom) to ensure
> that latest master of core and plugins can be easily used and tested, as
> well as the too typical use case of PRs in progress across multiple modules.
>
> The main goal is to increase velocity by providing developers automatic
> artifacts and the result of tests against those artifacts before these
> combinations of PRs are merged. In order to create some safety nets that
> encourage contributions and facilitate reviews.

There have been a lot of comments on that pull request, but I wanted to bring
one aspect of the discussion back to the development mailing list.

As I have been working this past week with jglick on the incrementals
publishing, I'm recognizing that the yaml we sketched out previously requires
some logic to be re-implemented in a few different places.

Take the following snippet for example:

  plugins:
    - groupId: org.jenkins-ci.plugins
      artifactId: git
      ref: PR-1663 # or how are pull requests handled here?


Whether it's the Jenkins Essentials tooling, custom-war-packager, or any
test/maven tooling, something is going to have to translate those GAV
coordinates into an actual artifact which is either downloaded, or locally
referenced.

In an incrementals build, a pull-request built Git plugin artifact would actually be
located:
    https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/git/3.8.1-rc1663.454c4a5e4d30/git-3.8.1-rc1663.454c4a5e4d30.hpi

I'm not sure how I would build the tooling to use the above YAML to resolve to
the URL, which makes me wonder: why not just have the full artifact URL listed
in the Bill of Materials? For example:

  plugins:
    - groupId: org.jenkins-ci.plugins
      artifactId: git
      url: https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/git/3.8.1-rc1663.454c4a5e4d30/git-3.8.1-rc1663.454c4a5e4d30.hpi


From the Jenkins Essentials perspective, all I really need is an artifact ID
and URL for the artifact, but I'm not sure if that's all that's needed for the
other use-cases here.


Just thought I would raise the question since I don't feel like I understand
how to go from the Bill of Materials to binaries right now.



Cheers
-R Tyler Croy




--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180427165458.GG2935%40grape.lasagna.io.
For more options, visit https://groups.google.com/d/optout.

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bill of Materials

Jesse Glick-4
On Fri, Apr 27, 2018 at 12:54 PM, R. Tyler Croy <[hidden email]> wrote:

> why not just have the full artifact URL listed
> in the Bill of Materials? For example:
>
>   plugins:
>     - groupId: org.jenkins-ci.plugins
>       artifactId: git
>       url: https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/git/3.8.1-rc1663.454c4a5e4d30/git-3.8.1-rc1663.454c4a5e4d30.hpi
>
> From the Jenkins Essentials perspective, all I really need is an artifact ID
> and URL for the artifact, but I'm not sure if that's all that's needed for the
> other use-cases here.

You could do that, though a plain `version` attribute would suffice,
so long as the tool pulling binaries includes the Incrementals repo in
its list of locations. _If_ you wish to permit branch references or
local disk paths or other options in this format, then you would
anyway need some alternative ways of referring to binaries.

The more interesting question to my mind is how you make edits to the
manifest like the above. After you have more than a handful of plugins
with versions like these, manually copying and pasting such
URLs/versions would be cumbersome. I think JENKINS-50953 can be used
to track this. For example, I might run something like

$ update-incrementals configuration/essentials.yaml

or

$ update-incrementals configuration/essentials.yaml git workflow-api

which for each selected component (or all) would check the current
version; gather a list of all deployed versions (from
`maven_metadata.xml`) that compare as strictly “newer” than that;
filter those by commits which are ancestors of current `master`; pick
the newest of the resulting versions; and edit the file to select that
version.

You could then commit the result and push to a pull request, which
would get validated by a PR builder somehow TBD, and if all automated
checks pass it would be eligible for merging, meaning deployment to
production.

WDYT? Admittedly the above is a very rough sketch, which I need to
flesh out quite a bit.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1YDTcmNQmgp1yb7nE3MmO6uFiEY1%3DTaAkFcf_gMBqtRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bill of Materials

R. Tyler Croy
(replies inline)

On Mon, 30 Apr 2018, Jesse Glick wrote:

> On Fri, Apr 27, 2018 at 12:54 PM, R. Tyler Croy <[hidden email]> wrote:
> > why not just have the full artifact URL listed
> > in the Bill of Materials? For example:
> >
> >   plugins:
> >     - groupId: org.jenkins-ci.plugins
> >       artifactId: git
> >       url: https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/git/3.8.1-rc1663.454c4a5e4d30/git-3.8.1-rc1663.454c4a5e4d30.hpi
> >
> > From the Jenkins Essentials perspective, all I really need is an artifact ID
> > and URL for the artifact, but I'm not sure if that's all that's needed for the
> > other use-cases here.
>
> You could do that, though a plain `version` attribute would suffice,
> so long as the tool pulling binaries includes the Incrementals repo in
> its list of locations. _If_ you wish to permit branch references or
> local disk paths or other options in this format, then you would
> anyway need some alternative ways of referring to binaries.
>
> The more interesting question to my mind is how you make edits to the
> manifest like the above. After you have more than a handful of plugins
> with versions like these, manually copying and pasting such
> URLs/versions would be cumbersome. I think JENKINS-50953 can be used
> to track this. For example, I might run something like
>
> $ update-incrementals configuration/essentials.yaml
>
> or
>
> $ update-incrementals configuration/essentials.yaml git workflow-api
>
> which for each selected component (or all) would check the current
> version; gather a list of all deployed versions (from
> `maven_metadata.xml`) that compare as strictly ???newer??? than that;
> filter those by commits which are ancestors of current `master`; pick
> the newest of the resulting versions; and edit the file to select that
> version.
>
> You could then commit the result and push to a pull request, which
> would get validated by a PR builder somehow TBD, and if all automated
> checks pass it would be eligible for merging, meaning deployment to
> production.
>
> WDYT? Admittedly the above is a very rough sketch, which I need to
> flesh out quite a bit.


If an update-incrementals tool existed, then yes, that would address my
concerns here. So long as something can automatically update my yaml file, i'm
happy to use it, I more wanted to make sure that we weren't going to have two
or three implementations of what `update-incrementals` effectively should be
providing.


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180430204749.GL2935%40grape.lasagna.io.
For more options, visit https://groups.google.com/d/optout.

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bill of Materials

Jesse Glick-4
On Mon, Apr 30, 2018 at 4:47 PM, R. Tyler Croy <[hidden email]> wrote:
> If an update-incrementals tool existed, then yes, that would address my
> concerns here.

OK, good to know I am not completely off base here.

> I more wanted to make sure that we weren't going to have two
> or three implementations of what `update-incrementals` effectively should be
> providing.

There might be two or three scripts to deal with different syntaxes;
for example, it would be useful to have an equivalent to `mvn
versions:use-latest-releases` which grokked Incrementals and updated a
`pom.xml`. But yes they should share an underlying library of some
kind.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3wTjrORgYGbhMgUJZ1REqEub82sJZvkZNRN_DN9V4r1Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bill of Materials

Oleg Nenashev
Hi all,

Just to provide an update on this topic, I have added support of Bill of Materials (in the current draft version) to Custom WAR Packager and used it in several automation flows. E.g. see Integration Testing for Artifact Manager S3. I can also rework Remoting (PR) and Stapler (PR) flows to use BOM there if it is needed for a reference implementation.

Do I gravely need BOM in these pull requests? The answer is "No" as Jesse pointed out. Custom WAR packager uses its own YAML format which can be integrated into essentials.yml format created by Raul Arabaolaza for integration tests. This format is generally more flexible for packaging WARs for testing purposes (build settings, relative FS paths, support of embedding System properties and Groovy scripts, etc.). In this PR and JEP comments I have pointed out several weaknesses of the current BOM proposal:
  • The provided justification for BOM is true, but actually it is already achieved by Custom WAR Packager.
    • BOM does not make much difference there as described above
    • "We want to unify the description format across projects within Jenkins (e.g. Jenkins Essentials, Jenkins X, Docker Packaging, etc.)." is the real thing we should focus on IMHO, but it needs to be expanded (see proposals below)
  • BOM format is not specified. The JEP proposal just provides an example without specifying fields. It refers the K8s notation, but that's it. There is no link to K8s specification and the version, and this specification may drift over time. It leads to all my further comments here
  • "Metadata/labels" are not specified. What is a "must", what is "optional"?
    • E.g. what is field 'name' and how should it be used?
  • Purpose of "Metadata/annotations" section is not clear.
    • There is neither examples nor documentation
    • I would expect to be able to put any custom data there. E.g. for Custom WAR packager I would expect to put build and packaging settings there. OTOH for Jenkins Essentials I would expect this section to have some deployment rules and so
    • We had a private discussion with Carlos, and his explanation was that "annotations" in K8s notation is just a key-value mapping without object support. It does not allow passing complex configurations via them and hence does not allow using BOM as a sole input for test automation flows (like ATH which is used for justification)
  • "Core" section needs specification
    • I'd guess it defines version of WAR file, but it's not clear from documentation. Note that version of WAR and version of the bundled Jenkins Core may differ (timestamped releases, custom WAR builds, etc.)
    • "Core" does not define "artifactId" or "groupId". It means that only standard Jenkins WAR "org.jenkins-ci.main:jenkins-war" may be used. What if I want to define BOM for a custom product based on Jenkins? It is a real use-case for me
  • Vague "components" section. It is not clear what can be defined there and how it should be handled.
    • Is it only JARs? What about Jenkins Modules, Resources, System Groovy Scripts, etc.
    • There is no "type" field or whatever which would allow to distinguish it
  • Plugins and Components support only "version" and "ref" fields
    • In test automation flows we need to test unreleased components in repositories, and it would be really useful to just reference components by path to sources and by path to HPI/JAR like it's done in custom WAR packager
  • It is not clear how "environment" should be used.
    • Can we define multiple environments? Or just a single one?
    • Can environments override plugins/components? Or do they just append new ones?
  • "status" section requires all components/plugins to have artifactId/groupId/versions
    • What if I add a "Groovy init gook" script from a repository as a component? This component has no real version since it's not released separately. Am I expected to mock them somehow (e.g. commitId)?
    • Does BOM expect all listed components to be available in Maven repo?

IMHO BOM specification should be created and documented properly, maybe using one of standard YAML specification languages.


Even with the comments above, I think that the BOM is actually useful and should be finalized. Why?

  • We really need an inter-exchange format. As Jesse said somewhere, Maven POM/BOM is not a silver-bullet in this area since it does not allow passing extra metadata easily
  • Examples of inter-exchange:
    • Custom Packager generates a specification of bundled components which is then used by runPCT/runATH steps to define defaults for running steps (INFRA-1623)
    • WAR bundles ship BOMs internally so that tools like PCT can easily determine what's inside without parsing files
    • BOM can be used as a source for repackaging third-party WARs. E.g. somebody may be building a custom Docker image on the top of jenkins/blueocean, and BOM could help to resolve dependencies there.
    • Jenkins Essentials could be shipping BOM so that the same set of "Verified plugins" could be used by users who are not ready to switch to AWS but still want to get extra insurance about the plugin cross-compatibility

So I am +1 regarding finalizing BOM as JEP.


Bes regards,

Oleg


On Tuesday, May 1, 2018 at 3:17:44 PM UTC+2, Jesse Glick wrote:
On Mon, Apr 30, 2018 at 4:47 PM, R. Tyler Croy <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="j-f1jCi8AQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">ty...@...> wrote:
> If an update-incrementals tool existed, then yes, that would address my
> concerns here.

OK, good to know I am not completely off base here.

> I more wanted to make sure that we weren't going to have two
> or three implementations of what `update-incrementals` effectively should be
> providing.

There might be two or three scripts to deal with different syntaxes;
for example, it would be useful to have an equivalent to `mvn
versions:use-latest-releases` which grokked Incrementals and updated a
`pom.xml`. But yes they should share an underlying library of some
kind.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/542d0afb-86bd-4f00-a714-96a2bec639b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bill of Materials

Carlos Sanchez



On Fri, May 11, 2018 at 11:44 AM Oleg Nenashev <[hidden email]> wrote:
Hi all,

Just to provide an update on this topic, I have added support of Bill of Materials (in the current draft version) to Custom WAR Packager and used it in several automation flows. E.g. see Integration Testing for Artifact Manager S3. I can also rework Remoting (PR) and Stapler (PR) flows to use BOM there if it is needed for a reference implementation.

Do I gravely need BOM in these pull requests? The answer is "No" as Jesse pointed out. Custom WAR packager uses its own YAML format which can be integrated into essentials.yml format created by Raul Arabaolaza for integration tests. This format is generally more flexible for packaging WARs for testing purposes (build settings, relative FS paths, support of embedding System properties and Groovy scripts, etc.). In this PR and JEP comments I have pointed out several weaknesses of the current BOM proposal:
  • The provided justification for BOM is true, but actually it is already achieved by Custom WAR Packager.
    • BOM does not make much difference there as described above
    • "We want to unify the description format across projects within Jenkins (e.g. Jenkins Essentials, Jenkins X, Docker Packaging, etc.)." is the real thing we should focus on IMHO, but it needs to be expanded (see proposals below)
  • BOM format is not specified. The JEP proposal just provides an example without specifying fields. It refers the K8s notation, but that's it. There is no link to K8s specification and the version, and this specification may drift over time. It leads to all my further comments here
  • "Metadata/labels" are not specified. What is a "must", what is "optional"?
    • E.g. what is field 'name' and how should it be used?
  • Purpose of "Metadata/annotations" section is not clear.
    • There is neither examples nor documentation
    • I would expect to be able to put any custom data there. E.g. for Custom WAR packager I would expect to put build and packaging settings there. OTOH for Jenkins Essentials I would expect this section to have some deployment rules and so
    • We had a private discussion with Carlos, and his explanation was that "annotations" in K8s notation is just a key-value mapping without object support. It does not allow passing complex configurations via them and hence does not allow using BOM as a sole input for test automation flows (like ATH which is used for justification)
  • "Core" section needs specification
    • I'd guess it defines version of WAR file, but it's not clear from documentation. Note that version of WAR and version of the bundled Jenkins Core may differ (timestamped releases, custom WAR builds, etc.)
    • "Core" does not define "artifactId" or "groupId". It means that only standard Jenkins WAR "org.jenkins-ci.main:jenkins-war" may be used. What if I want to define BOM for a custom product based on Jenkins? It is a real use-case for me
  • Vague "components" section. It is not clear what can be defined there and how it should be handled.
    • Is it only JARs? What about Jenkins Modules, Resources, System Groovy Scripts, etc.
    • There is no "type" field or whatever which would allow to distinguish it
  • Plugins and Components support only "version" and "ref" fields
    • In test automation flows we need to test unreleased components in repositories, and it would be really useful to just reference components by path to sources and by path to HPI/JAR like it's done in custom WAR packager
  • It is not clear how "environment" should be used.
    • Can we define multiple environments? Or just a single one?
    • Can environments override plugins/components? Or do they just append new ones?
  • "status" section requires all components/plugins to have artifactId/groupId/versions
    • What if I add a "Groovy init gook" script from a repository as a component? This component has no real version since it's not released separately. Am I expected to mock them somehow (e.g. commitId)?
    • Does BOM expect all listed components to be available in Maven repo?

IMHO BOM specification should be created and documented properly, maybe using one of standard YAML specification languages.


Even with the comments above, I think that the BOM is actually useful and should be finalized. Why?

  • We really need an inter-exchange format. As Jesse said somewhere, Maven POM/BOM is not a silver-bullet in this area since it does not allow passing extra metadata easily
  • Examples of inter-exchange:
    • Custom Packager generates a specification of bundled components which is then used by runPCT/runATH steps to define defaults for running steps (INFRA-1623)
    • WAR bundles ship BOMs internally so that tools like PCT can easily determine what's inside without parsing files
    • BOM can be used as a source for repackaging third-party WARs. E.g. somebody may be building a custom Docker image on the top of jenkins/blueocean, and BOM could help to resolve dependencies there.
    • Jenkins Essentials could be shipping BOM so that the same set of "Verified plugins" could be used by users who are not ready to switch to AWS but still want to get extra insurance about the plugin cross-compatibility

So I am +1 regarding finalizing BOM as JEP.


Thanks for the input. Yes there are missing bits and that was the point of circulating the definition. The format is the least important of things, more need to focus on the interoperability and common definition for war packager, essentials and any future usage
 


Bes regards,

Oleg


On Tuesday, May 1, 2018 at 3:17:44 PM UTC+2, Jesse Glick wrote:
On Mon, Apr 30, 2018 at 4:47 PM, R. Tyler Croy <[hidden email]> wrote:
> If an update-incrementals tool existed, then yes, that would address my
> concerns here.

OK, good to know I am not completely off base here.

> I more wanted to make sure that we weren't going to have two
> or three implementations of what `update-incrementals` effectively should be
> providing.

There might be two or three scripts to deal with different syntaxes;
for example, it would be useful to have an equivalent to `mvn
versions:use-latest-releases` which grokked Incrementals and updated a
`pom.xml`. But yes they should share an underlying library of some
kind.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/542d0afb-86bd-4f00-a714-96a2bec639b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CALHFn6NYkni%3Ds8vct-FOqrAoFWhQ01M0yci3dO1Q4-UBDbAfLg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bill of Materials

Jesse Glick-4
In reply to this post by Oleg Nenashev
On Fri, May 11, 2018 at 6:43 AM, Oleg Nenashev <[hidden email]> wrote:
> We really need an inter-exchange format. As Jesse said somewhere, Maven
> POM/BOM is not a silver-bullet in this area since it does not allow passing
> extra metadata easily

Indeed the v4 POM format does not permit custom metadata inline in
`<dependency>`s. On the other hand you can always use a POM to define
component versions and then mix it with other metadata for particular
purposes. This is certainly friendlier to existing developer workflows
than attempting to generate or update POMs from another source. We
anyway need a Maven BOM (i.e., `pom.xml` with
`<dependencyManagement>`) just to slim down per-plugin POMs and make
it easy to develop against the “latest” versions of other stuff.

I think the trouble here is that we have a bunch of different tools
and workflows with different needs, and it is not that clear to me how
much overlap there really is, or what problems we are trying to solve
with a unified format. For example, for Evergreen deployment, we just
need a list of components with versions (all assumed to come from
`master`), which could easily fit in a POM if you also differentiate
“environments” like AWS somewhere. For `buildPlugin` integration
testing, it makes the most sense to pick up versions of other
components from the Maven `test` scope, which is already used for
functional tests. (You also need an optional branch override for
`incrementals:update`, but this can easily be handled in mojo
configuration.)

> Custom Packager generates a specification of bundled components which is
> then used by runPCT/runATH steps to define defaults for running steps

Details of these test suites are configured via the `essentials.yaml`.
I think you do not also need a BOM, if you are picking up `test`
scope.

> WAR bundles ship BOMs internally so that tools like PCT can easily determine
> what's inside without parsing files

The code to look up component versions in PCT already exists. Are
there some subtleties not captured by the traditional logic?

> BOM can be used as a source for repackaging third-party WARs.

This is Custom WAR Packager IIUC?

> Jenkins Essentials could be shipping BOM

Well, Evergreen will be “shipping” a flat list of core & plugin URLs,
if I gather Tyler’s intention correctly—the minimum information needed
by the client. It is an open question what build process generates
this list and from what source.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1Nq%3DSzsWQJ0WzVBZbBhh4HGX1Cr0XRiD-%3DPxPX_6gN_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.