External Fingerprint Storage for Jenkins

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

External Fingerprint Storage for Jenkins

Sumit Sarin
Hi all,
It brings me great joy in introducing the Jenkins community to one of the ongoing Google Summer of Code (GSoC) projects: External Fingerprint Storage for Jenkins.

File fingerprinting is a way to track which version of a file is being used by a job/build, making dependency tracking easy. The fingerprint engine of Jenkins can track usages of artifacts, credentials, files, etc. within the system. It does this by maintaining a local XML-based database. This leads to dependence on the physical disk of the Jenkins master. Hence, the core idea of this project is to extend Jenkins core to support storing of fingerprints in an external storage, which would also allow tracking them across the entire CI/CD flow. 

The goals of this project include:
  • Building a pluggable storage engine, which would allow fingerprints to be stored in external storages managed by storage system specific plugins.
  • Reference implementation in the form of Redis backed fingerprint storage plugin.
  • Allowing fingerprints to be traced across Jenkins instances (possibly via another plugin)
The code for this project currently lives in two places: 
So whilst we are working towards this project, we would love to receive community suggestions and feedback as to how we can make this project even better, and how maybe we can solve some problems for the users/developers. Here is a link to our draft design document, which is soon to be converted into a JEP.

I would also like to invite everyone interested to join us for a demo of the project at the Cloud Native SIG Meeting, for which the doodle is open for voting.

All other information regarding the project can be found at our up to date project page.

Best Regards,
Sumit Sarin




--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/c1c90be9-dc80-4504-ac6a-0eb007607cefo%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: External Fingerprint Storage for Jenkins

Sumit Sarin
Hi all,
Yesterday we had a Cloud Native SIG meeting where we presented the demo for the External Fingerprint Storage project and had some insightful discussions in regards to making the project even better. I immensely thank everyone who participated!

If somebody wanted to attend but missed it, the recording and slides are linked. 
Also the PR for this project's JEP is also ready.

Thanks again,

Best Regards,
Sumit Sarin

On Tuesday, 9 June 2020 14:22:05 UTC+5:30, Sumit Sarin wrote:
Hi all,
It brings me great joy in introducing the Jenkins community to one of the ongoing Google Summer of Code (GSoC) projects: External Fingerprint Storage for Jenkins.

File fingerprinting is a way to track which version of a file is being used by a job/build, making dependency tracking easy. The fingerprint engine of Jenkins can track usages of artifacts, credentials, files, etc. within the system. It does this by maintaining a local XML-based database. This leads to dependence on the physical disk of the Jenkins master. Hence, the core idea of this project is to extend Jenkins core to support storing of fingerprints in an external storage, which would also allow tracking them across the entire CI/CD flow. 

The goals of this project include:
  • Building a pluggable storage engine, which would allow fingerprints to be stored in external storages managed by storage system specific plugins.
  • Reference implementation in the form of Redis backed fingerprint storage plugin.
  • Allowing fingerprints to be traced across Jenkins instances (possibly via another plugin)
The code for this project currently lives in two places: 
  1. <a href="https://github.com/jenkinsci/jenkins/pull/4731" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fpull%2F4731\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGkOz2J-fYI4xBJ9psJTpGvlH9FEQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fpull%2F4731\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGkOz2J-fYI4xBJ9psJTpGvlH9FEQ&#39;;return true;">PR introducing pluggable storage in Jenkins core.
  2. <a href="https://github.com/jenkinsci/redis-fingerprint-storage-plugin" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fredis-fingerprint-storage-plugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFqHkZEntLjJaDtBQ8ZKb8N9pSsGg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fredis-fingerprint-storage-plugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFqHkZEntLjJaDtBQ8ZKb8N9pSsGg&#39;;return true;">Redis Fingerprint Storage Plugin (Reference implementation)
So whilst we are working towards this project, we would love to receive community suggestions and feedback as to how we can make this project even better, and how maybe we can solve some problems for the users/developers. Here is a link to our draft <a href="https://docs.google.com/document/d/1_LhdsOdvxUDLgyo8vAB1PJ5-85csr7YVI3WkEyNv42w/edit#" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://docs.google.com/document/d/1_LhdsOdvxUDLgyo8vAB1PJ5-85csr7YVI3WkEyNv42w/edit#&#39;;return true;" onclick="this.href=&#39;https://docs.google.com/document/d/1_LhdsOdvxUDLgyo8vAB1PJ5-85csr7YVI3WkEyNv42w/edit#&#39;;return true;">design document, which is soon to be converted into a JEP.

I would also like to invite everyone interested to join us for a demo of the project at the Cloud Native SIG Meeting, for which the <a href="https://doodle.com/poll/8kvhxxed2b36hk7e" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fdoodle.com%2Fpoll%2F8kvhxxed2b36hk7e\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFkYQCA9uzR9ns5c5T0OV31U01pYA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fdoodle.com%2Fpoll%2F8kvhxxed2b36hk7e\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFkYQCA9uzR9ns5c5T0OV31U01pYA&#39;;return true;">doodle is open for voting.

All other information regarding the project can be found at our up to date <a href="https://www.jenkins.io/projects/gsoc/2020/projects/external-fingerprint-storage/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.jenkins.io%2Fprojects%2Fgsoc%2F2020%2Fprojects%2Fexternal-fingerprint-storage%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNESPWb9i1Jy3XZ0gJ1UlyGPOpxSuw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.jenkins.io%2Fprojects%2Fgsoc%2F2020%2Fprojects%2Fexternal-fingerprint-storage%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNESPWb9i1Jy3XZ0gJ1UlyGPOpxSuw&#39;;return true;">project page.

Best Regards,
Sumit Sarin




--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/d6aab8d9-a265-4c70-90be-e3a61224c538o%40googlegroups.com.