Accelerating Jenkins development with Jenkins Essentials

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Accelerating Jenkins development with Jenkins Essentials

R. Tyler Croy

After pounding my head against my whiteboard for a couple months, I'm finally
ready to talk about this idea which was first discussed at Jenkins World 2017
in San Francisco.

At the Contributor Summit we held, Kohsuke challenged us to make Jenkins easy
enough for an end-user to go from "zero" (downloading Jenkins) to continuously
delivering a project in under 5 clicks and under 5 minutes.

Jenkins Essentials has this goal in mind, with an additional goal of making
Jenkins development more efficient.

I have proposed a draft of JEP-300 which outlines the problem domain and
overview of Jenkins Essentials more thoroughly

rendered: https://github.com/rtyler/jep/tree/essentials-overview-300/jep/300
pull request to jenkinsci/jep: https://github.com/jenkinsci/jep/pull/48


To be successful with Jenkins Essentials, I'm endeavouring to improve
pre-release verification as well as post-release monitoring. Some of the
initial tasks that my colleague Raul and I are tackling can be found on this
board: https://issues.jenkins-ci.org/secure/RapidBoard.jspa?rapidView=396


I'm still sketching out the Developer Experience aspects of this in another
thread once I have some thoughts written down.


Please take a look at the document and let me know what you think on this
thread :)



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180206160253.dkz7whf7h74bda64%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Evergreen packaging for Jenkins Essentials

R. Tyler Croy
In the same vein as my previous email, I've written up a design document which
outlines the current approach for packaging I intend to use for Jenkins
Essentials:

    https://github.com/jenkinsci/jep/tree/master/jep/301

I think it should be fairly straight-forward and understandable why I'm taking
the approach described, but if there any ambiguity do let me know.

Please take a look at the document and let me know what you think on this
thread :)



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180207212838.ndyahjf7psu5obk2%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

stephenconnolly
Looks good to me.

My only concern is mixing three VMs in the one container. Python, node and java... would be nicer if that could be reduced... but ack that it’s non-trivial to do so

On Wed 7 Feb 2018 at 21:28, R. Tyler Croy <[hidden email]> wrote:
In the same vein as my previous email, I've written up a design document which
outlines the current approach for packaging I intend to use for Jenkins
Essentials:

    https://github.com/jenkinsci/jep/tree/master/jep/301

I think it should be fairly straight-forward and understandable why I'm taking
the approach described, but if there any ambiguity do let me know.

Please take a look at the document and let me know what you think on this
thread :)



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180207212838.ndyahjf7psu5obk2%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.
--
Sent from my phone

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMxu9ZkdQJK%3D%2Bq8hj%3DFpHfn81j0eaaPh5dBWo8znuvtiBA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

R. Tyler Croy
(replies inline)

On Wed, 07 Feb 2018, Stephen Connolly wrote:

> Looks good to me.
>
> My only concern is mixing three VMs in the one container. Python, node and
> java... would be nicer if that could be reduced... but ack that it???s
> non-trivial to do so


I understand the desire to reduce dependencies but there are severe tradeoffs
to doing so, unfortunately :(

Java is an obvious dependency of Jenkins. Unfortunately Java is beefy at
runtime (especially with Jenkins), and doesn't have great support for acting as
a Server-sent Events client, along with some other niceities that I need for
the evergreen-client implementation.

As for Python, that's obviously a dependency for supervisord, but it also
shares some of the same limitations as Java for implementing an
evergreen-client. Python would definitely be my preference for implementing the
client tooling but alas. Also, I have not found a more mature, and featureful
process supervisor, in C, Node, etc, other than supervisord. I've been using it
in various projects with great success for a long time and it's well worth (to
me) the disk space occupied by the alpine python package.


Node is unfortunately the best tool for the job of the evergreen-client, and
believe me I'm not happy to be advocating for that position, since it dooms me
to writing much more JavaScript than I would like to :)


All that said, this is definitely not locked in stone. One of the big
advantages to defaulting to the container as the package means we can readily
swap this out in the future if we learn more, or our needs change.


Anywho, thanks for taking a read and providing some feedback! I hope I've
helped explain why there are some similar but different dependencies in the
jenkins/evergreen design.


Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180208001217.k7xynhz7zt4fdamk%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Jenkins development with Jenkins Essentials

surya gaddipati-2
In reply to this post by R. Tyler Croy
Hi, 

This is exciting new development. 

Couple of comments

1.  Packaging supervisord with another executable is a docker anti pattern. I assume  any serious  jenkins deployment is on a scheduler like swarm/kubernetes ect where supervisord is not necessary. 

2. Scripting should be done via yaml files, not environment variables( init.d/groovy). All essential plugins should be made compatible with configuration-as-code.

3. Pipeline while powerful is contrary to the goal of getting new users started quickly. From my personal experience, very few new users have the appetite to learn groovy scripting to setup their builds, they just want to something to run `npm test`. 
    A declarative yaml file like travisci,circle ect is the *correct* approach here. 




On Tuesday, February 6, 2018 at 10:03:05 AM UTC-6, R Tyler Croy wrote:

After pounding my head against my whiteboard for a couple months, I'm finally
ready to talk about this idea which was first discussed at Jenkins World 2017
in San Francisco.

At the Contributor Summit we held, Kohsuke challenged us to make Jenkins easy
enough for an end-user to go from "zero" (downloading Jenkins) to continuously
delivering a project in under 5 clicks and under 5 minutes.

Jenkins Essentials has this goal in mind, with an additional goal of making
Jenkins development more efficient.

I have proposed a draft of JEP-300 which outlines the problem domain and
overview of Jenkins Essentials more thoroughly

rendered: <a href="https://github.com/rtyler/jep/tree/essentials-overview-300/jep/300" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frtyler%2Fjep%2Ftree%2Fessentials-overview-300%2Fjep%2F300\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG-m32GRhEsjRsklGLqJv-WdGSwzw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frtyler%2Fjep%2Ftree%2Fessentials-overview-300%2Fjep%2F300\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG-m32GRhEsjRsklGLqJv-WdGSwzw&#39;;return true;">https://github.com/rtyler/jep/tree/essentials-overview-300/jep/300
pull request to jenkinsci/jep: <a href="https://github.com/jenkinsci/jep/pull/48" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjep%2Fpull%2F48\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHDvR-5ctYYgrjXZxm8W2Iu5lecFw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjep%2Fpull%2F48\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHDvR-5ctYYgrjXZxm8W2Iu5lecFw&#39;;return true;">https://github.com/jenkinsci/jep/pull/48


To be successful with Jenkins Essentials, I'm endeavouring to improve
pre-release verification as well as post-release monitoring. Some of the
initial tasks that my colleague Raul and I are tackling can be found on this
board: <a href="https://issues.jenkins-ci.org/secure/RapidBoard.jspa?rapidView=396" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fissues.jenkins-ci.org%2Fsecure%2FRapidBoard.jspa%3FrapidView%3D396\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHAsLwcMKw9xG6T78bA97VvoMf0iQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fissues.jenkins-ci.org%2Fsecure%2FRapidBoard.jspa%3FrapidView%3D396\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHAsLwcMKw9xG6T78bA97VvoMf0iQ&#39;;return true;">https://issues.jenkins-ci.org/secure/RapidBoard.jspa?rapidView=396


I'm still sketching out the Developer Experience aspects of this in another
thread once I have some thoughts written down.


Please take a look at the document and let me know what you think on this
thread :)



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <<a href="https://github.com/rtyler" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frtyler\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFtsnCUZ085B8982iTQ2KFqz5gYhw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frtyler\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFtsnCUZ085B8982iTQ2KFqz5gYhw&#39;;return true;">https://github.com/rtyler>
  Chatter: <<a href="https://twitter.com/agentdero" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Ftwitter.com%2Fagentdero\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEILlTAfpmlCsYIT_Phy_UKs4kUlw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Ftwitter.com%2Fagentdero\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEILlTAfpmlCsYIT_Phy_UKs4kUlw&#39;;return true;">https://twitter.com/agentdero>
     xmpp: <a href="javascript:" target="_blank" gdf-obfuscated-mailto="LOr0QtvBAAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rty...@...

  % gpg --keyserver <a href="http://keys.gnupg.net" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fkeys.gnupg.net\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGGvem0n8cSWWfyOfhzR-uVSpk4NQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fkeys.gnupg.net\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGGvem0n8cSWWfyOfhzR-uVSpk4NQ&#39;;return true;">keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/a7ff4dc8-c5e1-4c5a-b041-403d61db8768%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Jenkins development with Jenkins Essentials

Kohsuke Kawaguchi
Administrator
In reply to this post by R. Tyler Croy
Yay! I think this is an important step for the Jenkins project. I'm really excited.

We started talking about this in some of the events adjacent to FOSDEM. I think people generally grok the idea and they are happy that we are attacking these challenges.

On Tue, Feb 6, 2018 at 8:03 AM R. Tyler Croy <[hidden email]> wrote:

After pounding my head against my whiteboard for a couple months, I'm finally
ready to talk about this idea which was first discussed at Jenkins World 2017
in San Francisco.

At the Contributor Summit we held, Kohsuke challenged us to make Jenkins easy
enough for an end-user to go from "zero" (downloading Jenkins) to continuously
delivering a project in under 5 clicks and under 5 minutes.

Jenkins Essentials has this goal in mind, with an additional goal of making
Jenkins development more efficient.

I have proposed a draft of JEP-300 which outlines the problem domain and
overview of Jenkins Essentials more thoroughly

rendered: https://github.com/rtyler/jep/tree/essentials-overview-300/jep/300
pull request to jenkinsci/jep: https://github.com/jenkinsci/jep/pull/48


To be successful with Jenkins Essentials, I'm endeavouring to improve
pre-release verification as well as post-release monitoring. Some of the
initial tasks that my colleague Raul and I are tackling can be found on this
board: https://issues.jenkins-ci.org/secure/RapidBoard.jspa?rapidView=396


I'm still sketching out the Developer Experience aspects of this in another
thread once I have some thoughts written down.


Please take a look at the document and let me know what you think on this
thread :)



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180206160253.dkz7whf7h74bda64%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.
--
Kohsuke Kawaguchi

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4zRnEdRvjBS8OTW3sTUpGEyYh4L6OcLjQZ38sr1_21bXA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

Jesse Glick-4
In reply to this post by R. Tyler Croy
https://github.com/jenkinsci/jep/blob/master/jep/301/README.adoc#plugins

does not make sense to me. A critical benefit of this system as I
understood it was that we could deliver a complete software
package—including Jenkins core and plugins—as a unit. So why are you
treating core and plugins as distinct things? You should either

· Package a particular version of core, and of all “essential”
plugins, together in a particular version of a Docker image, and have
people use a new tag of this image when they want to update anything.
That would be the normal way to manage software in Docker, though it
appears to contradict your intention of using `evergreen-client` to
perform upgrades.

or

· Have the image contain only the supervisor, and have it download
specific versions of Jenkins core and plugins on demand; and update
the image itself only when the supervisor changes (presumably rarely).
That also bypasses the concern noted in #security as the image would
not need to be updated merely because of a core security advisory.


https://github.com/jenkinsci/jep/blob/master/jep/300/README.adoc#sane-defaults

I do not think it is wise to try to automagically detect a cloud
environment. Rather, we should offer variant images which are
preconfigured for specific environments like AWS. Or offer startup
flags etc. to select such preconfigurations, and tailor the
installation scripts for those environments to pass the right flag.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0o3N68iK_N3b0jH5g-N3KcfG-9rzgGrk_QXqBQBdP0iA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

Jesse Glick-4
BTW

On Thu, Feb 8, 2018 at 5:16 PM, Jesse Glick <[hidden email]> wrote:
> why are you treating core and plugins as distinct things?

was meant as a fairly technical comment about the packaging design.
The goal here makes a lot of sense.


On this topic, is there any information on how you would propose new
snapshots of the system (core + plugin versions, I guess) get
“promoted” or “tagged” or whatever by Jenkins devs?

https://github.com/jenkinsci/jep/blob/master/jep/301/README.adoc#infrastructure-requirements

mentions that there would be some kind of build and test
infrastructure, but that is about it. A topic for another JEP?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2Fqg33g%2B7jOTY4Q6RsaGD%3DkfUKmsOcVuXtZzqNwLqkNQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Jenkins development with Jenkins Essentials

Kohsuke Kawaguchi
Administrator
In reply to this post by Kohsuke Kawaguchi
While this thread is quiet, between the previous rounds of discussions in Jenkins World contributor summit and the recent mailing list discussion, I think there's a consensus of the general problem space. I intend to accept this JEP here within several days.  So please roll in your +1s!

Tyler will be working on this area, and he has been championing the education/arming of users as well as making Jenkins accessible to the broader users, on top of all the other things he has worked on over the past decade. I think he's well qualified to drive this effort, and as such I intend to delegate subsequent JEPs in this space to Tyler.


On Thu, Feb 8, 2018 at 11:05 AM Kohsuke Kawaguchi <[hidden email]> wrote:
Yay! I think this is an important step for the Jenkins project. I'm really excited.

We started talking about this in some of the events adjacent to FOSDEM. I think people generally grok the idea and they are happy that we are attacking these challenges.


On Tue, Feb 6, 2018 at 8:03 AM R. Tyler Croy <[hidden email]> wrote:

After pounding my head against my whiteboard for a couple months, I'm finally
ready to talk about this idea which was first discussed at Jenkins World 2017
in San Francisco.

At the Contributor Summit we held, Kohsuke challenged us to make Jenkins easy
enough for an end-user to go from "zero" (downloading Jenkins) to continuously
delivering a project in under 5 clicks and under 5 minutes.

Jenkins Essentials has this goal in mind, with an additional goal of making
Jenkins development more efficient.

I have proposed a draft of JEP-300 which outlines the problem domain and
overview of Jenkins Essentials more thoroughly

rendered: https://github.com/rtyler/jep/tree/essentials-overview-300/jep/300
pull request to jenkinsci/jep: https://github.com/jenkinsci/jep/pull/48


To be successful with Jenkins Essentials, I'm endeavouring to improve
pre-release verification as well as post-release monitoring. Some of the
initial tasks that my colleague Raul and I are tackling can be found on this
board: https://issues.jenkins-ci.org/secure/RapidBoard.jspa?rapidView=396


I'm still sketching out the Developer Experience aspects of this in another
thread once I have some thoughts written down.


Please take a look at the document and let me know what you think on this
thread :)



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180206160253.dkz7whf7h74bda64%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.
--
Kohsuke Kawaguchi
--
Kohsuke Kawaguchi

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4z0jxf0toO3x7DNXscBtQ4mpVD%2BvsCz136vgiKOm_ossw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

R. Tyler Croy
In reply to this post by Jesse Glick-4
(replies inline)

On Thu, 08 Feb 2018, Jesse Glick wrote:

> https://github.com/jenkinsci/jep/blob/master/jep/301/README.adoc#plugins
>
> does not make sense to me. A critical benefit of this system as I
> understood it was that we could deliver a complete software
> package???including Jenkins core and plugins???as a unit. So why are you
> treating core and plugins as distinct things? You should either
>
> · Package a particular version of core, and of all ???essential???
> plugins, together in a particular version of a Docker image, and have
> people use a new tag of this image when they want to update anything.
> That would be the normal way to manage software in Docker, though it
> appears to contradict your intention of using `evergreen-client` to
> perform upgrades.
>
> or
>
> · Have the image contain only the supervisor, and have it download
> specific versions of Jenkins core and plugins on demand; and update
> the image itself only when the supervisor changes (presumably rarely).
> That also bypasses the concern noted in #security as the image would
> not need to be updated merely because of a core security advisory.

I apologize for the delay, I needed some time to think about and understand the
points you're raising here. I think we're conceptually on the same page, and
it took a bit of thinking and consultation with Kohsuke to help me fully
understand your feedback here.

I think the design document was unclear, and had a hidden premature
optimization/assumption. I have been thus far deriving jenkins/evergreen from
jenkins/jenkins simply to inherit some scripts, permissions, and other bits.
This is definitely not necessary, and I agree the packaging should follow the
second approach you describe, more or less:

 1. Come online
 2. Grab the latest, greatest, and secure-iest bits
 3. Do a Jenkins \o/

I will make the JEP-301 document more clear on that.


> https://github.com/jenkinsci/jep/blob/master/jep/300/README.adoc#sane-defaults
>
> I do not think it is wise to try to automagically detect a cloud
> environment. Rather, we should offer variant images which are
> preconfigured for specific environments like AWS. Or offer startup
> flags etc. to select such preconfigurations, and tailor the
> installation scripts for those environments to pass the right flag.


I /think/ I understand where the reasoning would be for "flavors" depending on
the cloud environment (for example), but to make sure I'm not making any more
assumptions, would you mind expanding a bit on what your concerns are regarding
a singular "all-in-one" package which can include the necessary cloud-detection
auto-configuration tooling?


Thanks for taking the time to review both documents and provide thoughtful
feedback. I'll try to improve on my delay times, work responsibilities, you
know the drillThanks for taking the time to review both documents and provide
thoughtful feedback. I'll try to improve on my delay times, work
responsibilities, you know the drill.


*looks wearily at google calendar*

I'll try to land some updates to the prototype and these design documents with
your feedback tomorrow (Feb 14th).



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180214000359.x72xio6o5m6qads3%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Jenkins development with Jenkins Essentials

R. Tyler Croy
In reply to this post by surya gaddipati-2
(replies inline)

On Thu, 08 Feb 2018, Surya Gaddipati wrote:

> Hi,
>
> This is exciting new development.
>
> Couple of comments
>
> 1.  Packaging supervisord with another executable is a docker anti pattern.
> I assume  any serious  jenkins deployment is on a scheduler like
> swarm/kubernetes ect where supervisord is not necessary.


I understand the point made but I disagree with the notion that "serious"
Jenkins deployments are using Swarm or Kubernetes. The Jenkins project itself
(for example) has been running Docker in production for over four years now,
without a container orchestrator of any kind. This includes
https://ci.jenkins.io/. We use the garethr/docker Puppet module
(https://forge.puppet.com/garethr/docker) which itself has millions of
downloads by folks managing Docker services "the hard way." That said, we also
have a Kubernetes environment where we're deploying newer applications (e.g.
https://plugins.jenkins.io/) into. So I /agree/ with the point that Kubernetes
is likely a good candidate for replacing some of the Jenkins lifecycle
management work that supervisord is currently performing, which is why I
mention Kubernetes in JEP-301 at all. However, I strongly
disagree that the market is "there" yet, I think making a container
orchestrator a requirement at this point would only inhibit adoption and
learning. I am instead erring on the side of "simple and fast" from a user
perspective, the closest we can get (IMHO) to the `java -jar jenkins.war`
experience, the easier the adoption and experimentation will be for the user
audience I want to reach.


> 2. Scripting should be done via yaml files, not environment variables(
> init.d/groovy). All essential plugins should be made compatible with
> configuration-as-code.

I fully intend on using as much support as possible from the Configuration as
Code effort which Ewelina and Nicolas are working on. I do not intend to
undertake, as part of Jenkins Essentials, patching or updating plugins in order
to support it however.

My order of priority for scripting Jenkins is:

 1. Configuration as Code
 2. Groovy scripting (bleh)
 3. Hopefully very little if any scary hacks.


So I think we're mostly on same page here :)


> 3. Pipeline while powerful is contrary to the goal of getting new users
> started quickly. From my personal experience, very few new users have the
> appetite to learn groovy scripting to setup their builds, they just want to
> something to run `npm test`.
>     A declarative yaml file like travisci,circle ect is the *correct*
> approach here.

I agree that this is an important goal for Jenkins as a whole to consider:
making Pipeline easier to use and be successful with. I believe those currently
hacking on Pipeline, like abayer, would agree with that sentiment. What *is* in
the scope of this work is making Jenkins easier to understand "out of the box"
(https://github.com/jenkinsci/jep/tree/master/jep/300#obvious-path) which I
believe is going to entail a number of copy-and-pasteable examples of common
Jenkinsfiles, and other forms of built-in examples and documentation.

Overall, thank you very much for taking the time to read JEPs and provide
feedback. The way I am interpreting your overall feedback, and please correct
me if I'm wrong, is that the direction is promising but want to encourage good
fundamentals on the implementation.



Anywho, thanks again :)





- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180214000422.hkio3rohqdgzn4pd%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

Manuel Jesús Recena Soto-2
In reply to this post by R. Tyler Croy
Hello Tyler,

A great document.

Regards,

2018-02-07 22:28 GMT+01:00 R. Tyler Croy <[hidden email]>:
In the same vein as my previous email, I've written up a design document which
outlines the current approach for packaging I intend to use for Jenkins
Essentials:

    https://github.com/jenkinsci/jep/tree/master/jep/301

I think it should be fairly straight-forward and understandable why I'm taking
the approach described, but if there any ambiguity do let me know.

Please take a look at the document and let me know what you think on this
thread :)



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180207212838.ndyahjf7psu5obk2%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.



--
Manuel Recena Soto
* manuelrecena.com [/blog]
* linkedin.com/in/recena

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CABa-UoeD0_hBbMUhsJ%2B_qMsn5evsyDpsgosiKb45ouub79sELQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

Manuel Jesús Recena Soto-2
In reply to this post by stephenconnolly


2018-02-08 0:14 GMT+01:00 Stephen Connolly <[hidden email]>:
Looks good to me.

My only concern is mixing three VMs in the one container. Python, node and java... would be nicer if that could be reduced... but ack that it’s non-trivial to do so

I agree with this Stephen's comment. I would try to reduce the technology stack to Java (totally needed) and Python.

Regards,
 

On Wed 7 Feb 2018 at 21:28, R. Tyler Croy <[hidden email]> wrote:
In the same vein as my previous email, I've written up a design document which
outlines the current approach for packaging I intend to use for Jenkins
Essentials:

    https://github.com/jenkinsci/jep/tree/master/jep/301

I think it should be fairly straight-forward and understandable why I'm taking
the approach described, but if there any ambiguity do let me know.

Please take a look at the document and let me know what you think on this
thread :)



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180207212838.ndyahjf7psu5obk2%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.
--
Sent from my phone

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMxu9ZkdQJK%3D%2Bq8hj%3DFpHfn81j0eaaPh5dBWo8znuvtiBA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
Manuel Recena Soto
* manuelrecena.com [/blog]
* linkedin.com/in/recena

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CABa-Uod8x-MoZqkRFLsymtaCQ14iarDpo_Nh2RVhTxAFOqLBXA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

Oleg Nenashev
The proposal looks good to me.
Also +1 for the JEP-300 writeup. My suggestion would be to also create an Evergreen subproject page once it is approved.

BR, Oleg

среда, 14 февраля 2018 г., 8:39:26 UTC+1 пользователь Manuel Recena Soto написал:


2018-02-08 0:14 GMT+01:00 Stephen Connolly <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="UJT6wkI5BgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">stephen.al...@gmail.com>:
Looks good to me.

My only concern is mixing three VMs in the one container. Python, node and java... would be nicer if that could be reduced... but ack that it’s non-trivial to do so

I agree with this Stephen's comment. I would try to reduce the technology stack to Java (totally needed) and Python.

Regards,
 

On Wed 7 Feb 2018 at 21:28, R. Tyler Croy <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="UJT6wkI5BgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">ty...@...> wrote:
In the same vein as my previous email, I've written up a design document which
outlines the current approach for packaging I intend to use for Jenkins
Essentials:

    <a href="https://github.com/jenkinsci/jep/tree/master/jep/301" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjep%2Ftree%2Fmaster%2Fjep%2F301\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHlvzMZlXi7OnS264RVR_gfNtMCEA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjep%2Ftree%2Fmaster%2Fjep%2F301\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHlvzMZlXi7OnS264RVR_gfNtMCEA&#39;;return true;">https://github.com/jenkinsci/jep/tree/master/jep/301

I think it should be fairly straight-forward and understandable why I'm taking
the approach described, but if there any ambiguity do let me know.

Please take a look at the document and let me know what you think on this
thread :)



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <<a href="https://github.com/rtyler" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frtyler\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFtsnCUZ085B8982iTQ2KFqz5gYhw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frtyler\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFtsnCUZ085B8982iTQ2KFqz5gYhw&#39;;return true;">https://github.com/rtyler>
  Chatter: <<a href="https://twitter.com/agentdero" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Ftwitter.com%2Fagentdero\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEILlTAfpmlCsYIT_Phy_UKs4kUlw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Ftwitter.com%2Fagentdero\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEILlTAfpmlCsYIT_Phy_UKs4kUlw&#39;;return true;">https://twitter.com/agentdero>
     xmpp: <a href="javascript:" target="_blank" gdf-obfuscated-mailto="UJT6wkI5BgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">rty...@...

  % gpg --keyserver <a href="http://keys.gnupg.net" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fkeys.gnupg.net\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGGvem0n8cSWWfyOfhzR-uVSpk4NQ&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fkeys.gnupg.net\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGGvem0n8cSWWfyOfhzR-uVSpk4NQ&#39;;return true;">keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="UJT6wkI5BgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-dev/20180207212838.ndyahjf7psu5obk2%40blackberry.coupleofllamas.com" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-dev/20180207212838.ndyahjf7psu5obk2%40blackberry.coupleofllamas.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-dev/20180207212838.ndyahjf7psu5obk2%40blackberry.coupleofllamas.com&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-dev/20180207212838.ndyahjf7psu5obk2%40blackberry.coupleofllamas.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.
--
Sent from my phone

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="UJT6wkI5BgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMxu9ZkdQJK%3D%2Bq8hj%3DFpHfn81j0eaaPh5dBWo8znuvtiBA%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMxu9ZkdQJK%3D%2Bq8hj%3DFpHfn81j0eaaPh5dBWo8znuvtiBA%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMxu9ZkdQJK%3D%2Bq8hj%3DFpHfn81j0eaaPh5dBWo8znuvtiBA%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMxu9ZkdQJK%3D%2Bq8hj%3DFpHfn81j0eaaPh5dBWo8znuvtiBA%40mail.gmail.com.

For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.



--
Manuel Recena Soto
* <a href="http://manuelrecena.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fmanuelrecena.com\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG4NJbiESZx9fV8i2cY6epVCP-68A&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fmanuelrecena.com\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG4NJbiESZx9fV8i2cY6epVCP-68A&#39;;return true;">manuelrecena.com [/blog]
* <a href="http://linkedin.com/in/recena" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Flinkedin.com%2Fin%2Frecena\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEFGz8fhx05G8xMhTN65ITHGPYlsw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Flinkedin.com%2Fin%2Frecena\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEFGz8fhx05G8xMhTN65ITHGPYlsw&#39;;return true;">linkedin.com/in/recena

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/4e6f0c39-c9aa-4bc1-9d5b-b223adbdd056%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

Jesse Glick-4
In reply to this post by R. Tyler Croy
On Tue, Feb 13, 2018 at 7:03 PM, R. Tyler Croy <[hidden email]> wrote:
> I have been thus far deriving jenkins/evergreen from
> jenkins/jenkins simply to inherit some scripts, permissions, and other bits.

Yeah, that makes sense as a way to get things up and running quickly.
Once we have an accepted process for tagging a “release” of “Jenkins
Essentials” as a whole, as implied by

https://github.com/jenkinsci/jep/blob/master/jep/300/README.adoc#auto-update

then we can rewrite the image to handle core consistently.

>> I do not think it is wise to try to automagically detect a cloud
>> environment. […]
>
> […what are your concerns] regarding
> a singular "all-in-one" package which can include the necessary cloud-detection
> auto-configuration tooling?

A general distrust of anything magical and opaque that might or might
not work depending on factors too complex for users to see or
understand. If we have a recommended way for Jenkins to run in AWS,
polish it up and put it on Marketplace, if possible. If we have a
recommended way for Jenkins to run in a self-managed Kubernetes
cluster, publish an official Helm chart for it. But keep the basic
Docker image, i.e., whatever happens when I run something like

docker run --rm -v jh:/var/jenkins jenkins/evergreen

straightforward and predictable.

If you are worried about discoverability of our preconfigurations, you
can always add an `AdministrativeMonitor` that says something like

> Hey there! From sniffing around, it looks like you are running this Jenkins service in Azure, but you are not taking advantage of the preintegration work we have done for you.
> For more information, see: https://azuremarketplace.microsoft.com/etc.
> [[Dismiss]]

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1e-ehoMU9NhBaX1TwDsPaV800o%3Drto7D3Yj5VG8ZeGpw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Jenkins development with Jenkins Essentials

nicolas de loof-2
In reply to this post by R. Tyler Croy


2018-02-14 1:04 GMT+01:00 R. Tyler Croy <[hidden email]>:
(replies inline)

On Thu, 08 Feb 2018, Surya Gaddipati wrote:

> Hi,
>
> This is exciting new development.
>
> Couple of comments
>
> 1.  Packaging supervisord with another executable is a docker anti pattern.
> I assume  any serious  jenkins deployment is on a scheduler like
> swarm/kubernetes ect where supervisord is not necessary.


I understand the point made but I disagree with the notion that "serious"
Jenkins deployments are using Swarm or Kubernetes. The Jenkins project itself
(for example) has been running Docker in production for over four years now,
without a container orchestrator of any kind. This includes
https://ci.jenkins.io/. We use the garethr/docker Puppet module
(https://forge.puppet.com/garethr/docker) which itself has millions of
downloads by folks managing Docker services "the hard way." That said, we also
have a Kubernetes environment where we're deploying newer applications (e.g.
https://plugins.jenkins.io/) into. So I /agree/ with the point that Kubernetes
is likely a good candidate for replacing some of the Jenkins lifecycle
management work that supervisord is currently performing, which is why I
mention Kubernetes in JEP-301 at all. However, I strongly
disagree that the market is "there" yet, I think making a container
orchestrator a requirement at this point would only inhibit adoption and
learning. I am instead erring on the side of "simple and fast" from a user
perspective, the closest we can get (IMHO) to the `java -jar jenkins.war`
experience, the easier the adoption and experimentation will be for the user
audience I want to reach.

I haven't found any references to supervisord in JEP pull request, neither about kubernetes - did I miss something ?
Anyway I tend to agree with this comment about packaging supervisord with jenkins essential within a docker image as a bad practice. I don't think this has anything to relate to running this as plain docker or with some orchestrator, this just makes the docker image a mutable instance, which should be avoided. Without much details on where this discussion comes from I can't really tell much about alternatives.  
 


> 2. Scripting should be done via yaml files, not environment variables(
> init.d/groovy). All essential plugins should be made compatible with
> configuration-as-code.

I fully intend on using as much support as possible from the Configuration as
Code effort which Ewelina and Nicolas are working on. I do not intend to
undertake, as part of Jenkins Essentials, patching or updating plugins in order
to support it however.

My order of priority for scripting Jenkins is:

 1. Configuration as Code
 2. Groovy scripting (bleh)
 3. Hopefully very little if any scary hacks.


So I think we're mostly on same page here :)


Configuration-as-Code is not ready yet to support this, but for sure we will keep an eye on this and will try to implement this use-case as well when this becomes relevant.
 

> 3. Pipeline while powerful is contrary to the goal of getting new users
> started quickly. From my personal experience, very few new users have the
> appetite to learn groovy scripting to setup their builds, they just want to
> something to run `npm test`.
>     A declarative yaml file like travisci,circle ect is the *correct*
> approach here.

I agree that this is an important goal for Jenkins as a whole to consider:
making Pipeline easier to use and be successful with. I believe those currently
hacking on Pipeline, like abayer, would agree with that sentiment. What *is* in
the scope of this work is making Jenkins easier to understand "out of the box"
(https://github.com/jenkinsci/jep/tree/master/jep/300#obvious-path) which I
believe is going to entail a number of copy-and-pasteable examples of common
Jenkinsfiles, and other forms of built-in examples and documentation.

Overall, thank you very much for taking the time to read JEPs and provide
feedback. The way I am interpreting your overall feedback, and please correct
me if I'm wrong, is that the direction is promising but want to encourage good
fundamentals on the implementation.



Anywho, thanks again :)





- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180214000422.hkio3rohqdgzn4pd%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJz%3DqW6dSdu6%2B5VriSLbxc7ZrZ1DJP1bY5mVg%2BihUbwt2WQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Evergreen packaging for Jenkins Essentials

R. Tyler Croy
In reply to this post by Jesse Glick-4
(replies inline)

On Wed, 14 Feb 2018, Jesse Glick wrote:

> On Tue, Feb 13, 2018 at 7:03 PM, R. Tyler Croy <[hidden email]> wrote:
> > I have been thus far deriving jenkins/evergreen from
> > jenkins/jenkins simply to inherit some scripts, permissions, and other bits.
>
> Yeah, that makes sense as a way to get things up and running quickly.
> Once we have an accepted process for tagging a ???release??? of ???Jenkins
> Essentials??? as a whole, as implied by
>
> https://github.com/jenkinsci/jep/blob/master/jep/300/README.adoc#auto-update
>
> then we can rewrite the image to handle core consistently.
>
> >> I do not think it is wise to try to automagically detect a cloud
> >> environment. [???]
> >
> > [???what are your concerns] regarding
> > a singular "all-in-one" package which can include the necessary cloud-detection
> > auto-configuration tooling?
>
> A general distrust of anything magical and opaque that might or might
> not work depending on factors too complex for users to see or
> understand. If we have a recommended way for Jenkins to run in AWS,
> polish it up and put it on Marketplace, if possible. If we have a
> recommended way for Jenkins to run in a self-managed Kubernetes
> cluster, publish an official Helm chart for it. But keep the basic
> Docker image, i.e., whatever happens when I run something like
>
> docker run --rm -v jh:/var/jenkins jenkins/evergreen
>
> straightforward and predictable.
>
> If you are worried about discoverability of our preconfigurations, you
> can always add an `AdministrativeMonitor` that says something like
>
> > Hey there! From sniffing around, it looks like you are running this Jenkins service in Azure, but you are not taking advantage of the preintegration work we have done for you.
> > For more information, see: https://azuremarketplace.microsoft.com/etc.
> > [[Dismiss]]


Yeah, this is where I thought you were going with the original feedback and I
agree, it's probably a good idea if for no other reason than to reduce the
opportunity for unintend side effects. I'll look at how I can work this nuance
into the JEP-300, I agree it's important to view different target platforms
(AWS, Azure, Kubernetes) differently and worth capturing that in this overview
document.



Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180214151009.paa3jmyl3n3o53ea%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Jenkins development with Jenkins Essentials

Jesse Glick-4
In reply to this post by R. Tyler Croy
On Tue, Feb 13, 2018 at 7:04 PM, R. Tyler Croy <[hidden email]> wrote:
> I think making a container
> orchestrator a requirement at this point would only inhibit adoption and
> learning. I am instead erring on the side of "simple and fast" from a user
> perspective, the closest we can get (IMHO) to the `java -jar jenkins.war`
> experience, the easier the adoption and experimentation will be

FTR I agree with this approach for the vanilla `jenkins/evergreen`
image: having a single, ready-to-run image trumps other
considerations. A `docker-run` command can in fact be _easier_ than
the traditional Jenkins installation experience because you do not
need to even download a WAR and figure out where to put it, install a
compatible JRE, etc.; or deal with native package management issues
such as version conflicts. We are closer to the old JNLP “Run Now!”
button.

If we have distinct packagings for more sophisticated environments, as
I recommended earlier, then we can define those packagings in a way
that more naturally fits those frameworks, for example by keeping the
supervisor and Evergreen client in distinct containers in a pod.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3Tm76SgUnxH_8TSktR%2BE0-7Oz38%2B7Gp65b%3D_e9Aesj3g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Jenkins development with Jenkins Essentials

R. Tyler Croy
In reply to this post by nicolas de loof-2
(replies inline)

On Wed, 14 Feb 2018, nicolas de loof wrote:

>
> I haven't found any references to supervisord in JEP pull request, neither
> about kubernetes - did I miss something ?
> Anyway I tend to agree with this comment about packaging supervisord with
> jenkins essential within a docker image as a bad practice. I don't think
> this has anything to relate to running this as plain docker or with some
> orchestrator, this just makes the docker image a mutable instance, which
> should be avoided. Without much details on where this discussion comes from
> I can't really tell much about alternatives.


Surya was referring to the second document in this thread which I sent out:
    https://github.com/jenkinsci/jep/tree/master/jep/301


I understand the "bad practice" argument against this approach but I haven't
yet seen a counter-proposal which accomplishes what jenkins/evergreen needs to
accomplish, without using supervisord in the "basic" container case, or a
Kubernetes-like thing in the "advanced" case. I'm personally comfortable with
being less dogmatic about what goes into the container, understanding that
there's a future on the horizon filled with Kubernetes :)


As to the concern about the image mutability, the evergreen-client would only
be effectively changing content in the mapped `/var/jenkins_home` which must by
the very nature of Jenkins be mutable. I'll update JEP-301 to make that more
explicit.



Thanks for taking a look-see.



- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20180214182600.56ebwim6vx44ljfk%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Jenkins development with Jenkins Essentials

nicolas de loof-2
Something I don't get here is why you need an external evergreen-client to manage updates, and can't just get this running from within Jenkins.

I also wonder how you manage jenkins.war version here, which is bundled in the docker image vs upgrades. Does this mean the image comes with a default jenkins.war but won't override when used on an existing JENKINS_HOME ? And then upgrading and restarting service won't actually reflect a core upgrade ?



2018-02-14 19:26 GMT+01:00 R. Tyler Croy <[hidden email]>:
(replies inline)

On Wed, 14 Feb 2018, nicolas de loof wrote:

>
> I haven't found any references to supervisord in JEP pull request, neither
> about kubernetes - did I miss something ?
> Anyway I tend to agree with this comment about packaging supervisord with
> jenkins essential within a docker image as a bad practice. I don't think
> this has anything to relate to running this as plain docker or with some
> orchestrator, this just makes the docker image a mutable instance, which
> should be avoided. Without much details on where this discussion comes from
> I can't really tell much about alternatives.


Surya was referring to the second document in this thread which I sent out:
    https://github.com/jenkinsci/jep/tree/master/jep/301


I understand the "bad practice" argument against this approach but I haven't
yet seen a counter-proposal which accomplishes what jenkins/evergreen needs to
accomplish, without using supervisord in the "basic" container case, or a
Kubernetes-like thing in the "advanced" case. I'm personally comfortable with
being less dogmatic about what goes into the container, understanding that
there's a future on the horizon filled with Kubernetes :)


As to the concern about the image mutability, the evergreen-client would only
be effectively changing content in the mapped `/var/jenkins_home` which must by
the very nature of Jenkins be mutable. I'll update JEP-301 to make that more
explicit.



Thanks for taking a look-see.



- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: [hidden email]

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJz%3DFG5p-eT8VE6SK3CA7SoOsap9KkYX1R3DC8mNXeHdZsg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
12