Java 11 Preview availability in Weekly releases. Status update (JEP-211)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Java 11 Preview availability in Weekly releases. Status update (JEP-211)

Oleg Nenashev
Dear all,

I would like to briefly summarize the status of the ongoing Java 11 support effort in the Jenkins project. Java 11 has been released on Sep 25, and this is an LTS Java version which will stay around for many years. In the Jenkins project we are interested to support this version, and there has been a good progress since we have started this effort in June 2018. There is a JEP-211 document describing the process.

Today we have discussed the rollout plan at the Governance meeting, and our I would like to start a thread before the tomorrow's Platform SIG meeting. Our current goal is the Preview Availability in Jenkins weekly releases (EPIC - JENKINS-52012), and I think we could announce such availability on the next week. The goal of this milestone is to announce Java 11 support preview among Jenkins contributors, plugin maintainers and early adopters. It is not a general availability, and Java 11 will stay protected by a feature flag for now. See the Rollout Plan for details.

Current status
  • By Nov 19 2018 we have integrated all Jenkins core patches we planned for the preview availability. The results were presented at the Platform SIG meeting (meeting notes, status update slides)
  • All major patches have been integrated into Jenkins Jenkins 2.145, so technically it is possible to run the current LTS with Java 11 if you pass the feature flag and libraries properly (old guidelines)
  • We have started a Wiki page for known compatibility issues. Thanks Baptiste!
  • At that call on Nov 19 we identified 3 main blockers for the preview availability, see below
  • Blocker 1. Pipeline: Support plugin release with Java 11 compatibility patches (JENKINS-52187)
    • The final patch is in progress (thanks Sam), but we plan to deploy a workaround for now: workflow-support-plugin/pull/82 . We will release it only as an experimental release for Java11-only
    • To deploy the patch, we will create a temporary update center for Java 11 experimental releases (INFRA-1870). The approach is confirmed with the Jenkins infrastructure team members
    • Users of standard and experimental Jenkins update centers will not see the experimental release, because it will be filtered out there. JEP-211 will document the process
  • Blocker 2. Docker packaging - OK
    • Jenkins master and 3 agent packages now offer the "jdk11" tags as well as versioned releases (e.g. "2.154-jdk11"). Thanks to Baptiste, Carlos and Olivier for that!
    • There is an open action item for the BlueOcean packaging, but we ship experimental versions in https://hub.docker.com/r/jenkins/jenkins-experimental/ based on this repo. It is enough for the preview availability announcement
  • Blocker 3. Testing and test automation - OK
    • Jenkins Core CI flow includes testing with JDK 11 for almost 1 month (link)
    • Jenkins Acceptance Test Harness now runs for both Java 8 and Java 11, thanks to Lucie, Oliver and Ramon for this effort. There is no major regressions discovered (link)
    • Jenkins Plugin POMs have been updated to support building plugins with OpenJDK11. Plugin POM 3.29 is recommended
    • Jenkins Pipeline Library was updated to support testing with Java 11 in buildPlugin() and other steps (example)
    • Plugin Compatibility Tester now supports Java 11 in both CLI and Docker modes, we are executing tests against the recommended plugins. So far so good
    • There was a LOT of exploratory testing performed, starting Java 10 and 11-ea testing in June 2017. We confirmed that all major functionality works correctly on both Linux and Windows. In particular, plugins like Pipeline, JobDSL, JCasC were tested. Maven Integration plugin also works well with Java 11 (but it will require builds with OpenJDK 11). Although we discovered some minor glitches, Jenkins seems to operate well on Java 11, barring the Pipeline: Support patch delivery
  • What else has happened?
    • Mark Waite has delivered a fix to Jenkins Core so that Java Web Start interface does not appear in Java 11, because Javaws was removed from it (JENKINS-52282)
    • Adrien and Baptiste has prototyped Java 11 support in Jenkins Evergreen, see the video (demo recording)
    • There were experiments around Java 11 support in other components like Jenkinsfile Runner, but it is not in the scope for preview availability
Once Java 11 support is released, we expect a number regressions in plugins and Jenkins core. One of the concerns are exotic platforms with native libraries, and of course other Java versions. There is also a risk of 3rd-party library incompatibilities with Java 11. To mitigate the risks, we take the following steps...
  1. Creating a Known Java 11 compatibility issues page, like it has been done before for changes like JEP-200 (wiki)
  2. Forming a Java 11 support team. This team will operate within Platform SIG, and it will be providing help with issues related to Java 11, including bug triage and pull request reviews and, if time allows, fixes in the core and critical plugins
  3. Starting preview availability to get more adoption and feedback before the general availability is announced for Jenkins weekly. This is why this thread is started
  4. Providing developer documentation so that Jenkins contributors can verify, investigate and fix their plugins. We will also provide a temporary experimental update center for Java 11 so that plugin developers can stage their fixes for evaluation without impacting Java 8 users. This documentation is to be published next week
  5. Posting blogposts and scheduling a Jenkins Online Meetup to talk about Java 11 preview availability so that early adopters get more information about the process (usage, known issues and risks, reporting guidelines, etc.)
Taking the current status, I propose to go forward with Java 11 Preview availability next week. It includes accepting the JEP-211 and the announcements mentioned above. We will appreciate your feedback in the JEP and in developer mailing list. If you are interested to provide more feedback about the JEP, please do it by Monday, Dec 10.

If you are interested to know more about Java 11 support details, please join the Platform SIG channel. We will also have a SIG meeting tomorrow at 2PM UTC, the video will be broadcasted in YouTube (link). We will appreciate any feedback.

Thanks in advance,
Oleg Nenashev
Platform SIG leader and JEP-211 sponsor


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLC9Jir4-6rY8kteF03SudBEjAOGeU-Lfy34zU52Vb9FXg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Java 11 Preview availability in Weekly releases. Status update (JEP-211)

Oleg Nenashev

Hi all,


At the platform SIG meeting on Dec 04 we have signed-off the preview availability release for Java 11 support with Jenkins and decided to go forward with it (meeting notes). Currently the Java 11 support team is preparing the final bits, mostly around documentation. We have scheduled a Jenkins Online Meetup to announce the Java 11 preview availability. It will happen on Dec 18, 4PM UTC. Meetup page is here.


During the last week we have addressed some feedback about JEP-211. This feedback has been addressed in the JEP over the last week:

  • Experimental Update Center for Java 11 was explicitly documented as it was agreed with the INFRA team (reasoning and implementation)

  • Security section was updated to accommodate the feedback from Daniel Beck and Wadeck Follonier (link)

  • Developer Tool sections have been updated to reflect changes we needed to deliver the preview availability (Maven HPI Plugin, Plugin POM, etc.)

  • Added explicit reasoning for Java 10 and Java 12+ support to address comment from Samuel Gabriel in JENKINS-51998. These versions will not be supported as a part of JEP-211 (reasoning)

  • Integrated copy-edit patches from Liam Newman

As a JEP-211 BDFL Delegate, I think that the JEP got enough feedback and discussion since June 2018. The Platform SIG meeting also signed off accepting the JEP at the meeting. So I made the decision to accept JEP-211. This is a process JEP, and the scope may be slightly adjusted in EPICs going forward (GA in Weekly and LTS). Thanks to everybody for the feedback!


Just to summarize the status of the release itself, diff from the previous email.

  • Blocker 1. Pipeline: Support plugin (JENKINS-52187) - OK

    • Java 11 experimental update center has been deployed. Thanks to Daniel Beck for his help!

    • Jenkins 2.155 and Docker packaging have been updated to use the Experimental update center by default

    • Pipeline: Support plugin with the fix has been released to the update center. Version: 3.0-java11-alpha-1

  • Blocker 2. Docker packaging - OK

    • Jenkins 2.155 releases have been successfully deployed to Docker

    • Experimental BlueOcean packaging has been updated to 2.155. I have also set up a CD flow for the images. See the documentation here

  • Blocker 3. Testing and test automation - OK

    • We have continued the Plugin Compat Tester testing for Java 11 towards the Weekly GA. We hit some serious issues in the tooling (e.g. SUREFIRE-1588 effectively makes the recent PCT Docker image useless if plugins are not updated to Plugin POM 3.28 or above). As a workaround we run PCT locally (Adrien, Kevin, Oleg), but we have other issues there

    • We will surely need some updates in PCT to create a fully automated flow, to be scheduled towards Java 11 GA in weekly

    • There is ongoing work to update Pipeline Library so that runATH() and essentialsTest() can be executed with Java 11. we plan to complete it towards the Java 11 test automation

  • Documentation - OK

There is no other tasks left in JENKINS-52012 except one minor infra ticket, so I believe we can go forward with the preview announcements once the blog post is fully reviewed and ready to be published.


Best regards,

Oleg




On Thu, Dec 6, 2018 at 1:26 AM Oleg Nenashev <[hidden email]> wrote:
Dear all,

I would like to briefly summarize the status of the ongoing Java 11 support effort in the Jenkins project. Java 11 has been released on Sep 25, and this is an LTS Java version which will stay around for many years. In the Jenkins project we are interested to support this version, and there has been a good progress since we have started this effort in June 2018. There is a JEP-211 document describing the process.

Today we have discussed the rollout plan at the Governance meeting, and our I would like to start a thread before the tomorrow's Platform SIG meeting. Our current goal is the Preview Availability in Jenkins weekly releases (EPIC - JENKINS-52012), and I think we could announce such availability on the next week. The goal of this milestone is to announce Java 11 support preview among Jenkins contributors, plugin maintainers and early adopters. It is not a general availability, and Java 11 will stay protected by a feature flag for now. See the Rollout Plan for details.

Current status
  • By Nov 19 2018 we have integrated all Jenkins core patches we planned for the preview availability. The results were presented at the Platform SIG meeting (meeting notes, status update slides)
  • All major patches have been integrated into Jenkins Jenkins 2.145, so technically it is possible to run the current LTS with Java 11 if you pass the feature flag and libraries properly (old guidelines)
  • We have started a Wiki page for known compatibility issues. Thanks Baptiste!
  • At that call on Nov 19 we identified 3 main blockers for the preview availability, see below
  • Blocker 1. Pipeline: Support plugin release with Java 11 compatibility patches (JENKINS-52187)
    • The final patch is in progress (thanks Sam), but we plan to deploy a workaround for now: workflow-support-plugin/pull/82 . We will release it only as an experimental release for Java11-only
    • To deploy the patch, we will create a temporary update center for Java 11 experimental releases (INFRA-1870). The approach is confirmed with the Jenkins infrastructure team members
    • Users of standard and experimental Jenkins update centers will not see the experimental release, because it will be filtered out there. JEP-211 will document the process
  • Blocker 2. Docker packaging - OK
    • Jenkins master and 3 agent packages now offer the "jdk11" tags as well as versioned releases (e.g. "2.154-jdk11"). Thanks to Baptiste, Carlos and Olivier for that!
    • There is an open action item for the BlueOcean packaging, but we ship experimental versions in https://hub.docker.com/r/jenkins/jenkins-experimental/ based on this repo. It is enough for the preview availability announcement
  • Blocker 3. Testing and test automation - OK
    • Jenkins Core CI flow includes testing with JDK 11 for almost 1 month (link)
    • Jenkins Acceptance Test Harness now runs for both Java 8 and Java 11, thanks to Lucie, Oliver and Ramon for this effort. There is no major regressions discovered (link)
    • Jenkins Plugin POMs have been updated to support building plugins with OpenJDK11. Plugin POM 3.29 is recommended
    • Jenkins Pipeline Library was updated to support testing with Java 11 in buildPlugin() and other steps (example)
    • Plugin Compatibility Tester now supports Java 11 in both CLI and Docker modes, we are executing tests against the recommended plugins. So far so good
    • There was a LOT of exploratory testing performed, starting Java 10 and 11-ea testing in June 2017. We confirmed that all major functionality works correctly on both Linux and Windows. In particular, plugins like Pipeline, JobDSL, JCasC were tested. Maven Integration plugin also works well with Java 11 (but it will require builds with OpenJDK 11). Although we discovered some minor glitches, Jenkins seems to operate well on Java 11, barring the Pipeline: Support patch delivery
  • What else has happened?
    • Mark Waite has delivered a fix to Jenkins Core so that Java Web Start interface does not appear in Java 11, because Javaws was removed from it (JENKINS-52282)
    • Adrien and Baptiste has prototyped Java 11 support in Jenkins Evergreen, see the video (demo recording)
    • There were experiments around Java 11 support in other components like Jenkinsfile Runner, but it is not in the scope for preview availability
Once Java 11 support is released, we expect a number regressions in plugins and Jenkins core. One of the concerns are exotic platforms with native libraries, and of course other Java versions. There is also a risk of 3rd-party library incompatibilities with Java 11. To mitigate the risks, we take the following steps...
  1. Creating a Known Java 11 compatibility issues page, like it has been done before for changes like JEP-200 (wiki)
  2. Forming a Java 11 support team. This team will operate within Platform SIG, and it will be providing help with issues related to Java 11, including bug triage and pull request reviews and, if time allows, fixes in the core and critical plugins
  3. Starting preview availability to get more adoption and feedback before the general availability is announced for Jenkins weekly. This is why this thread is started
  4. Providing developer documentation so that Jenkins contributors can verify, investigate and fix their plugins. We will also provide a temporary experimental update center for Java 11 so that plugin developers can stage their fixes for evaluation without impacting Java 8 users. This documentation is to be published next week
  5. Posting blogposts and scheduling a Jenkins Online Meetup to talk about Java 11 preview availability so that early adopters get more information about the process (usage, known issues and risks, reporting guidelines, etc.)
Taking the current status, I propose to go forward with Java 11 Preview availability next week. It includes accepting the JEP-211 and the announcements mentioned above. We will appreciate your feedback in the JEP and in developer mailing list. If you are interested to provide more feedback about the JEP, please do it by Monday, Dec 10.

If you are interested to know more about Java 11 support details, please join the Platform SIG channel. We will also have a SIG meeting tomorrow at 2PM UTC, the video will be broadcasted in YouTube (link). We will appreciate any feedback.

Thanks in advance,
Oleg Nenashev
Platform SIG leader and JEP-211 sponsor


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLDtfdvQKRdEFkF2TX0nCe7iEOBoncSDDBu04-MwnaE_eQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Java 11 Preview availability in Weekly releases. Status update (JEP-211)

Jesse Glick-4
On Tue, Dec 11, 2018 at 1:08 PM Oleg Nenashev <[hidden email]> wrote:
> SUREFIRE-1588 effectively makes the recent PCT Docker image useless if plugins are not updated to Plugin POM 3.28 or above

You do not need to update plugin POMs; you can work around this with a
`~/.m2/settings.xml` in the image. (Or simply adjust the image to use
a different base image that incorporates a non-broken Java version.)

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2%2B1SFN5G-aBc6zmUFe-hvYyXw0%2BtPjZ2MhPRtbptNi8w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Java 11 Preview availability in Weekly releases. Status update (JEP-211)

Oleg Nenashev
It was fixed slightly differently in https://github.com/jenkinsci/plugin-compat-tester/pull/93 . Overriding argline is not cool, but it does the job for a tool like PCT.
We still hit a number of issues while running PCT with Java 11 in Docker (especially the "Stream stdin corrupted. Expected comma after third character in command " fun we see in many places now), but we are moving forward

BR, Oleg

On Tuesday, December 11, 2018 at 9:04:41 PM UTC+1, Jesse Glick wrote:
On Tue, Dec 11, 2018 at 1:08 PM Oleg Nenashev <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="D_HuexiNCAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">o.v.ne...@...> wrote:
> SUREFIRE-1588 effectively makes the recent PCT Docker image useless if plugins are not updated to Plugin POM 3.28 or above

You do not need to update plugin POMs; you can work around this with a
`~/.m2/settings.xml` in the image. (Or simply adjust the image to use
a different base image that incorporates a non-broken Java version.)

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/e3b46430-d572-497b-b0c6-876314227a64%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.