Out-of-order Jenkins 2.100 release with JENKINS-48761/JENKINS-48754 fixes

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Out-of-order Jenkins 2.100 release with JENKINS-48761/JENKINS-48754 fixes

Oleg Nenashev

Hi all,

In Jenkins 2.98 before the new year I have integrated Remoting 3.15 into the weekly. In this release I introduced some new convenience API methods for hudson.remoting.Callables, and in PR #3145 I adopted this APIs in MasterToSlaveCallable implementations. No wonder instances with older Remoting versions on agents fell apart... It caused JENKINS-48761 and most likely JENKINS-48754.

SSHLauncher-only instances were fine + there is a workaround (update Remoting on agents to version 3.15). Docker agent images and Swarm Plugin Client with Remoting 3.15 have been also released along with Jenkins 2.98... But the community ratings are pretty bad (33 positives, 15 negatives), so after the discussion we decided to go forward with the out-of-order release. https://github.com/jenkinsci/jenkins/pull/3212 with a hotfix has been integrated, I hope to get it released soon.

What I did:
  • Fixed breaking API usages non-compatible with old Remoting versions
  • Created tests for old agent connection possibility within the Jenkins core test suite. Now we have smoke tests, which at least confirms that the agent is able to connect, execute jobs and pass node monitors
  • Currently the tests run against Remoting 2.62, which is 1.5 years old
Next action items:
  • Extend Remoting integration testing in the core
    • For now - just extra tests to improve coverage of MasterToSlaveCallables and other Remoting-related logic
    • In longer term I would like to have a "Remoting Compatibility Tester" test suite
  • JENKINS-48766 - Introduce the "Minimum Supported Remoting Version" in the core and in Remoting
    • Current state: Any Remoting version can theoretically connect if one of its protocols is accepted by master. Nobody knows when it will fail due to API compatibility in master-to-agent calls
    • With the proposed change Jenkins admins will start getting warnings about unsupported Remoting versions, so that they will be able to proactively investigate the issues.
    • The change will help us to setup more testing and maybe advanced static analysis for API compatibility
  • Revisit Remoting Upgradeability stories (JENKINS-44099), which could have prevented the issue by keeping agents up to date
My apologies for any inconvenience I caused. As always, I highly recommend using LTS on production instances to avoid the boilerplate risks in weekly releases. If you have any feedback/questions, please do not hesitate to respond to this thread or to send me a private message.

Best regards,

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5caad75d-6bf0-45dc-8a5f-eb2aad148995%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.