Upgrading Digital Ocean Cloud Plugin - First run always fails

[Gavin Mogan]

I'm trying to fix the plain text credentials issue in the digital ocean plugin. I've updated it to use credentials, on to testing it, and having an agent issue. The droplet spins up, it connects, the job runs, and then errors with a stacktrace (at the end of email). Then if i run it again it runs on the same agent just fine without error.

Its complaining about `java.lang.ClassNotFoundException: javax.servlet.ServletException` does this mean that some do api is returning a servletexception and then remoting can't seralize it? or is it a known error with 2.222.4? am I missing the plugins somewhere in these stacktraces?

Gavin

Started by user unknown or anonymous
Running as SYSTEM
Building remotely on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff in workspace /jenkins/workspace/testdo
[testdo] $ /bin/sh -xe /tmp/jenkins4548124856065621833.sh
+ whoami
root
+ hostname
jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
+ uptime
 07:03:51 up 1 min,  0 users,  load average: 2.58, 1.11, 0.42
+ uname -a
Linux jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
FATAL: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to channel
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:288)
		at com.sun.proxy.$Proxy5.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:317)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
		at hudson.util.ProcessTree.get(ProcessTree.java:432)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:811)
	at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:931)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:998)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1856)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:432)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
	at hudson.remoting.Channel.call(Channel.java:1004)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1856)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
Finished: FAILURE

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DuuJiekO12Q0yvKkTQd-%3DeYKwDVSSnLU3JHwZg%3DBicVjLQ%40mail.gmail.com.

[Oleg Nenashev]

It is a known error fixed in 2.235.

https://issues.jenkins.io/browse/JENKINS-62006

On Thursday, February 11, 2021 at 8:19:30 AM UTC+1 [hidden email] wrote:

I'm trying to fix the plain text credentials issue in the digital ocean plugin. I've updated it to use credentials, on to testing it, and having an agent issue. The droplet spins up, it connects, the job runs, and then errors with a stacktrace (at the end of email). Then if i run it again it runs on the same agent just fine without error.

Its complaining about `java.lang.ClassNotFoundException: javax.servlet.ServletException` does this mean that some do api is returning a servletexception and then remoting can't seralize it? or is it a known error with 2.222.4? am I missing the plugins somewhere in these stacktraces?

Gavin

Started by user unknown or anonymous
Running as SYSTEM
Building remotely on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff in workspace /jenkins/workspace/testdo
[testdo] $ /bin/sh -xe /tmp/jenkins4548124856065621833.sh
+ whoami
root
+ hostname
jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
+ uptime
 07:03:51 up 1 min,  0 users,  load average: 2.58, 1.11, 0.42
+ uname -a
Linux jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
FATAL: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to channel
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:288)
		at com.sun.proxy.$Proxy5.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:317)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
		at hudson.util.ProcessTree.get(ProcessTree.java:432)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:811)
	at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:931)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:998)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1856)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:432)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
	at hudson.remoting.Channel.call(Channel.java:1004)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1856)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
Finished: FAILURE

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ca060873-eeba-4fc4-baa5-6e38aab7f597n%40googlegroups.com.

[Gavin Mogan]

I've tried it with 2.235.5 and 2.263.4 with the same stacktrace.

My PR is up at https://github.com/jenkinsci/digitalocean-plugin/pull/50 I don't see anything I might have changed related to it. I'll try it again without my changes to see if i caused it. I looked at the PR linked inside the jira ticket and it looks like its only core changes.

Gavin

On Thu, Feb 11, 2021 at 12:53 AM Oleg Nenashev <[hidden email]> wrote:

It is a known error fixed in 2.235.

https://issues.jenkins.io/browse/JENKINS-62006

On Thursday, February 11, 2021 at 8:19:30 AM UTC+1 [hidden email] wrote:

I'm trying to fix the plain text credentials issue in the digital ocean plugin. I've updated it to use credentials, on to testing it, and having an agent issue. The droplet spins up, it connects, the job runs, and then errors with a stacktrace (at the end of email). Then if i run it again it runs on the same agent just fine without error.

Its complaining about `java.lang.ClassNotFoundException: javax.servlet.ServletException` does this mean that some do api is returning a servletexception and then remoting can't seralize it? or is it a known error with 2.222.4? am I missing the plugins somewhere in these stacktraces?

Gavin

Started by user unknown or anonymous
Running as SYSTEM
Building remotely on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff in workspace /jenkins/workspace/testdo
[testdo] $ /bin/sh -xe /tmp/jenkins4548124856065621833.sh
+ whoami
root
+ hostname
jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
+ uptime
 07:03:51 up 1 min,  0 users,  load average: 2.58, 1.11, 0.42
+ uname -a
Linux jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
FATAL: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to channel
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:288)
		at com.sun.proxy.$Proxy5.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:317)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
		at hudson.util.ProcessTree.get(ProcessTree.java:432)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:811)
	at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:931)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:998)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1856)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:432)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
	at hudson.remoting.Channel.call(Channel.java:1004)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1856)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
Finished: FAILURE

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ca060873-eeba-4fc4-baa5-6e38aab7f597n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DuuHQUF1mDx8k9PGVqG4x_jeBse-RpZSFPtAatfSLSybzg%40mail.gmail.com.

[Gavin Mogan]

Okay managed to do a bunch of permutations.

2.60.3 works just fine on both first and second freestyle job

2.263.4 fails on first job, but will succeed on the second if the agent is still running

2.263.4 works fine on first and second with pipeline

https://github.com/jenkinsci/remoting/blob/remoting-4.5/src/main/java/hudson/remoting/Channel.java#L1001

UserRequest.getClassLoader(callable) where callable is Launcher$RemoteLauncher$KillTask returns a WebAppClassLoader. Is that possibly why its trying to load ServletError?

I've been trying to attach a debugger and track this down more but Its well outside my java knowledge.

How can I get a debugger inside of a MasterToSlaveCallable, specifically https://github.com/jenkinsci/jenkins/blob/8c6327f4c6777d135e374e3d507ba9c20820bfc6/core/src/main/java/hudson/Launcher.java#L1164

I've tried attaching it to the hpi:run process, and the agent/remoting.jar process. I think the remoting process is the way to do it, but I can't figure out how to get a breakpoint inside of killtask

On Thu, Feb 11, 2021 at 8:11 PM Gavin Mogan <[hidden email]> wrote:

I've tried it with 2.235.5 and 2.263.4 with the same stacktrace.

My PR is up at https://github.com/jenkinsci/digitalocean-plugin/pull/50 I don't see anything I might have changed related to it. I'll try it again without my changes to see if i caused it. I looked at the PR linked inside the jira ticket and it looks like its only core changes.

Gavin

On Thu, Feb 11, 2021 at 12:53 AM Oleg Nenashev <[hidden email]> wrote:

It is a known error fixed in 2.235.

https://issues.jenkins.io/browse/JENKINS-62006

On Thursday, February 11, 2021 at 8:19:30 AM UTC+1 [hidden email] wrote:

I'm trying to fix the plain text credentials issue in the digital ocean plugin. I've updated it to use credentials, on to testing it, and having an agent issue. The droplet spins up, it connects, the job runs, and then errors with a stacktrace (at the end of email). Then if i run it again it runs on the same agent just fine without error.

Its complaining about `java.lang.ClassNotFoundException: javax.servlet.ServletException` does this mean that some do api is returning a servletexception and then remoting can't seralize it? or is it a known error with 2.222.4? am I missing the plugins somewhere in these stacktraces?

Gavin

Started by user unknown or anonymous
Running as SYSTEM
Building remotely on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff in workspace /jenkins/workspace/testdo
[testdo] $ /bin/sh -xe /tmp/jenkins4548124856065621833.sh
+ whoami
root
+ hostname
jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
+ uptime
 07:03:51 up 1 min,  0 users,  load average: 2.58, 1.11, 0.42
+ uname -a
Linux jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
FATAL: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to channel
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:288)
		at com.sun.proxy.$Proxy5.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:317)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
		at hudson.util.ProcessTree.get(ProcessTree.java:432)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:811)
	at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:931)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:998)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1856)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:432)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
	at hudson.remoting.Channel.call(Channel.java:1004)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1856)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
Finished: FAILURE

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ca060873-eeba-4fc4-baa5-6e38aab7f597n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DuvJTqzQ86UhUY6yNCgZz7C-hNg9_-O4VTKPP2OgOv0%2BWQ%40mail.gmail.com.

[Gavin Mogan]

The patch is merged, but before I cut a release, I'd like to address the freestyle issue.

Is anyone available to pair with me to diagnose this? Or can I get someone credit to look into it?

On Sat, Feb 13, 2021 at 7:10 PM Gavin Mogan <[hidden email]> wrote:

Okay managed to do a bunch of permutations.

2.60.3 works just fine on both first and second freestyle job

2.263.4 fails on first job, but will succeed on the second if the agent is still running

2.263.4 works fine on first and second with pipeline

https://github.com/jenkinsci/remoting/blob/remoting-4.5/src/main/java/hudson/remoting/Channel.java#L1001

UserRequest.getClassLoader(callable) where callable is Launcher$RemoteLauncher$KillTask returns a WebAppClassLoader. Is that possibly why its trying to load ServletError?

I've been trying to attach a debugger and track this down more but Its well outside my java knowledge.

How can I get a debugger inside of a MasterToSlaveCallable, specifically https://github.com/jenkinsci/jenkins/blob/8c6327f4c6777d135e374e3d507ba9c20820bfc6/core/src/main/java/hudson/Launcher.java#L1164

I've tried attaching it to the hpi:run process, and the agent/remoting.jar process. I think the remoting process is the way to do it, but I can't figure out how to get a breakpoint inside of killtask

On Thu, Feb 11, 2021 at 8:11 PM Gavin Mogan <[hidden email]> wrote:

I've tried it with 2.235.5 and 2.263.4 with the same stacktrace.

My PR is up at https://github.com/jenkinsci/digitalocean-plugin/pull/50 I don't see anything I might have changed related to it. I'll try it again without my changes to see if i caused it. I looked at the PR linked inside the jira ticket and it looks like its only core changes.

Gavin

On Thu, Feb 11, 2021 at 12:53 AM Oleg Nenashev <[hidden email]> wrote:

It is a known error fixed in 2.235.

https://issues.jenkins.io/browse/JENKINS-62006

On Thursday, February 11, 2021 at 8:19:30 AM UTC+1 [hidden email] wrote:

I'm trying to fix the plain text credentials issue in the digital ocean plugin. I've updated it to use credentials, on to testing it, and having an agent issue. The droplet spins up, it connects, the job runs, and then errors with a stacktrace (at the end of email). Then if i run it again it runs on the same agent just fine without error.

Its complaining about `java.lang.ClassNotFoundException: javax.servlet.ServletException` does this mean that some do api is returning a servletexception and then remoting can't seralize it? or is it a known error with 2.222.4? am I missing the plugins somewhere in these stacktraces?

Gavin

Started by user unknown or anonymous
Running as SYSTEM
Building remotely on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff in workspace /jenkins/workspace/testdo
[testdo] $ /bin/sh -xe /tmp/jenkins4548124856065621833.sh
+ whoami
root
+ hostname
jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
+ uptime
 07:03:51 up 1 min,  0 users,  load average: 2.58, 1.11, 0.42
+ uname -a
Linux jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
FATAL: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to channel
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:288)
		at com.sun.proxy.$Proxy5.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:317)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
		at hudson.util.ProcessTree.get(ProcessTree.java:432)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:811)
	at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:931)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:998)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1856)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:432)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
	at hudson.remoting.Channel.call(Channel.java:1004)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1856)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
Finished: FAILURE

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ca060873-eeba-4fc4-baa5-6e38aab7f597n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DusTywAs19iLx1ZrBtAuZVvdEEFRk8P5QKgKRmXbpmKeFw%40mail.gmail.com.

[Tim Jacomb]

I've had a bit of a look at this,

I patched the plugin to enable logging via workDir, unfortunately it didn't give much more than what's in the log for the freestyle build. (https://gist.github.com/timja/4b28f173d5d2925c82781109dce57362)

I had a look through these slides from Oleg https://speakerdeck.com/onenashev/day-of-jenkins-2017-dealing-with-agent-connectivity-issues?slide=66

But it didn't help any more than say enable logging

FTR I've got a very similar error in the slack plugin on freestyle builds when uploading a file from the agent.

Anyone got any tips for debugging this? Specific logs to enable and how to enable them?

The agent is started here:

https://github.com/jenkinsci/digitalocean-plugin/blob/ca1112c5c52a842c6340401f02f536fc07c6620a/src/main/java/com/dubture/jenkins/digitalocean/DigitalOceanComputerLauncher.java#L190

I patched it to be:

+            String launchString = "java " + jvmOpts + " -jar /tmp/agent.jar -workDir /root/.jenkins/remotingWorkDir";

Thanks

Tim

On Wed, 24 Feb 2021 at 21:23, 'Gavin Mogan' via Jenkins Developers <[hidden email]> wrote:

The patch is merged, but before I cut a release, I'd like to address the freestyle issue.

Is anyone available to pair with me to diagnose this? Or can I get someone credit to look into it?

On Sat, Feb 13, 2021 at 7:10 PM Gavin Mogan <[hidden email]> wrote:

Okay managed to do a bunch of permutations.

2.60.3 works just fine on both first and second freestyle job

2.263.4 fails on first job, but will succeed on the second if the agent is still running

2.263.4 works fine on first and second with pipeline

https://github.com/jenkinsci/remoting/blob/remoting-4.5/src/main/java/hudson/remoting/Channel.java#L1001

UserRequest.getClassLoader(callable) where callable is Launcher$RemoteLauncher$KillTask returns a WebAppClassLoader. Is that possibly why its trying to load ServletError?

I've been trying to attach a debugger and track this down more but Its well outside my java knowledge.

How can I get a debugger inside of a MasterToSlaveCallable, specifically https://github.com/jenkinsci/jenkins/blob/8c6327f4c6777d135e374e3d507ba9c20820bfc6/core/src/main/java/hudson/Launcher.java#L1164

I've tried attaching it to the hpi:run process, and the agent/remoting.jar process. I think the remoting process is the way to do it, but I can't figure out how to get a breakpoint inside of killtask

On Thu, Feb 11, 2021 at 8:11 PM Gavin Mogan <[hidden email]> wrote:

I've tried it with 2.235.5 and 2.263.4 with the same stacktrace.

My PR is up at https://github.com/jenkinsci/digitalocean-plugin/pull/50 I don't see anything I might have changed related to it. I'll try it again without my changes to see if i caused it. I looked at the PR linked inside the jira ticket and it looks like its only core changes.

Gavin

On Thu, Feb 11, 2021 at 12:53 AM Oleg Nenashev <[hidden email]> wrote:

It is a known error fixed in 2.235.

https://issues.jenkins.io/browse/JENKINS-62006

On Thursday, February 11, 2021 at 8:19:30 AM UTC+1 [hidden email] wrote:

I'm trying to fix the plain text credentials issue in the digital ocean plugin. I've updated it to use credentials, on to testing it, and having an agent issue. The droplet spins up, it connects, the job runs, and then errors with a stacktrace (at the end of email). Then if i run it again it runs on the same agent just fine without error.

Its complaining about `java.lang.ClassNotFoundException: javax.servlet.ServletException` does this mean that some do api is returning a servletexception and then remoting can't seralize it? or is it a known error with 2.222.4? am I missing the plugins somewhere in these stacktraces?

Gavin

Started by user unknown or anonymous
Running as SYSTEM
Building remotely on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff in workspace /jenkins/workspace/testdo
[testdo] $ /bin/sh -xe /tmp/jenkins4548124856065621833.sh
+ whoami
root
+ hostname
jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
+ uptime
 07:03:51 up 1 min,  0 users,  load average: 2.58, 1.11, 0.42
+ uname -a
Linux jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
FATAL: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to channel
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:288)
		at com.sun.proxy.$Proxy5.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:317)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
		at hudson.util.ProcessTree.get(ProcessTree.java:432)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:811)
	at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:931)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:998)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1856)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:432)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
	at hudson.remoting.Channel.call(Channel.java:1004)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1856)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
Finished: FAILURE

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ca060873-eeba-4fc4-baa5-6e38aab7f597n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DusTywAs19iLx1ZrBtAuZVvdEEFRk8P5QKgKRmXbpmKeFw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAH-3BidSCOCNEv%3Dw0uUv3GSjGK-%3D7uZeHhmQiLPo1iO%3Dcwgr9g%40mail.gmail.com.

[Gavin Mogan]

Okay if anyone else wants to try, you can use my referral link and get yourself a bunch of free credits.https://m.do.co/c/7d6859326b6a

I'm just going to mark it as a known issue in the release notes as I don't have any idea how to debug this any more.

Gavin

On Thu, Feb 25, 2021 at 12:54 AM Tim Jacomb <[hidden email]> wrote:

I've had a bit of a look at this,

I patched the plugin to enable logging via workDir, unfortunately it didn't give much more than what's in the log for the freestyle build. (https://gist.github.com/timja/4b28f173d5d2925c82781109dce57362)

I had a look through these slides from Oleg https://speakerdeck.com/onenashev/day-of-jenkins-2017-dealing-with-agent-connectivity-issues?slide=66

But it didn't help any more than say enable logging

FTR I've got a very similar error in the slack plugin on freestyle builds when uploading a file from the agent.

Anyone got any tips for debugging this? Specific logs to enable and how to enable them?

The agent is started here:

https://github.com/jenkinsci/digitalocean-plugin/blob/ca1112c5c52a842c6340401f02f536fc07c6620a/src/main/java/com/dubture/jenkins/digitalocean/DigitalOceanComputerLauncher.java#L190

I patched it to be:

+            String launchString = "java " + jvmOpts + " -jar /tmp/agent.jar -workDir /root/.jenkins/remotingWorkDir";

Thanks

Tim

On Wed, 24 Feb 2021 at 21:23, 'Gavin Mogan' via Jenkins Developers <[hidden email]> wrote:

The patch is merged, but before I cut a release, I'd like to address the freestyle issue.

Is anyone available to pair with me to diagnose this? Or can I get someone credit to look into it?

On Sat, Feb 13, 2021 at 7:10 PM Gavin Mogan <[hidden email]> wrote:

Okay managed to do a bunch of permutations.

2.60.3 works just fine on both first and second freestyle job

2.263.4 fails on first job, but will succeed on the second if the agent is still running

2.263.4 works fine on first and second with pipeline

https://github.com/jenkinsci/remoting/blob/remoting-4.5/src/main/java/hudson/remoting/Channel.java#L1001

UserRequest.getClassLoader(callable) where callable is Launcher$RemoteLauncher$KillTask returns a WebAppClassLoader. Is that possibly why its trying to load ServletError?

I've been trying to attach a debugger and track this down more but Its well outside my java knowledge.

How can I get a debugger inside of a MasterToSlaveCallable, specifically https://github.com/jenkinsci/jenkins/blob/8c6327f4c6777d135e374e3d507ba9c20820bfc6/core/src/main/java/hudson/Launcher.java#L1164

I've tried attaching it to the hpi:run process, and the agent/remoting.jar process. I think the remoting process is the way to do it, but I can't figure out how to get a breakpoint inside of killtask

On Thu, Feb 11, 2021 at 8:11 PM Gavin Mogan <[hidden email]> wrote:

I've tried it with 2.235.5 and 2.263.4 with the same stacktrace.

My PR is up at https://github.com/jenkinsci/digitalocean-plugin/pull/50 I don't see anything I might have changed related to it. I'll try it again without my changes to see if i caused it. I looked at the PR linked inside the jira ticket and it looks like its only core changes.

Gavin

On Thu, Feb 11, 2021 at 12:53 AM Oleg Nenashev <[hidden email]> wrote:

It is a known error fixed in 2.235.

https://issues.jenkins.io/browse/JENKINS-62006

On Thursday, February 11, 2021 at 8:19:30 AM UTC+1 [hidden email] wrote:

I'm trying to fix the plain text credentials issue in the digital ocean plugin. I've updated it to use credentials, on to testing it, and having an agent issue. The droplet spins up, it connects, the job runs, and then errors with a stacktrace (at the end of email). Then if i run it again it runs on the same agent just fine without error.

Its complaining about `java.lang.ClassNotFoundException: javax.servlet.ServletException` does this mean that some do api is returning a servletexception and then remoting can't seralize it? or is it a known error with 2.222.4? am I missing the plugins somewhere in these stacktraces?

Gavin

Started by user unknown or anonymous
Running as SYSTEM
Building remotely on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff in workspace /jenkins/workspace/testdo
[testdo] $ /bin/sh -xe /tmp/jenkins4548124856065621833.sh
+ whoami
root
+ hostname
jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
+ uptime
 07:03:51 up 1 min,  0 users,  load average: 2.58, 1.11, 0.42
+ uname -a
Linux jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
FATAL: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to channel
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:288)
		at com.sun.proxy.$Proxy5.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:317)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
		at hudson.util.ProcessTree.get(ProcessTree.java:432)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:811)
	at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:931)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:998)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1856)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:432)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
	at hudson.remoting.Channel.call(Channel.java:1004)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1856)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
Finished: FAILURE

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ca060873-eeba-4fc4-baa5-6e38aab7f597n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DusTywAs19iLx1ZrBtAuZVvdEEFRk8P5QKgKRmXbpmKeFw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAH-3BidSCOCNEv%3Dw0uUv3GSjGK-%3D7uZeHhmQiLPo1iO%3Dcwgr9g%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_Duv%3D%3DN4vyssrtv3OQASqpQEW7y5-GWio%3DYZ7s%2BUwsO1A9Q%40mail.gmail.com.

[Tim Jacomb]

This should fix it I believe: https://github.com/jenkinsci/jenkins/pull/5367

On Wed, 3 Mar 2021 at 02:24, 'Gavin Mogan' via Jenkins Developers <[hidden email]> wrote:

Okay if anyone else wants to try, you can use my referral link and get yourself a bunch of free credits.https://m.do.co/c/7d6859326b6a

I'm just going to mark it as a known issue in the release notes as I don't have any idea how to debug this any more.

Gavin

On Thu, Feb 25, 2021 at 12:54 AM Tim Jacomb <[hidden email]> wrote:

I've had a bit of a look at this,

I patched the plugin to enable logging via workDir, unfortunately it didn't give much more than what's in the log for the freestyle build. (https://gist.github.com/timja/4b28f173d5d2925c82781109dce57362)

I had a look through these slides from Oleg https://speakerdeck.com/onenashev/day-of-jenkins-2017-dealing-with-agent-connectivity-issues?slide=66

But it didn't help any more than say enable logging

FTR I've got a very similar error in the slack plugin on freestyle builds when uploading a file from the agent.

Anyone got any tips for debugging this? Specific logs to enable and how to enable them?

The agent is started here:

https://github.com/jenkinsci/digitalocean-plugin/blob/ca1112c5c52a842c6340401f02f536fc07c6620a/src/main/java/com/dubture/jenkins/digitalocean/DigitalOceanComputerLauncher.java#L190

I patched it to be:

+            String launchString = "java " + jvmOpts + " -jar /tmp/agent.jar -workDir /root/.jenkins/remotingWorkDir";

Thanks

Tim

On Wed, 24 Feb 2021 at 21:23, 'Gavin Mogan' via Jenkins Developers <[hidden email]> wrote:

The patch is merged, but before I cut a release, I'd like to address the freestyle issue.

Is anyone available to pair with me to diagnose this? Or can I get someone credit to look into it?

On Sat, Feb 13, 2021 at 7:10 PM Gavin Mogan <[hidden email]> wrote:

Okay managed to do a bunch of permutations.

2.60.3 works just fine on both first and second freestyle job

2.263.4 fails on first job, but will succeed on the second if the agent is still running

2.263.4 works fine on first and second with pipeline

https://github.com/jenkinsci/remoting/blob/remoting-4.5/src/main/java/hudson/remoting/Channel.java#L1001

UserRequest.getClassLoader(callable) where callable is Launcher$RemoteLauncher$KillTask returns a WebAppClassLoader. Is that possibly why its trying to load ServletError?

I've been trying to attach a debugger and track this down more but Its well outside my java knowledge.

How can I get a debugger inside of a MasterToSlaveCallable, specifically https://github.com/jenkinsci/jenkins/blob/8c6327f4c6777d135e374e3d507ba9c20820bfc6/core/src/main/java/hudson/Launcher.java#L1164

I've tried attaching it to the hpi:run process, and the agent/remoting.jar process. I think the remoting process is the way to do it, but I can't figure out how to get a breakpoint inside of killtask

On Thu, Feb 11, 2021 at 8:11 PM Gavin Mogan <[hidden email]> wrote:

I've tried it with 2.235.5 and 2.263.4 with the same stacktrace.

My PR is up at https://github.com/jenkinsci/digitalocean-plugin/pull/50 I don't see anything I might have changed related to it. I'll try it again without my changes to see if i caused it. I looked at the PR linked inside the jira ticket and it looks like its only core changes.

Gavin

On Thu, Feb 11, 2021 at 12:53 AM Oleg Nenashev <[hidden email]> wrote:

It is a known error fixed in 2.235.

https://issues.jenkins.io/browse/JENKINS-62006

On Thursday, February 11, 2021 at 8:19:30 AM UTC+1 [hidden email] wrote:

I'm trying to fix the plain text credentials issue in the digital ocean plugin. I've updated it to use credentials, on to testing it, and having an agent issue. The droplet spins up, it connects, the job runs, and then errors with a stacktrace (at the end of email). Then if i run it again it runs on the same agent just fine without error.

Its complaining about `java.lang.ClassNotFoundException: javax.servlet.ServletException` does this mean that some do api is returning a servletexception and then remoting can't seralize it? or is it a known error with 2.222.4? am I missing the plugins somewhere in these stacktraces?

Gavin

Started by user unknown or anonymous
Running as SYSTEM
Building remotely on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff in workspace /jenkins/workspace/testdo
[testdo] $ /bin/sh -xe /tmp/jenkins4548124856065621833.sh
+ whoami
root
+ hostname
jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
+ uptime
 07:03:51 up 1 min,  0 users,  load average: 2.58, 1.11, 0.42
+ uname -a
Linux jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
FATAL: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to channel
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:288)
		at com.sun.proxy.$Proxy5.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:317)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
		at hudson.util.ProcessTree.get(ProcessTree.java:432)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:811)
	at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:931)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:998)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1856)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:432)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1100)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1091)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on jenkins-do-do-f0a19f9e-351c-48e3-9c45-3ca58d524eff failed
	at hudson.remoting.Channel.call(Channel.java:1004)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1083)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1856)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
Finished: FAILURE

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ca060873-eeba-4fc4-baa5-6e38aab7f597n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DusTywAs19iLx1ZrBtAuZVvdEEFRk8P5QKgKRmXbpmKeFw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAH-3BidSCOCNEv%3Dw0uUv3GSjGK-%3D7uZeHhmQiLPo1iO%3Dcwgr9g%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_Duv%3D%3DN4vyssrtv3OQASqpQEW7y5-GWio%3DYZ7s%2BUwsO1A9Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAH-3Bic48cJ9DRKduRUKDPD3ThhndyyAu_dH9UuU%3DPitkb7ebg%40mail.gmail.com.