[Issue 1705] New - Hudson startes 100's of ssh sessions for offline slaves

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[Issue 1705] New - Hudson startes 100's of ssh sessions for offline slaves

rajp-2
https://hudson.dev.java.net/issues/show_bug.cgi?id=1705
                 Issue #|1705
                 Summary|Hudson startes 100's of ssh sessions for offline slave
                        |s
               Component|hudson
                 Version|current
                Platform|All
              OS/Version|Windows XP
                     URL|
                  Status|NEW
       Status whiteboard|
                Keywords|
              Resolution|
              Issue type|DEFECT
                Priority|P1
            Subcomponent|master-slave
             Assigned to|issues@hudson
             Reported by|rajp






------- Additional comments from [hidden email] Fri May 16 00:00:02 +0000 2008 -------
Im using Hudson on a windows xp machine with cygwin.
the slaves connected using cygwin ssh.

In the last 2 releases of hudson,
Over a few hours, for some reason heaps of ssh.exe sessions are started.
I have a few slaves which as offline. Whenever hudson try's to connect to them
it starts a new ssh.exe session but dosnt kill it when the connection failes.

I noticed the following error caused an extra 4 ssh sessions to be started.
-----
SEVERE: Unable to launch the slave agent for linux02
java.io.EOFException: unexpected stream termination
        at hudson.remoting.Channel.<init>(Channel.java:258)
        at hudson.model.Slave$ComputerImpl.setChannel(Slave.java:390)
        at hudson.model.Slave$ComputerImpl$1.run(Slave.java:342)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:619)
-------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 1705] Hudson startes 100's of ssh sessions for offline slaves

scm_issue_link
https://hudson.dev.java.net/issues/show_bug.cgi?id=1705



User scm_issue_link changed the following:

                What    |Old value                 |New value
================================================================================
                  Status|NEW                       |RESOLVED
--------------------------------------------------------------------------------
              Resolution|                          |FIXED
--------------------------------------------------------------------------------




------- Additional comments from [hidden email] Mon May 19 01:35:01 +0000 2008 -------
Code changed in hudson
User: : kohsuke
Path:
 trunk/hudson/main/core/src/main/java/hudson/slaves/CommandLauncher.java
 trunk/www/changelog.html
http://fisheye4.cenqua.com/changelog/hudson/?cs=9434
Log:
[FIXED HUDSON-1705] In case of abnormal termination, kill the process


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 1705] Hudson startes 100's of ssh sessions for offline slaves

lloydchang
In reply to this post by rajp-2
https://hudson.dev.java.net/issues/show_bug.cgi?id=1705



User lloydchang changed the following:

                What    |Old value                 |New value
================================================================================
                  Status|RESOLVED                  |REOPENED
--------------------------------------------------------------------------------
              Resolution|FIXED                     |
--------------------------------------------------------------------------------




------- Additional comments from [hidden email] Mon Jun 23 07:23:57 +0000 2008 -------
Using Hudson 1.224, I still see the issue.  Hudson is calling a Cygwin shell
script, which launches SSH, then sleeps a long time (or else Hudson continues to
re-launch the shell script).
 
In Windows and Cygwin process tree, I see bash.exe, ssh.exe, and sleep.exe
proceses left open by Hudson, even after I stop the Hudson server from the UI,
then kill the Hudson Java process.  I double-checked via Process Explorer -
http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx , bash.exe,
ssh.exe, and sleep.exe all end up with Parent IDs of 1
 
To try working around the issue, I tried having SSH script sleep for a long
time, to avoid respawning, but the root cause remains.  I think Hudson forks off
sub-processes that become parent-less zombies somehow.
 
It is similar to the behavior described in
http://www.cygwin.com/ml/cygwin-xfree/2006-12/msg00082.html , whereas someone
proposed a background monitoring process workaround in
http://lists.samba.org/archive/rsync/2003-November/007736.html
 
Due to above issues, I switched to a Hudson *nix server, and started using the
SSH plugin to avoid the root cause.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 1705] Hudson startes 100's of ssh sessions for offline slaves

scm_issue_link
In reply to this post by rajp-2
https://hudson.dev.java.net/issues/show_bug.cgi?id=1705



User scm_issue_link changed the following:

                What    |Old value                 |New value
================================================================================
                  Status|REOPENED                  |RESOLVED
--------------------------------------------------------------------------------
              Resolution|                          |FIXED
--------------------------------------------------------------------------------




------- Additional comments from [hidden email] Tue Jul 22 21:40:57 +0000 2008 -------
Code changed in hudson
User: : kohsuke
Path:
 trunk/hudson/main/core/src/main/java/hudson/slaves/SlaveComputer.java
 trunk/www/changelog.html
http://fisheye4.cenqua.com/changelog/hudson/?cs=10955
Log:
[FIXED HUDSON-1705]
A trouble-shooting of another incident revealed this problem. I believe this was the cause of 1705. See the comment in the code for the analysis.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]