[JIRA] Created: (HUDSON-5703) Exception leaves zombie processes for slaves started by command on master

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[JIRA] Created: (HUDSON-5703) Exception leaves zombie processes for slaves started by command on master

Hudson issues mailing list
Exception leaves zombie processes for slaves started by command on master
-------------------------------------------------------------------------

                 Key: HUDSON-5703
                 URL: http://issues.hudson-ci.org/browse/HUDSON-5703
             Project: Hudson
          Issue Type: Bug
          Components: master-slave
    Affects Versions: current
            Reporter: jsiirola


I have several slaves that are started via the "Launch slave via execution of command on the Master."  The command is a bash script that acquires Kerberos credentials and then ssh's over to the slave.  Periodically, something happens that kills all the connections launched this way.  Regardless, the original bash process is not collected by Hudson, and is left as a {{zombie}} or {{defunct}} process.

The relevant excerpt from the hudson log appears to be:
{noformat}
Feb 17, 2010 12:43:14 PM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel <host>
java.io.EOFException
        at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
        at hudson.remoting.Channel$ReaderThread.run(Channel.java:852)
{noformat}

I wonder if the fix is something as simple as adding a {{proc.destroy()}} to {{CommandLauncher.java}}?:
{code:borderStyle=solid}
@Override
public void onClosed(Channel channel, IOException cause) {
    if (cause != null) {
        cause.printStackTrace(
            listener.error(hudson.model.Messages.Slave_Terminated(getTimestamp())));
    }
    ProcessTree.get().killAll(proc, cookie);
    proc.destroy();
}
{code}


--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.hudson-ci.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[JIRA] Commented: (HUDSON-5703) Exception leaves zombie processes for slaves started by command on master

Hudson issues mailing list

    [ http://issues.hudson-ci.org/browse/HUDSON-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=136184#action_136184 ]

jsiirola commented on HUDSON-5703:
----------------------------------

Steps to reproduce (verified on 1.347 running on RHEL 5.4 within the embedded Winstone server):

# Configure a slave to start by running a command on the master
#* set the command to \\
   {{bash -c 'ssh localhost "cd ~/localhost; java -jar ~/localhost/slave.jar"'}}
# Allow the slave to connect
# Restart Hudson using {{/safeRestart}}

\\
This will orphan the ssh process, leaving it as "defunct" until the entire java server process is restarted.

> Exception leaves zombie processes for slaves started by command on master
> -------------------------------------------------------------------------
>
>                 Key: HUDSON-5703
>                 URL: http://issues.hudson-ci.org/browse/HUDSON-5703
>             Project: Hudson
>          Issue Type: Bug
>          Components: master-slave
>    Affects Versions: current
>            Reporter: jsiirola
>
> I have several slaves that are started via the "Launch slave via execution of command on the Master."  The command is a bash script that acquires Kerberos credentials and then ssh's over to the slave.  Periodically, something happens that kills all the connections launched this way.  Regardless, the original bash process is not collected by Hudson, and is left as a {{zombie}} or {{defunct}} process.
> The relevant excerpt from the hudson log appears to be:
> {noformat}
> Feb 17, 2010 12:43:14 PM hudson.remoting.Channel$ReaderThread run
> SEVERE: I/O error in channel <host>
> java.io.EOFException
>         at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
>         at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
>         at hudson.remoting.Channel$ReaderThread.run(Channel.java:852)
> {noformat}
> I wonder if the fix is something as simple as adding a {{proc.destroy()}} to {{CommandLauncher.java}}?:
> {code:borderStyle=solid}
> @Override
> public void onClosed(Channel channel, IOException cause) {
>     if (cause != null) {
>         cause.printStackTrace(
>             listener.error(hudson.model.Messages.Slave_Terminated(getTimestamp())));
>     }
>     ProcessTree.get().killAll(proc, cookie);
>     proc.destroy();
> }
> {code}

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.hudson-ci.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]