[Issue 3524] New - sending artifacts to master is very slow

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Issue 3524] New - sending artifacts to master is very slow

javakoe
https://hudson.dev.java.net/issues/show_bug.cgi?id=3524
                 Issue #|3524
                 Summary|sending artifacts to master is very slow
               Component|hudson
                 Version|current
                Platform|PC
              OS/Version|Linux
                     URL|
                  Status|NEW
       Status whiteboard|
                Keywords|
              Resolution|
              Issue type|ENHANCEMENT
                Priority|P3
            Subcomponent|master-slave
             Assigned to|issues@hudson
             Reported by|javakoe






------- Additional comments from [hidden email] Mon Apr 20 15:54:05 +0000 2009 -------
Our Hudson builds a .war file of about 100MB. Sending this artifact from the
slave to the master over Gbit ethernet takes about 2 or 3 minutes. Gbit
ethernet should do this in about 1 second theoreticly. Practicly scp takes
about 4 seconds to copy 100MB.

I made this testcase.
-> New freestyle build -> Add build step 'Execute Shell' -> enter 'dd if=/dev/
urandom of=this_is_big.bin bs=1M count=100' -> Archive artifacts
'this_is_big.bin' -> Schedule at slave -> dd takes about 45 seconds -> job
takes about 3 minutes.

ngrep shows me a lot of serialized java stuff, but very little effective data
is send per packet.

I'm using Hudson 1.300 for master and slave.
I hope people can reproduce and fix it.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 3524] sending artifacts to master is very slow

javakoe
https://hudson.dev.java.net/issues/show_bug.cgi?id=3524






------- Additional comments from [hidden email] Mon Apr 20 15:58:21 +0000 2009 -------
I forgot to mention:
java -version
java version "1.6.0_04"
Java(TM) SE Runtime Environment (build 1.6.0_04-b12)
Java HotSpot(TM) Server VM (build 10.0-b19, mixed mode)

Linux linux15 2.6.22.18-rw5 #1 SMP Thu Mar 6 15:10:11 CET 2008 i686 GNU/Linux


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 3524] sending artifacts to master is very slow

rexc
In reply to this post by javakoe
https://hudson.dev.java.net/issues/show_bug.cgi?id=3524



User rexc changed the following:

                What    |Old value                 |New value
================================================================================
              OS/Version|Linux                     |FreeBSD
--------------------------------------------------------------------------------




------- Additional comments from [hidden email] Thu Apr 23 01:56:15 +0000 2009 -------
I have a similar slow issue with copying artifacts from slave to master.

The master is version 1.299 on FreeBSD with java 1.6, under tomcat
The slave is a Mac OS X 10.4 with JVM of 1.5

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 3524] sending artifacts to master is very slow

javakoe
In reply to this post by javakoe
https://hudson.dev.java.net/issues/show_bug.cgi?id=3524






------- Additional comments from [hidden email] Fri Apr 24 14:52:58 +0000 2009 -------
Created an attachment (id=679)
print the number of bytes send to the master


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 3524] sending artifacts to master is very slow

javakoe
In reply to this post by javakoe
https://hudson.dev.java.net/issues/show_bug.cgi?id=3524



User javakoe changed the following:

                What    |Old value                 |New value
================================================================================
              Issue type|ENHANCEMENT               |DEFECT
--------------------------------------------------------------------------------
              OS/Version|FreeBSD                   |All
--------------------------------------------------------------------------------




------- Additional comments from [hidden email] Fri Apr 24 14:56:31 +0000 2009 -------
My patch displays the nr of bytes (in the byte[] in the Chunk) send in the
Channel.
The output while sending the artifact from the testcase is a lot of lines
repeating like this:
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 14
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512
channel write nr of bytes: 512

So, this is not a lot of bytes at once.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 3524] sending artifacts to master is very slow

javakoe
In reply to this post by javakoe
https://hudson.dev.java.net/issues/show_bug.cgi?id=3524



User javakoe changed the following:

                What    |Old value                 |New value
================================================================================
                  Status|NEW                       |RESOLVED
--------------------------------------------------------------------------------
              Resolution|                          |FIXED
--------------------------------------------------------------------------------




------- Additional comments from [hidden email] Mon May  4 09:43:21 +0000 2009 -------
It is better since 1.302.
Still scp copies the file in 3 seconds and hudson in 30 seconds, but it is a
lot better than a couple of minutes to copy the file.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 3524] sending artifacts to master is very slow

thedotedge-2
In reply to this post by javakoe
https://hudson.dev.java.net/issues/show_bug.cgi?id=3524






------- Additional comments from [hidden email] Wed Jul  1 09:06:06 +0000 2009 -------
1.313 on CentOS 5.2, the speed is around 30kBps vs around 450kBps via scp.
It's still better, but still very slow.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]