[Issue 3558] New - local file copy with many files is slow

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[Issue 3558] New - local file copy with many files is slow

mmalaidini
https://hudson.dev.java.net/issues/show_bug.cgi?id=3558
                 Issue #|3558
                 Summary|local file copy with many files is slow
               Component|hudson
                 Version|current
                Platform|PC
              OS/Version|Linux
                     URL|
                  Status|NEW
       Status whiteboard|
                Keywords|
              Resolution|
              Issue type|ENHANCEMENT
                Priority|P3
            Subcomponent|core
             Assigned to|issues@hudson
             Reported by|mmalaidini






------- Additional comments from [hidden email] Thu Apr 23 20:16:34 +0000 2009 -------
Our build includes Clover reports which can generate 6.7 GB of data distributed
in 766 directories and 94135 files. After the report is completed it is archived
as an artifact.

It takes 2hours and 40 minutes for the operation to finish.
A raw file copy (cp -R) takes roughly 36 minutes.

In order to reproduce this issue:
*) create a freestyle build
*) Add build step 'Execute shell'
*) Enter 'time for i in {1..10000}; do dd if=/dev/urandom of="file$i.bin" bs=10K
count=1; done'

On our machine the entire build takes 1 minute and 23 seconds, while the file
generation takes roughly 49 seconds.

If you generate the files in a shell and do a 'cp -R' it takes roughly 3
seconds, that is way less than 30 something seconds above.

Increase number of files and size to make it more noticeable.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 3558] local file copy with many files is slow

mdonohue
https://hudson.dev.java.net/issues/show_bug.cgi?id=3558






------- Additional comments from [hidden email] Sat Apr 25 15:52:43 +0000 2009 -------
The thread from the mailing list about this:
http://www.nabble.com/Slow-file-copy-in-archive-tt23086795.html

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]