[Issue 2001] New - SVN check out slow performance issue

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[Issue 2001] New - SVN check out slow performance issue

ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001
                 Issue #|2001
                 Summary|SVN check out slow performance issue
               Component|hudson
                 Version|current
                Platform|Sun
              OS/Version|Solaris
                     URL|
                  Status|NEW
       Status whiteboard|
                Keywords|
              Resolution|
              Issue type|DEFECT
                Priority|P2
            Subcomponent|subversion
             Assigned to|issues@hudson
             Reported by|ssburlg






------- Additional comments from [hidden email] Tue Jul  1 23:17:38 +0000 2008 -------
The Subversion checkout in Hudson's continuous build is slow. Despite setting
the checkout to run against a memory based Solaris file system, the checkout is
still acting as slowly as to a regular filesystem. For example, the Subversion
checkout for a memory based Solaris file system took just 1 minutes and 30
seconds against 7 minutes and 9 seconds for Hudson's continuous build checkout
into a memory based Solaris file system (more than 4.5 times slower). Another
example, the Subversion checkout from NetBeans IDE took only 3 minutes and it
was done into harddisk file system which faster for 2 times comparing with
Hudson checkout into a memory based Solaris file system.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

Kohsuke Kawaguchi
Administrator
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001






------- Additional comments from [hidden email] Wed Jul  2 14:42:04 +0000 2008 -------
Does this involve master/slave set up? What is the SVN repository URL? Is this
http, ssh, svn, or something else?


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

ssburlg
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001






------- Additional comments from [hidden email] Wed Jul  2 16:25:30 +0000 2008 -------
"Does this involve master/slave set up?"

Nope.

"What is the SVN repository URL?"

http://toolssvn.sfbay.sun.com/svn/wow/trunk/platform

Note: it's on SWAN

"Is this http, ssh, svn, or something else?"

Not a ssh as far as I know. Yes, it's http for accessing to SVN repository.

Thank you Kohsuke for looking to this issue and should you have any questions,
please let me know.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

pgweiss
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001






------- Additional comments from [hidden email] Fri Aug 22 12:13:16 +0000 2008 -------
In my setup I have an svn+ssh URL.  The command line client (SVN 1.5.1 running
in Linux) takes 1 minutes 12 seconds to check out.  Hudson takes about 15
minutes.

Is there a possible workaround?  That is, can I get Hudson to not do the
checkout at all, rather, just use its built in svn support for polling, and I
will do the checkout as a build step?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

pgweiss
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001






------- Additional comments from [hidden email] Fri Aug 22 21:14:56 +0000 2008 -------
This appears to be an svnkit issue rather than a hudson issue.  Notice that
svnkit takes an order of magnitude longer to check out than does the native
svn.  

# $SVN is an svn+ssh:// url

[test] deng-pweis| ~/download/svnkit-1.2.0.4710/jsvn co $SVN/trunk/x
...
Checked out revision 208171.
~/download/svnkit-1.2.0.4710/jsvn co $SVN/trunk/x  65.93s user 392.37s system
63% cpu 12:05.85 total

[test] deng-pweis| du -sh x
1.3G    qres

[test] deng-pweis| rm -rf x

[test] deng-pweis| time /usr/bin/svn co $SVN/trunk/x
...
Checked out revision 208177.
/usr/bin/svn co $SVN/trunk/x  8.24s user 6.06s system 18% cpu 1:17.06 total

[test] deng-pweis| /usr/bin/svn --version
svn, version 1.5.1 (r32289)
   compiled Jul 24 2008, 20:58:29

Copyright (C) 2000-2008 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/
This product includes software developed by CollabNet (http://www.Collab.Net/).

The following repository access (RA) modules are available:

* ra_neon : Module for accessing a repository via WebDAV protocol using Neon.
  - handles 'http' scheme
  - handles 'https' scheme
* ra_svn : Module for accessing a repository using the svn network protocol.
  - with Cyrus SASL authentication
  - handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
  - handles 'file' scheme
* ra_serf : Module for accessing a repository via WebDAV protocol using serf.
  - handles 'http' scheme
  - handles 'https' scheme

[test] deng-pweis| /usr/bin/java -version
java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
Java HotSpot(TM) 64-Bit Server VM (build 10.0-b23, mixed mode)

[test] deng-pweis| ~/download/svnkit-1.2.0.4710/jsvn --version
jsvn, version 1.2.0 (r4710)

Copyright (C) 2004-2007 TMate Software.
SVNKit is open source (GPL) software, see http://svnkit.com/ for more
information.
SVNKit is pure Java (TM) version of Subversion, see http://
subversion.tigris.org/

The following repository access (RA) modules are available:

* org.tmatesoft.svn.core.internal.io.dav : Module for accessing a repository
via WebDAV protocol.
  - handles 'http' scheme
  - handles 'https' scheme
* org.tmatesoft.svn.core.internal.io.svn: Module for accessing a repository
using the svn network protocol.
  - handles 'svn' scheme
* org.tmatesoft.svn.core.internal.io.fs: Module for accessing a repository on
local disk.
  - handles 'file' scheme (only FSFS repositories are supported)




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

Kohsuke Kawaguchi
Administrator
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001



User kohsuke changed the following:

                What    |Old value                 |New value
================================================================================
                  Status|NEW                       |RESOLVED
--------------------------------------------------------------------------------
              Resolution|                          |FIXED
--------------------------------------------------------------------------------




------- Additional comments from [hidden email] Wed Nov  5 18:17:51 +0000 2008 -------
The original problem reported by ssburlg and the later addition by pgweiss are
likely different as they involve different transports.

In 1.260 I fixed a bug in SVNKit, which was breaking the use of persistent HTTP
connections entirely. So hopefully this fixes the problem reported by ssburlg.

Also in 1.260, we bumped up SVNKit to 1.2.0 release version, and one of the
changelogs include svn+ssh performance improvement, which hopefully addresses
the problem reported by pgweiss.

If the new version won't show enough performance improvements, please feel free
to reopen.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

ssburlg
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001



User ssburlg changed the following:

                What    |Old value                 |New value
================================================================================
                  Status|RESOLVED                  |REOPENED
--------------------------------------------------------------------------------
                Priority|P2                        |P1
--------------------------------------------------------------------------------
              Resolution|FIXED                     |
--------------------------------------------------------------------------------




------- Additional comments from [hidden email] Thu Nov  6 21:26:54 +0000 2008 -------
The new version(1.260) doesn't showed performance improvements. Moreover, I
experienced performance degradation, when I ran job which did just check out and
it took 12 minutes, comparing with 9 minutes 22 second for version 1.230...

So, I downgraded Hudson back to 1.230.  

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

bhoyt
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001






------- Additional comments from [hidden email] Mon Dec  8 23:48:32 +0000 2008 -------
I'm seeing this issue in 1.262.  Subversion checkouts from a master are taking nearly 35 minutes, while
the same checkout from the command line svn client on that master zips along in just a few minutes.  
This is a svn+ssh URL.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

ijuma
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001






------- Additional comments from [hidden email] Wed Jan 21 11:33:45 +0000 2009 -------
Might be worth importing the newest SVNKit (1.2.2) to see if it helps.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

pgweiss
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001






------- Additional comments from [hidden email] Wed Jan 21 13:09:04 +0000 2009 -------
I am not optimistic about ever getting speeds in SVNKit up to anywhere near
what they are for the regular svn client.  I know Kohsuke is not in favor of
this, but I think the only way to address the speed issue is to have Hudson use
the command line client directly.  Or, we can learn to live with the slow
speeds.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

ijuma
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001






------- Additional comments from [hidden email] Wed Jan 21 13:13:12 +0000 2009 -------
As far as I am concerned, the speed we had at version 1.230 is good enough and
that used SVNKit if I understand correctly.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Issue 2001] SVN check out slow performance issue

Kohsuke Kawaguchi
Administrator
In reply to this post by ssburlg
https://hudson.dev.java.net/issues/show_bug.cgi?id=2001



User kohsuke changed the following:

                What    |Old value                 |New value
================================================================================
                  Status|REOPENED                  |RESOLVED
--------------------------------------------------------------------------------
                Priority|P1                        |P3
--------------------------------------------------------------------------------
              Resolution|                          |FIXED
--------------------------------------------------------------------------------




------- Additional comments from [hidden email] Thu Jul  9 21:07:58 +0000 2009 -------
The performance issue on Solaris is fixed. See issue #3702.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]