Problems with cvs log and -S option

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Problems with cvs log and -S option

Kayser, William H

I was having problems with builds because the log command was falling over halfway through.  I ran the command manually and discovered that the resulting output was MASSIVE because we have a lot of files and a ridiculously large number of revision tags.

 

I looked through the source and noticed that the –S option was not being used unless the build had a tag.  That seemed odd to me, so I changed the logic to always try using –S.

 

The effect was not only to fix my failure but I am pretty certain it resulted in a significant improvement in the log operation for all my builds.

 

So my question is why was the –S option omitted except for tags?  Here is the code in ChangeLogTask.  This gets executed on the slave.

 

            //whk why not do this in all cases?

            if (getTag() != null) {

                CvsVersion myCvsVersion = new CvsVersion();

                myCvsVersion.setProject(getProject());

                myCvsVersion.setTaskName("cvsversion");

                myCvsVersion.setCvsRoot(getCvsRoot());

                myCvsVersion.setCvsRsh(getCvsRsh());

                myCvsVersion.setPassfile(getPassFile());

                myCvsVersion.setDest(m_dir);

                myCvsVersion.execute();

                if (myCvsVersion.supportsCvsLogWithSOption()) {

                    addCommandArgument("-S");

                }

            }

 

I’ve been running with this change for a couple of days with different master/slave configurations and windows clients.

 

The only problem I had was when one of my slaves used an older version of wincvs that didn’t support the –S option.  The version checking is supposed to account for that but for some reason the supports… call returned the wrong result.  I’m not sure how big of a problem that is.

Reply | Threaded
Open this post in threaded view
|

Re: Problems with cvs log and -S option

Kohsuke Kawaguchi
Administrator
Kayser, Bill wrote:
> I was having problems with builds because the log command was falling
> over halfway through.  I ran the command manually and discovered that
> the resulting output was MASSIVE because we have a lot of files and a
> ridiculously large number of revision tags.

Jesse just committed a fix in the next 1.102 for the regression I
introduced earlier, so "cvs log" shouldn't run against the whole
workspace any more.

So for example, when you change just 5 files, cvs log runs only against
5 files. I think that should effectively achieve the performance
improvements -S would achieve.

Do you think we should still do the -S handling?

> I looked through the source and noticed that the -S option was not being
> used unless the build had a tag.  That seemed odd to me, so I changed
> the logic to always try using -S.

This code is from Ant, and I checked its revision log but it didn't have
any comment as to why this is limited to tag. Maybe it's the fact that
it runs a separate cvs version command.

Hudson should be able to run the command once and use the result to
decide whether to set -S or not. So if we are to do this, we should do that.


> The effect was not only to fix my failure but I am pretty certain it
> resulted in a significant improvement in the log operation for all my
> builds.
>
>  
>
> So my question is why was the -S option omitted except for tags?  Here
> is the code in ChangeLogTask.  This gets executed on the slave.
>
>  
>
>             //whk why not do this in all cases?
>
>             if (getTag() != null) {
>
>                 CvsVersion myCvsVersion = new CvsVersion();
>
>                 myCvsVersion.setProject(getProject());
>
>                 myCvsVersion.setTaskName("cvsversion");
>
>                 myCvsVersion.setCvsRoot(getCvsRoot());
>
>                 myCvsVersion.setCvsRsh(getCvsRsh());
>
>                 myCvsVersion.setPassfile(getPassFile());
>
>                 myCvsVersion.setDest(m_dir);
>
>                 myCvsVersion.execute();
>
>                 if (myCvsVersion.supportsCvsLogWithSOption()) {
>
>                     addCommandArgument("-S");
>
>                 }
>
>             }
>
>  
>
> I've been running with this change for a couple of days with different
> master/slave configurations and windows clients.
>
>  
>
> The only problem I had was when one of my slaves used an older version
> of wincvs that didn't support the -S option.  The version checking is
> supposed to account for that but for some reason the supports... call
> returned the wrong result.  I'm not sure how big of a problem that is.
>
>

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Re: Problems with cvs log and -S option

Kayser, William H


> -----Original Message-----
> From: Kohsuke Kawaguchi [mailto:[hidden email]]


> Jesse just committed a fix in the next 1.102 for the regression I
> introduced earlier, so "cvs log" shouldn't run against the whole
> workspace any more.
>
> So for example, when you change just 5 files, cvs log runs only
against
> 5 files. I think that should effectively achieve the performance
> improvements -S would achieve.
>
> Do you think we should still do the -S handling?

My vote would be to use the -S option all the time.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Problems with cvs log and -S option

Kohsuke Kawaguchi
Administrator
Kayser, Bill wrote:
>> Do you think we should still do the -S handling?
>
> My vote would be to use the -S option all the time.

OK. Could you file an issue for this in the issue tracker?

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Re: Problems with cvs log and -S option

Kayser, William H
In reply to this post by Kohsuke Kawaguchi
I tested this out.  I am still having the problem.  It seems as if cvs
log is running against the whole workspace--I get pages of cvs log
commands.  Then the server disconnects in the middle of the command, as
before.

Is this because I am _not_ using the cvs update mode?

Running build 1.103

> -----Original Message-----
> From: Kohsuke Kawaguchi [mailto:[hidden email]]
> Sent: Saturday, April 14, 2007 8:14 AM
> To: [hidden email]
> Subject: Re: Problems with cvs log and -S option
>
> Kayser, Bill wrote:
> > I was having problems with builds because the log command was
falling
> > over halfway through.  I ran the command manually and discovered
that
> > the resulting output was MASSIVE because we have a lot of files and
a
> > ridiculously large number of revision tags.
>
> Jesse just committed a fix in the next 1.102 for the regression I
> introduced earlier, so "cvs log" shouldn't run against the whole
> workspace any more.
>
> So for example, when you change just 5 files, cvs log runs only
against
> 5 files. I think that should effectively achieve the performance
> improvements -S would achieve.
>
> Do you think we should still do the -S handling?
>
> > I looked through the source and noticed that the -S option was not
being
> > used unless the build had a tag.  That seemed odd to me, so I
changed
> > the logic to always try using -S.
>
> This code is from Ant, and I checked its revision log but it didn't
have

> any comment as to why this is limited to tag. Maybe it's the fact that
> it runs a separate cvs version command.
>
> Hudson should be able to run the command once and use the result to
> decide whether to set -S or not. So if we are to do this, we should do
> that.
>
>
> > The effect was not only to fix my failure but I am pretty certain it
> > resulted in a significant improvement in the log operation for all
my
> > builds.
> >
> >
> >
> > So my question is why was the -S option omitted except for tags?
Here

> > is the code in ChangeLogTask.  This gets executed on the slave.
> >
> >
> >
> >             //whk why not do this in all cases?
> >
> >             if (getTag() != null) {
> >
> >                 CvsVersion myCvsVersion = new CvsVersion();
> >
> >                 myCvsVersion.setProject(getProject());
> >
> >                 myCvsVersion.setTaskName("cvsversion");
> >
> >                 myCvsVersion.setCvsRoot(getCvsRoot());
> >
> >                 myCvsVersion.setCvsRsh(getCvsRsh());
> >
> >                 myCvsVersion.setPassfile(getPassFile());
> >
> >                 myCvsVersion.setDest(m_dir);
> >
> >                 myCvsVersion.execute();
> >
> >                 if (myCvsVersion.supportsCvsLogWithSOption()) {
> >
> >                     addCommandArgument("-S");
> >
> >                 }
> >
> >             }
> >
> >
> >
> > I've been running with this change for a couple of days with
different
> > master/slave configurations and windows clients.
> >
> >
> >
> > The only problem I had was when one of my slaves used an older
version
> > of wincvs that didn't support the -S option.  The version checking
is
> > supposed to account for that but for some reason the supports...
call
> > returned the wrong result.  I'm not sure how big of a problem that
is.
> >
> >
>
>
> --
> Kohsuke Kawaguchi
> Sun Microsystems                   [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Problems with cvs log and -S option

Kohsuke Kawaguchi
Administrator
Kayser, Bill wrote:
> I tested this out.  I am still having the problem.  It seems as if cvs
> log is running against the whole workspace--I get pages of cvs log
> commands.  Then the server disconnects in the middle of the command, as
> before.
>
> Is this because I am _not_ using the cvs update mode?

Ah, yes. When you use cvs update, Hudson finds out which files were
updated, and only run cvs log against them. When you run full checkout,
no such processing happens.

I guess that's an argument for supporting the -S option. Sometimes you
do want to check out a fresh copy.

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment