[JIRA] Created: (HUDSON-8346) Copy to slave incompatible with multi configuration project

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[JIRA] Created: (HUDSON-8346) Copy to slave incompatible with multi configuration project

Kohsuke Kawaguchi
Administrator
Copy to slave incompatible with multi configuration project
-----------------------------------------------------------

                 Key: HUDSON-8346
                 URL: http://issues.hudson-ci.org/browse/HUDSON-8346
             Project: Hudson
          Issue Type: Bug
          Components: copy-to-slave
    Affects Versions: current
            Reporter: psoetens
            Assignee: rseguy


copy-to-slave does nothing when the job runs on the master. This is the wrong behavior if the job is a multi-configuration project, because these jobs run with a modified workspace, depending on the parameter of the current job.

What the plugin should do when the job is running on the master, is to check if the file in the 'default' workspace is identical to the target file (it won't be in a multi-config build) and if not, copy it to the new workspace.

For example, the job on master has this workspace:
$ pwd
.../hudson/jobs/bootstrap-toolchain/workspace
$ ls
buildconf  config-env.yml  config.yml  my-bootstrap.sh TARGET

where TARGET is the first category for the parametrization. The job will never run in this 'workspace' dir, but somewhere deep in the TARGET subdir. So copy-to-slave should copy the file from this workspace into the deep 'multi-config' workspace.

For example, when my job runs, these variables are set:
JOB_NAME=bootstrap-toolchain/TARGET=gnulinux,arch=amd64,branch=master
WORKSPACE=.../hudson/jobs/bootstrap-toolchain/workspace/TARGET/gnulinux/arch/amd64/branch/master

I hope this report is clear about the issue.


--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.hudson-ci.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[JIRA] Commented: (HUDSON-8346) Copy to slave incompatible with multi configuration project

Kohsuke Kawaguchi
Administrator

    [ http://issues.hudson-ci.org/browse/HUDSON-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=144994#action_144994 ]

rseguy commented on HUDSON-8346:
--------------------------------

Well, I can't say it's clear to me...

One first thing: Does multi-configuration project == matrix project? If yes, then I'll need more details since I don't use this kind of jobs.

> Copy to slave incompatible with multi configuration project
> -----------------------------------------------------------
>
>                 Key: HUDSON-8346
>                 URL: http://issues.hudson-ci.org/browse/HUDSON-8346
>             Project: Hudson
>          Issue Type: Bug
>          Components: copy-to-slave
>    Affects Versions: current
>            Reporter: psoetens
>            Assignee: rseguy
>
> copy-to-slave does nothing when the job runs on the master. This is the wrong behavior if the job is a multi-configuration project, because these jobs run with a modified workspace, depending on the parameter of the current job.
> What the plugin should do when the job is running on the master, is to check if the file in the 'default' workspace is identical to the target file (it won't be in a multi-config build) and if not, copy it to the new workspace.
> For example, the job on master has this workspace:
> $ pwd
> .../hudson/jobs/bootstrap-toolchain/workspace
> $ ls
> buildconf  config-env.yml  config.yml  my-bootstrap.sh TARGET
> where TARGET is the first category for the parametrization. The job will never run in this 'workspace' dir, but somewhere deep in the TARGET subdir. So copy-to-slave should copy the file from this workspace into the deep 'multi-config' workspace.
> For example, when my job runs, these variables are set:
> JOB_NAME=bootstrap-toolchain/TARGET=gnulinux,arch=amd64,branch=master
> WORKSPACE=.../hudson/jobs/bootstrap-toolchain/workspace/TARGET/gnulinux/arch/amd64/branch/master
> I hope this report is clear about the issue.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.hudson-ci.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[JIRA] Commented: (HUDSON-8346) Copy to slave incompatible with multi configuration project

Kohsuke Kawaguchi
Administrator
In reply to this post by Kohsuke Kawaguchi

    [ http://issues.hudson-ci.org/browse/HUDSON-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=145004#action_145004 ]

psoetens commented on HUDSON-8346:
----------------------------------

Well, if you click on 'New Job' there will be a radio button for 'Build multi-configuration project'. I don't have an option for 'matrix project', although in the config page of the created project, it's called 'Configuration Matrix'.

The point of a multi-config project is that it works in a deep subdir of the project's workspace. Somehow, the other tools, like the version control tools, know that they have to check-out to that directory, but your plugin doesn't even try, it just does the 'master' check instead of checking if source and target file are the same.

What I'd like to be the behavior of this plugin is similar to this script (bash syntax):

for slave in $nodes; do
   # always copy if: slave is not master OR source and target file are not the same file.
   if [ $slave -neq "master" -o -not $HUDSON/jobs/$JOB_NAME/$file_to_copy -ef $HOME/$WORKSPACE/$file_to_copy ]; then
     scp $HUDSON/jobs/$JOB_NAME/$file_to_copy $slave:$WORKSPACE
   fi
done

So to use jobs/$JOB_NAME as source directory and to use $WORKSPACE as target directory.

The JOB_NAME in the original post shows how it is actually encoded,so you'd need to strip everything after the last '/'.

Peter

> Copy to slave incompatible with multi configuration project
> -----------------------------------------------------------
>
>                 Key: HUDSON-8346
>                 URL: http://issues.hudson-ci.org/browse/HUDSON-8346
>             Project: Hudson
>          Issue Type: Bug
>          Components: copy-to-slave
>    Affects Versions: current
>            Reporter: psoetens
>            Assignee: rseguy
>
> copy-to-slave does nothing when the job runs on the master. This is the wrong behavior if the job is a multi-configuration project, because these jobs run with a modified workspace, depending on the parameter of the current job.
> What the plugin should do when the job is running on the master, is to check if the file in the 'default' workspace is identical to the target file (it won't be in a multi-config build) and if not, copy it to the new workspace.
> For example, the job on master has this workspace:
> $ pwd
> .../hudson/jobs/bootstrap-toolchain/workspace
> $ ls
> buildconf  config-env.yml  config.yml  my-bootstrap.sh TARGET
> where TARGET is the first category for the parametrization. The job will never run in this 'workspace' dir, but somewhere deep in the TARGET subdir. So copy-to-slave should copy the file from this workspace into the deep 'multi-config' workspace.
> For example, when my job runs, these variables are set:
> JOB_NAME=bootstrap-toolchain/TARGET=gnulinux,arch=amd64,branch=master
> WORKSPACE=.../hudson/jobs/bootstrap-toolchain/workspace/TARGET/gnulinux/arch/amd64/branch/master
> I hope this report is clear about the issue.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.hudson-ci.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[JIRA] Commented: (HUDSON-8346) Copy to slave incompatible with multi configuration project

Kohsuke Kawaguchi
Administrator
In reply to this post by Kohsuke Kawaguchi

    [ http://issues.hudson-ci.org/browse/HUDSON-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=145032#action_145032 ]

rseguy commented on HUDSON-8346:
--------------------------------

Thanks for the precision, I think I'll have to dive into the way Hudson handles that.
In fact, when you say {quote}the other tools, like the version control tools, know that they have to check-out to that directory, but your plugin doesn't even try, it just does the 'master' check instead of checking if source and target file are the same{quote} you're absolutely right, but it's on purpose/required: Hudson offers tons of methods to get the current workspace and so on, "but" (this should not be  "but" but "as a consequence" in a normal case) these methods return pointer to the actual location (on the slave, etc.).
The only one way to get the workspace on the master, for example when doing a copy-to-slave, is then to build the pointer manually (by appending {{/workspace}} to the job name). Which means: All possible cases handled by Hudson have to be manually handled in the Copy To Slave plugin.

> Copy to slave incompatible with multi configuration project
> -----------------------------------------------------------
>
>                 Key: HUDSON-8346
>                 URL: http://issues.hudson-ci.org/browse/HUDSON-8346
>             Project: Hudson
>          Issue Type: Bug
>          Components: copy-to-slave
>    Affects Versions: current
>            Reporter: psoetens
>            Assignee: rseguy
>
> copy-to-slave does nothing when the job runs on the master. This is the wrong behavior if the job is a multi-configuration project, because these jobs run with a modified workspace, depending on the parameter of the current job.
> What the plugin should do when the job is running on the master, is to check if the file in the 'default' workspace is identical to the target file (it won't be in a multi-config build) and if not, copy it to the new workspace.
> For example, the job on master has this workspace:
> $ pwd
> .../hudson/jobs/bootstrap-toolchain/workspace
> $ ls
> buildconf  config-env.yml  config.yml  my-bootstrap.sh TARGET
> where TARGET is the first category for the parametrization. The job will never run in this 'workspace' dir, but somewhere deep in the TARGET subdir. So copy-to-slave should copy the file from this workspace into the deep 'multi-config' workspace.
> For example, when my job runs, these variables are set:
> JOB_NAME=bootstrap-toolchain/TARGET=gnulinux,arch=amd64,branch=master
> WORKSPACE=.../hudson/jobs/bootstrap-toolchain/workspace/TARGET/gnulinux/arch/amd64/branch/master
> I hope this report is clear about the issue.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.hudson-ci.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira