Global ConsoleLogFilter support in WorkflowRun

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Global ConsoleLogFilter support in WorkflowRun

Ted Xiao
I wrote a plugin tee the console log to a central location by implementing the ConsoleLogFilter,  but it doesn't work for WorkflowRun.
Compared Run.java and WorkflowRun.java

In Run.java
OutputStream logger = new FileOutputStream(getLogFile(), true);
// Global log filters
for (ConsoleLogFilter filter : ConsoleLogFilter.all()) {
    logger
= filter.decorateLogger(build, logger);
}



But there is no decorateLogger either in onLoad or Run method, is this by design or accepting pull quests to add the feature?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/e5b0ad7b-2e44-4e74-b638-996c12eeccd6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Global ConsoleLogFilter support in WorkflowRun

Jesse Glick-4
On Fri, Nov 18, 2016 at 5:48 PM, Ted Xiao <[hidden email]> wrote:
> I wrote a plugin tee the console log to a central location by implementing
> the ConsoleLogFilter,  but it doesn't work for WorkflowRun.

What are you trying to accomplish? If JENKINS-38381, that is not the
way to do it.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr24kDrHFzW0kct72r1GJdNu2J1%3D4BFVa%2BGp1hwS-qaeHQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Global ConsoleLogFilter support in WorkflowRun

Ted Xiao
I think it is different to JENKINS-38381, the filter I wrote works like Unix tee command which forward the console log to a 3rd party log analyse system for real time monitoring and alerting, here is the implement
https://github.com/jenkinsci/splunk-devops-plugin/blob/master/splunk-devops/src/main/java/com/splunk/splunkjenkins/TeeConsoleLogFilter.java



On Saturday, November 19, 2016 at 1:31:08 AM UTC+8, Jesse Glick wrote:
On Fri, Nov 18, 2016 at 5:48 PM, Ted Xiao <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="FikufAjcBgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">xia...@...> wrote:
> I wrote a plugin tee the console log to a central location by implementing
> the ConsoleLogFilter,  but it doesn't work for WorkflowRun.

What are you trying to accomplish? If JENKINS-38381, that is not the
way to do it.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/c4f711ca-5f45-41a3-ba30-824254e5bd34%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Global ConsoleLogFilter support in WorkflowRun

Jesse Glick-4
On Mon, Nov 21, 2016 at 4:53 AM, Ted Xiao <[hidden email]> wrote:
> I think it is different to JENKINS-38381, the filter I wrote works like Unix
> tee command which forward the console log to a 3rd party log analyse system
> for real time monitoring and alerting

JENKINS-38381 accomplishes exactly that.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2-bv%2B8nTxstbofrQYv2se9jMARv5jOn46vexvPmTWi9g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Global ConsoleLogFilter support in WorkflowRun

Ted Xiao
Correct me if I am wrong,  through the PR https://github.com/jenkinsci/workflow-job-plugin/pull/27/files#diff-5ac278e2951f186bb7e2f76208e6ed05R76 , there is a comment // TODO JENKINS-30777 decorate with ConsoleLogFilter.all()
Does that mean I need to use explicit declare such as 

withContext(new MyConsoleLogFilter()) {
    sh 'process'
}


On Tuesday, November 22, 2016 at 12:46:31 AM UTC+8, Jesse Glick wrote:
On Mon, Nov 21, 2016 at 4:53 AM, Ted Xiao <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="Brpf8lbFBwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">xia...@...> wrote:
> I think it is different to JENKINS-38381, the filter I wrote works like Unix
> tee command which forward the console log to a 3rd party log analyse system
> for real time monitoring and alerting

JENKINS-38381 accomplishes exactly that.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/d516b1a2-13da-4305-a5cb-d43232819c9e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Global ConsoleLogFilter support in WorkflowRun

Jesse Glick-4
On Tue, Nov 22, 2016 at 4:29 PM, Ted Xiao <[hidden email]> wrote:
> through the PR
> https://github.com/jenkinsci/workflow-job-plugin/pull/27/files#diff-5ac278e2951f186bb7e2f76208e6ed05R76
> , there is a comment // TODO JENKINS-30777 decorate with
> ConsoleLogFilter.all()

Yes—if and when a use case for applying a global
(`@Extension`-registered) `ConsoleLogFilter` arises. AFAIK there are
none; under JENKINS-38381, sending log results to an external system
uses a different API. (`ConsoleLogFilter` is involved in the
implementation, but these are locally created filters, not global
extensions.) See the downstream
https://github.com/jenkinsci/logstash-plugin/pull/18 for a (crude)
example.

Please note that development on this system is currently on hold. It
is functionally complete and has basic test coverage, but is not
production ready. I have not yet written up a friendly design guide.

> Does that mean I need to use explicit declare such as
>
> withContext(new MyConsoleLogFilter()) {
>     sh 'process'
> }

No, it is transparent to job definitions.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3-sK0cmhH8ZMaafmpqcSQYjQhv5sTJo1MvzQAhTex%2BDw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Global ConsoleLogFilter support in WorkflowRun

Ted Xiao
it is much clear now, thank you very much, Jesse! 
it is true that most plugins such as timestamper-plugin,mask-passwords-plugin, logstash-plugin used build wrapper to decorate logger, and I am a little surprised that I was the first one to "abuse" the global ConsoleLogFilter feature.
Thanks, I will add a simple wrapper for WorkFlowRun job

On Wednesday, November 23, 2016 at 12:00:12 AM UTC+8, Jesse Glick wrote:
On Tue, Nov 22, 2016 at 4:29 PM, Ted Xiao <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="lc20bWQRCAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">xia...@...> wrote:
> through the PR
> <a href="https://github.com/jenkinsci/workflow-job-plugin/pull/27/files#diff-5ac278e2951f186bb7e2f76208e6ed05R76" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fworkflow-job-plugin%2Fpull%2F27%2Ffiles%23diff-5ac278e2951f186bb7e2f76208e6ed05R76\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNF9DnFAk_H3FEMeIPAV3xpl5cJIUQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fworkflow-job-plugin%2Fpull%2F27%2Ffiles%23diff-5ac278e2951f186bb7e2f76208e6ed05R76\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNF9DnFAk_H3FEMeIPAV3xpl5cJIUQ&#39;;return true;">https://github.com/jenkinsci/workflow-job-plugin/pull/27/files#diff-5ac278e2951f186bb7e2f76208e6ed05R76
> , there is a comment // TODO JENKINS-30777 decorate with
> ConsoleLogFilter.all()

Yes—if and when a use case for applying a global
(`@Extension`-registered) `ConsoleLogFilter` arises. AFAIK there are
none; under JENKINS-38381, sending log results to an external system
uses a different API. (`ConsoleLogFilter` is involved in the
implementation, but these are locally created filters, not global
extensions.) See the downstream
<a href="https://github.com/jenkinsci/logstash-plugin/pull/18" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Flogstash-plugin%2Fpull%2F18\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG7qSrWJcYndCVEtJ90-4b5Kl1jeg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Flogstash-plugin%2Fpull%2F18\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG7qSrWJcYndCVEtJ90-4b5Kl1jeg&#39;;return true;">https://github.com/jenkinsci/logstash-plugin/pull/18 for a (crude)
example.

Please note that development on this system is currently on hold. It
is functionally complete and has basic test coverage, but is not
production ready. I have not yet written up a friendly design guide.

> Does that mean I need to use explicit declare such as
>
> withContext(new MyConsoleLogFilter()) {
>     sh 'process'
> }

No, it is transparent to job definitions.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/b13dd559-9515-46d8-bb66-0e9962d9ceab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Global ConsoleLogFilter support in WorkflowRun

Carles Capdevila Tejada
In reply to this post by Jesse Glick-4

Hello there,

 

I’m developing a plugin to perform regexp replacements on the Jenkins log output: https://github.com/SoftwareBuildService/log-file-filter-plugin

 

It is based on global settings to perform the regexps replacements for any job. I’ve written a custom pipeline step to perform this filtering (in a manner similar to https://issues.jenkins-ci.org/browse/JENKINS-27486) but this is not ideal for our use case. I would like the ConsoleLogFilter to filter pipeline logs without the need for any step.


Has there been any progress on the topic?

 

Thank you for your attention.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ed0a2b87-5a12-4393-95d7-b7c079900f6c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Global ConsoleLogFilter support in WorkflowRun

Jesse Glick-4
On Thu, Jul 20, 2017 at 7:50 AM, Carles Capdevila Tejada
<[hidden email]> wrote:
> Has there been any progress on the topic?

No. I filed JENKINS-45693 to reflect the fact that JENKINS-30777 on
its own was useless.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3n-pvS%2ByiTOGG9mYNMXF4HceyoTqhcE4qy_1of0YG6%3DA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.