Fwd: [DISCUSS] Change Jenkins default logging format

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: [DISCUSS] Change Jenkins default logging format

Baptiste MATHUS
Hello everyone,

I am transferring this email I originally posted in the dev list a few days ago to gather more feedback.
To sum up: I am proposing to change the current logging format to a more consise and machine-friendly one-liner (when there's no stacktrace)

If you wish to see it changed, or not, please comment on the subject detailed below.

Thanks!

---------- Forwarded message ----------
From: Baptiste Mathus <[hidden email]>
Date: 2018-04-04 15:17 GMT+02:00
Subject: [DISCUSS] Change Jenkins default logging format
To: Jenkins Developers <[hidden email]>


Hello everyone,

Having worked on more things related to Jenkins logging recently, I've had the opportunity to remember my past pain when I was operating a Jenkins instance and sending logs to an ELK cluster.
Compared to almost everything else in the infrastructure, the logstash rules for Jenkins logs were unnecessarily complex.

The main pain-points, for me at least, had been the two-lines (sigh) per log default, and also the localized date format (or log level...). 
Even now, so many years after reading those, I still struggle daily to make sure I'm reading the right line/date associated to the message I'm reading on the line above.

I would like to propose we change the current logging format behavior to a more readable and more operation-friendly one.
This would result in something close to the following format:

[   INFO][2018-04-04 12:40:49] Logging initialized @180ms to org.eclipse.jetty.util.log.JavaUtilLog (from org.eclipse.jetty.util.log.Log initialized)
[   INFO][2018-04-04 12:40:49] Beginning extraction from war file (from winstone.Logger logInternal)
[WARNING][2018-04-04 12:40:49] Empty contextPath (from org.eclipse.jetty.server.handler.ContextHandler setContextPath)
[   INFO][2018-04-04 12:40:49] jetty-9.4.z-SNAPSHOT (from org.eclipse.jetty.server.Server doStart)

Instead of the usual:

Apr 04, 2018 12:36:41 PM org.eclipse.jetty.util.log.Log initialized
INFO: Logging initialized @354ms to org.eclipse.jetty.util.log.JavaUtilLog
Apr 04, 2018 12:36:41 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.handler.ContextHandler setContextPath
WARNING: Empty contextPath
Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.4.z-SNAPSHOT


WDYT?

If this looks interesting to people, I'm ready to file the associated JEP for it and possibly work on its implementation in the future.

Obviously, we would need some way to revert to the "legacy" format, at least for some time for users to adapt. But that is not something I'm particularly worried about.

-- Baptiste

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS60enT9U8n4Mu1X7dmifixdnKoU0303GZwOKZCvQKeAhA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

RE: [DISCUSS] Change Jenkins default logging format

Reinhold Fuereder

Hi Baptiste,

 

Definitely +1! That is I wish to see the default logging format changed to a one-liner.

 

 

This is more or less what in our Ansible based provisioning of Jenkins setup is already done:

 

# See http://stackoverflow.com/questions/194765/how-do-i-get-java-logging-output-to-appear-on-a-single-line

- name: setup | Configure log output format

  lineinfile: dest=/etc/default/jenkins

              insertafter='^JAVA_ARGS=.*'

              regexp='^JAVA_ARGS.*java.util.logging.*'

              line="JAVA_ARGS=\"$JAVA_ARGS -Djava.util.logging.SimpleFormatter.format='%1\$tY-%1\$tm-%1\$td %1\$tH:%1\$tM:%1\$tS %4\$4s [%2\$s]   %5\$s%6\$s%n'\""

 

 

Regards,

                Reinhold

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/VI1P190MB0255607BD409E578C2CCB3D1F7BC0%40VI1P190MB0255.EURP190.PROD.OUTLOOK.COM.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

RE: [DISCUSS] Change Jenkins default logging format

Osborn, Tammy (DNR)
In reply to this post by Baptiste MATHUS

This sounds like a great idea to me. I haven’t had to deal with the logs much but simpler is better when there’s no stack trace like Baptiste suggested.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Baptiste Mathus
Sent: Thursday, April 12, 2018 12:41 AM
To: [hidden email]
Subject: Fwd: [DISCUSS] Change Jenkins default logging format

 

Hello everyone,

 

I am transferring this email I originally posted in the dev list a few days ago to gather more feedback.

To sum up: I am proposing to change the current logging format to a more consise and machine-friendly one-liner (when there's no stacktrace)

 

If you wish to see it changed, or not, please comment on the subject detailed below.

 

Thanks!

 

---------- Forwarded message ----------
From: Baptiste Mathus <[hidden email]>
Date: 2018-04-04 15:17 GMT+02:00
Subject: [DISCUSS] Change Jenkins default logging format
To: Jenkins Developers <[hidden email]>

Hello everyone,

 

Having worked on more things related to Jenkins logging recently, I've had the opportunity to remember my past pain when I was operating a Jenkins instance and sending logs to an ELK cluster.

Compared to almost everything else in the infrastructure, the logstash rules for Jenkins logs were unnecessarily complex.

 

The main pain-points, for me at least, had been the two-lines (sigh) per log default, and also the localized date format (or log level...). 

Even now, so many years after reading those, I still struggle daily to make sure I'm reading the right line/date associated to the message I'm reading on the line above.

 

I would like to propose we change the current logging format behavior to a more readable and more operation-friendly one.

This would result in something close to the following format:

 

[   INFO][2018-04-04 12:40:49] Logging initialized @180ms to org.eclipse.jetty.util.log.JavaUtilLog (from org.eclipse.jetty.util.log.Log initialized)

[   INFO][2018-04-04 12:40:49] Beginning extraction from war file (from winstone.Logger logInternal)

[WARNING][2018-04-04 12:40:49] Empty contextPath (from org.eclipse.jetty.server.handler.ContextHandler setContextPath)

[   INFO][2018-04-04 12:40:49] jetty-9.4.z-SNAPSHOT (from org.eclipse.jetty.server.Server doStart)

 

Instead of the usual:

 

Apr 04, 2018 12:36:41 PM org.eclipse.jetty.util.log.Log initialized

INFO: Logging initialized @354ms to org.eclipse.jetty.util.log.JavaUtilLog

Apr 04, 2018 12:36:41 PM winstone.Logger logInternal

INFO: Beginning extraction from war file

Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.handler.ContextHandler setContextPath

WARNING: Empty contextPath

Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.Server doStart

INFO: jetty-9.4.z-SNAPSHOT

 

 

WDYT?

 

If this looks interesting to people, I'm ready to file the associated JEP for it and possibly work on its implementation in the future.

 

Obviously, we would need some way to revert to the "legacy" format, at least for some time for users to adapt. But that is not something I'm particularly worried about.

 

-- Baptiste

 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS60enT9U8n4Mu1X7dmifixdnKoU0303GZwOKZCvQKeAhA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/19E7B5A7F96E1C48B658A85C9D50910C4CA7B1CB%40WAXMXOLYMB011.WAX.wa.lcl.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Change Jenkins default logging format

Baptiste MATHUS
Sorry, I was unclear. There *will* still be stack traces IMO, they are critical to understand what is happening.
I meant: the logs will typically be one-line. 
But when a given log entry has an associated exception, then it will be printed mostly like now, i.e. with a variable number of lines. 

Joining lines when there are exceptions with log-analysis tools like logstash is mostly trivial, you typically detect "at ...", etc. and tell it that it goes with the previous log line.

Note that I guess we _could_ consider making the stacktraces possible to disable, but I think this is a bad idea. 
When something goes wrong, this is the first thing typically, for instance on this mailing list, that we would ask reporters to re-enable and come back when the issue has reoccurred with (or in JIRA reports).

Cheers

2018-04-12 15:21 GMT+02:00 Osborn, Tammy (DNR) <[hidden email]>:

This sounds like a great idea to me. I haven’t had to deal with the logs much but simpler is better when there’s no stack trace like Baptiste suggested.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Baptiste Mathus
Sent: Thursday, April 12, 2018 12:41 AM
To: [hidden email]
Subject: Fwd: [DISCUSS] Change Jenkins default logging format

 

Hello everyone,

 

I am transferring this email I originally posted in the dev list a few days ago to gather more feedback.

To sum up: I am proposing to change the current logging format to a more consise and machine-friendly one-liner (when there's no stacktrace)

 

If you wish to see it changed, or not, please comment on the subject detailed below.

 

Thanks!

 

---------- Forwarded message ----------
From: Baptiste Mathus <[hidden email]>
Date: 2018-04-04 15:17 GMT+02:00
Subject: [DISCUSS] Change Jenkins default logging format
To: Jenkins Developers <[hidden email]>

Hello everyone,

 

Having worked on more things related to Jenkins logging recently, I've had the opportunity to remember my past pain when I was operating a Jenkins instance and sending logs to an ELK cluster.

Compared to almost everything else in the infrastructure, the logstash rules for Jenkins logs were unnecessarily complex.

 

The main pain-points, for me at least, had been the two-lines (sigh) per log default, and also the localized date format (or log level...). 

Even now, so many years after reading those, I still struggle daily to make sure I'm reading the right line/date associated to the message I'm reading on the line above.

 

I would like to propose we change the current logging format behavior to a more readable and more operation-friendly one.

This would result in something close to the following format:

 

[   INFO][2018-04-04 12:40:49] Logging initialized @180ms to org.eclipse.jetty.util.log.JavaUtilLog (from org.eclipse.jetty.util.log.Log initialized)

[   INFO][2018-04-04 12:40:49] Beginning extraction from war file (from winstone.Logger logInternal)

[WARNING][2018-04-04 12:40:49] Empty contextPath (from org.eclipse.jetty.server.handler.ContextHandler setContextPath)

[   INFO][2018-04-04 12:40:49] jetty-9.4.z-SNAPSHOT (from org.eclipse.jetty.server.Server doStart)

 

Instead of the usual:

 

Apr 04, 2018 12:36:41 PM org.eclipse.jetty.util.log.Log initialized

INFO: Logging initialized @354ms to org.eclipse.jetty.util.log.JavaUtilLog

Apr 04, 2018 12:36:41 PM winstone.Logger logInternal

INFO: Beginning extraction from war file

Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.handler.ContextHandler setContextPath

WARNING: Empty contextPath

Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.Server doStart

INFO: jetty-9.4.z-SNAPSHOT

 

 

WDYT?

 

If this looks interesting to people, I'm ready to file the associated JEP for it and possibly work on its implementation in the future.

 

Obviously, we would need some way to revert to the "legacy" format, at least for some time for users to adapt. But that is not something I'm particularly worried about.

 

-- Baptiste

 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS60enT9U8n4Mu1X7dmifixdnKoU0303GZwOKZCvQKeAhA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/19E7B5A7F96E1C48B658A85C9D50910C4CA7B1CB%40WAXMXOLYMB011.WAX.wa.lcl.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS6Cie7RNu7aBXSsHRRJPuwzVtQT_uBr-epa9oVT7MNRpg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

RE: [DISCUSS] Change Jenkins default logging format

Osborn, Tammy (DNR)

I agree. My reply wasn’t well worded. I’d like to have the complete stacktraces included but when it’s just a routine message, have 1 liner.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Baptiste Mathus
Sent: Thursday, April 12, 2018 7:28 AM
To: [hidden email]
Subject: Re: [DISCUSS] Change Jenkins default logging format

 

Sorry, I was unclear. There *will* still be stack traces IMO, they are critical to understand what is happening.

I meant: the logs will typically be one-line. 

But when a given log entry has an associated exception, then it will be printed mostly like now, i.e. with a variable number of lines. 

 

Joining lines when there are exceptions with log-analysis tools like logstash is mostly trivial, you typically detect "at ...", etc. and tell it that it goes with the previous log line.

 

Note that I guess we _could_ consider making the stacktraces possible to disable, but I think this is a bad idea. 

When something goes wrong, this is the first thing typically, for instance on this mailing list, that we would ask reporters to re-enable and come back when the issue has reoccurred with (or in JIRA reports).

 

Cheers

 

2018-04-12 15:21 GMT+02:00 Osborn, Tammy (DNR) <[hidden email]>:

This sounds like a great idea to me. I haven’t had to deal with the logs much but simpler is better when there’s no stack trace like Baptiste suggested.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Baptiste Mathus
Sent: Thursday, April 12, 2018 12:41 AM
To: [hidden email]
Subject: Fwd: [DISCUSS] Change Jenkins default logging format

 

Hello everyone,

 

I am transferring this email I originally posted in the dev list a few days ago to gather more feedback.

To sum up: I am proposing to change the current logging format to a more consise and machine-friendly one-liner (when there's no stacktrace)

 

If you wish to see it changed, or not, please comment on the subject detailed below.

 

Thanks!

 

---------- Forwarded message ----------
From: Baptiste Mathus <[hidden email]>
Date: 2018-04-04 15:17 GMT+02:00
Subject: [DISCUSS] Change Jenkins default logging format
To: Jenkins Developers <[hidden email]>

Hello everyone,

 

Having worked on more things related to Jenkins logging recently, I've had the opportunity to remember my past pain when I was operating a Jenkins instance and sending logs to an ELK cluster.

Compared to almost everything else in the infrastructure, the logstash rules for Jenkins logs were unnecessarily complex.

 

The main pain-points, for me at least, had been the two-lines (sigh) per log default, and also the localized date format (or log level...). 

Even now, so many years after reading those, I still struggle daily to make sure I'm reading the right line/date associated to the message I'm reading on the line above.

 

I would like to propose we change the current logging format behavior to a more readable and more operation-friendly one.

This would result in something close to the following format:

 

[   INFO][2018-04-04 12:40:49] Logging initialized @180ms to org.eclipse.jetty.util.log.JavaUtilLog (from org.eclipse.jetty.util.log.Log initialized)

[   INFO][2018-04-04 12:40:49] Beginning extraction from war file (from winstone.Logger logInternal)

[WARNING][2018-04-04 12:40:49] Empty contextPath (from org.eclipse.jetty.server.handler.ContextHandler setContextPath)

[   INFO][2018-04-04 12:40:49] jetty-9.4.z-SNAPSHOT (from org.eclipse.jetty.server.Server doStart)

 

Instead of the usual:

 

Apr 04, 2018 12:36:41 PM org.eclipse.jetty.util.log.Log initialized

INFO: Logging initialized @354ms to org.eclipse.jetty.util.log.JavaUtilLog

Apr 04, 2018 12:36:41 PM winstone.Logger logInternal

INFO: Beginning extraction from war file

Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.handler.ContextHandler setContextPath

WARNING: Empty contextPath

Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.Server doStart

INFO: jetty-9.4.z-SNAPSHOT

 

 

WDYT?

 

If this looks interesting to people, I'm ready to file the associated JEP for it and possibly work on its implementation in the future.

 

Obviously, we would need some way to revert to the "legacy" format, at least for some time for users to adapt. But that is not something I'm particularly worried about.

 

-- Baptiste

 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS60enT9U8n4Mu1X7dmifixdnKoU0303GZwOKZCvQKeAhA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/19E7B5A7F96E1C48B658A85C9D50910C4CA7B1CB%40WAXMXOLYMB011.WAX.wa.lcl.


For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS6Cie7RNu7aBXSsHRRJPuwzVtQT_uBr-epa9oVT7MNRpg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/19E7B5A7F96E1C48B658A85C9D50910C4CA7B24A%40WAXMXOLYMB011.WAX.wa.lcl.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Change Jenkins default logging format

niristotle okram
In reply to this post by Baptiste MATHUS
Definitely yes. 

On Thu, Apr 12, 2018 at 2:40 AM, Baptiste Mathus <[hidden email]> wrote:
Hello everyone,

I am transferring this email I originally posted in the dev list a few days ago to gather more feedback.
To sum up: I am proposing to change the current logging format to a more consise and machine-friendly one-liner (when there's no stacktrace)

If you wish to see it changed, or not, please comment on the subject detailed below.

Thanks!

---------- Forwarded message ----------
From: Baptiste Mathus <[hidden email]>
Date: 2018-04-04 15:17 GMT+02:00
Subject: [DISCUSS] Change Jenkins default logging format
To: Jenkins Developers <[hidden email]>


Hello everyone,

Having worked on more things related to Jenkins logging recently, I've had the opportunity to remember my past pain when I was operating a Jenkins instance and sending logs to an ELK cluster.
Compared to almost everything else in the infrastructure, the logstash rules for Jenkins logs were unnecessarily complex.

The main pain-points, for me at least, had been the two-lines (sigh) per log default, and also the localized date format (or log level...). 
Even now, so many years after reading those, I still struggle daily to make sure I'm reading the right line/date associated to the message I'm reading on the line above.

I would like to propose we change the current logging format behavior to a more readable and more operation-friendly one.
This would result in something close to the following format:

[   INFO][2018-04-04 12:40:49] Logging initialized @180ms to org.eclipse.jetty.util.log.JavaUtilLog (from org.eclipse.jetty.util.log.Log initialized)
[   INFO][2018-04-04 12:40:49] Beginning extraction from war file (from winstone.Logger logInternal)
[WARNING][2018-04-04 12:40:49] Empty contextPath (from org.eclipse.jetty.server.handler.ContextHandler setContextPath)
[   INFO][2018-04-04 12:40:49] jetty-9.4.z-SNAPSHOT (from org.eclipse.jetty.server.Server doStart)

Instead of the usual:

Apr 04, 2018 12:36:41 PM org.eclipse.jetty.util.log.Log initialized
INFO: Logging initialized @354ms to org.eclipse.jetty.util.log.JavaUtilLog
Apr 04, 2018 12:36:41 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.handler.ContextHandler setContextPath
WARNING: Empty contextPath
Apr 04, 2018 12:36:42 PM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.4.z-SNAPSHOT


WDYT?

If this looks interesting to people, I'm ready to file the associated JEP for it and possibly work on its implementation in the future.

Obviously, we would need some way to revert to the "legacy" format, at least for some time for users to adapt. But that is not something I'm particularly worried about.

-- Baptiste

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS60enT9U8n4Mu1X7dmifixdnKoU0303GZwOKZCvQKeAhA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.



--
Regards
nirish okram

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAPzcO4jQvKjBdC_9R6%2BSQchbkr7j1UKcDnVObbmPXuOwcp5s1w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.