Idea: sampling of logs #60
Comments
|
I will add some more ideas. We need several different loggers. One is for error happened, I need log every step information but if nothing happened, I want log nothing. Which could be done by mapError( yourLogger => saveLog...) Another one is for performance log. You log every time we call every IO or method, log running time. But don't write them out. Instead, in your logger you record average time cost and max one (with all steps log for that max one). |
|
@zhiwilliam do you want to open PR with one of your ideas? |
|
Ah, sorry, I am not sure if I can provide PR since I am currently working
in a big company. Need to check with them to see if there's any restriction.
I did find you have a very closed implementation as the same idea as I had
in your test code. :) So I shall say sorry about that I didn't find it. I
have some code already to realize my idea. If my company allow will PR.
Best Regards,
William
…On Wed, Nov 18, 2020 at 12:04 AM przemyslaw wierzbicki < ***@***.***> wrote:
@zhiwilliam <https://github.com/zhiwilliam> do you want to open PR with
one of those?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#60 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACXX6HZGN7EOKFU5VW7HP6DSQNIUHANCNFSM4LHE3SLA>
.
|
|
feel free to reach me out on discord maybe we could put something together |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi!
Something I often struggle with is getting the log levels in production right. Either I get way too much noise, or run the risk of not having enough context to efficiently debug production errors.
One approach to improve this that I've used in the past is to do log sampling on a per-request-basis. It more or less works by buffering all log calls during the duration of the request, but only actually emits ~1% of requests' logs if nothing goes wrong. But, if a log line with level
warn >=is logged, emit everything that's been buffered. This gives you the ability to use liberal debug logging in production while still keeping your log volumes at manageable levels.I realize that this may very well be outside of the scope for this project, but wanted to open an issue for discussion!
The text was updated successfully, but these errors were encountered: