Allow configurable stacktrace encoding #1371

arwineap · 2023-10-09T17:14:39Z

This PR adds a StacktraceEncoder type to the EncoderConfig struct

Users can override the EncodeStacktrace field to configure how stacktraces are outputed This PR aims to resolve #514 by mirroring the behavior of the EncodeCaller field

The EncodeStacktrace field has been inserted as a required field, and has been added to NewDevelopmentConfig and NewProductionConfig as sane defaults however, as a required field it can cause a panic if a user is manually building their config without these helpers.

CLAassistant · 2023-10-09T17:14:49Z

All committers have signed the CLA.

codecov · 2023-10-10T12:39:37Z

Codecov Report

Merging #1371 (cad62a8) into master (87577d8) will decrease coverage by 0.13%.
The diff coverage is 84.12%.

❗ Current head cad62a8 differs from pull request most recent head d45920a. Consider uploading reports for the commit d45920a to get more accurate results

@@            Coverage Diff             @@
##           master    #1371      +/-   ##
==========================================
- Coverage   98.28%   98.15%   -0.13%     
==========================================
  Files          53       53              
  Lines        3493     3527      +34     
==========================================
+ Hits         3433     3462      +29     
- Misses         50       55       +5     
  Partials       10       10

Files	Coverage Δ
config.go	`100.00% <100.00%> (ø)`
zapcore/json_encoder.go	`100.00% <100.00%> (ø)`
zapcore/console_encoder.go	`97.05% <78.57%> (-2.95%)`	⬇️
zapcore/encoder.go	`83.45% <22.22%> (-4.24%)`	⬇️

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

abhinav

Thanks for the PR, @arwineap! This change looks largely in the right direction.
However:

The EncodeStacktrace field has been inserted as a required field [..] it can cause a panic if a user is manually building their config without these helpers

That's a breaking change and we can't merge that. The new field must be optional.
The console and JSON encoders already have handling for optional overrides like this one; this should be another such optional field. For example:

zap/zapcore/json_encoder.go

Lines 381 to 387 in 87577d8

    
           nameEncoder := final.EncodeName 
        
           // if no name encoder provided, fall back to FullNameEncoder for backwards 
        
           // compatibility 
        
           if nameEncoder == nil { 
        
           	nameEncoder = FullNameEncoder 
        
           }

zapcore/encoder.go

arwineap · 2023-10-10T17:03:36Z

Thank you for the clarification, I'm unsure how I missed the default catch for the nil cases. This has been added.

One other concern I had was with the console encoder; I'm not sure I can avoid calling getSliceEncoder() a second time because of the split in output between structured fields and unstructured panic data. This felt ineffecient, but the code seems cleaner than the alternative

abhinav

This is great, thanks. I've left some minor comments.

I have a couple open questions that I maybe need input from other maintainers on.

Should the encoder be primitives-based like the other encoder overrides or a full one capable of posting objects -- e.g. if someone parses the stack trace?
Should the fallback behavior be to encode the full stack trace, or should we take that as an indication that they don't want to encode the stack trace?

CC @prashantv @sywhang @mway

zapcore/encoder.go

abhinav · 2023-10-11T11:57:11Z

zapcore/json_encoder.go

+		if cur == final.buf.Len() {
+			// User-supplied EncodeStacktrace was a no-op. Fall back to strings to
+			// keep output JSON valid.
+			final.AppendString("")


Did you mean to pass ent.Stack here?

Suggested change

final.AppendString("")

final.AppendString(ent.Stack)

Good catch, this was actually intentional.

The idea being that a user can disable stacktrace key / values in json output by configuring StacktraceKey to be blank

However, if the user created the custom encoder which emptied out the stack trace, it seems like we should respect their key / value, and append their StacktraceKey with an empty string

The issue is that at this part of the code the StacktraceKey has already been added to the json output, so we need a string value to make the json valid, so an empty string makes sense. Perfectly reasonable to only add the Key if a value exists, just need some rearranging

It's a matter of design, and it's more important that it's consistent across all of the options, so I will defer to the zap team here 👍

abhinav · 2023-10-11T12:02:20Z

zapcore/console_encoder.go

+		for i := range arr.elems {
+			if i > 0 {
+				line.AppendString(c.ConsoleSeparator)
+			}
+			fmt.Fprint(line, arr.elems[i])
+		}


The contract for stacktraceEncoder is that it makes at most one Append* call.
So we don't need the full arr.elems handling here. It can just be:

Suggested change

for i := range arr.elems {

if i > 0 {

line.AppendString(c.ConsoleSeparator)

}

fmt.Fprint(line, arr.elems[i])

}

if len(arr.elems) > 0 {

fmt.Fprint(line, arr.elems[0])

} else {

// User-provided encoder was a no-op.

// Fall back to full encoder.

line.AppendString(ent.Stack)

}

The first AppendString in this call is actually a way to add the ConsoleSeparator between each line; it could probably be integrated into the Fprint to reduce branching here

abhinav · 2023-10-11T12:09:48Z

Related: We'll want to make a decision on #1373 before we release this because choices there affect the design for this.

devdrops · 2024-08-07T19:36:45Z

@abhinav @arwineap hi! Any news about this PR? Responses are really appreciated 🙂

arwineap · 2024-08-08T03:22:44Z

I will work to rebase this with modern changes; but depending on any decision in #1373 we may try to send the entire stack object into the function instead of the string

This PR adds a `StacktraceEncoder` type to the `EncoderConfig` struct Users can override the `EncodeStacktrace` field to configure how stacktraces are outputed This PR aims to resolve uber-go#514 by mirroring the behavior of the `EncodeCaller` field The `EncodeStacktrace` field has been inserted as a required field, and has been added to `NewDevelopmentConfig` and `NewProductionConfig` as sane defaults however, it is currently a required field and can cause a panic if a user is manually building their config without these helpers.

This is now unneeded because `EncodeStacktrace` will not be nil

improve test cases

Co-authored-by: Abhinav Gupta <[email protected]>

win5do · 2024-12-27T06:10:14Z

We need this API, the stacktrace contains newlines and tabs, which are very difficult to read in json.

abhinav reviewed Oct 10, 2023

View reviewed changes

zapcore/encoder.go Outdated Show resolved Hide resolved

zapcore/encoder.go Show resolved Hide resolved

abhinav reviewed Oct 11, 2023

View reviewed changes

abhinav mentioned this pull request Oct 11, 2023

Preserve the dedicated stack trace object all the way to encoder #1373

Open

arwineap and others added 7 commits October 19, 2024 13:27

Prevent unconfigured stacktraceEncoder from panicing

0ee3754

Adding ability to unmarshal text to a StacktraceEncoder for configs

5c2de5f

removing unneeded nil check

ffcd042

This is now unneeded because `EncodeStacktrace` will not be nil

Fix nil and noop EncodeStacktraces

90a7936

improve test cases

Update zapcore/encoder.go

4d83547

Co-authored-by: Abhinav Gupta <[email protected]>

Update zapcore/encoder.go

a7fe65f

Co-authored-by: Abhinav Gupta <[email protected]>

arwineap force-pushed the configurable-stacktrace-encoding branch from d45920a to a7fe65f Compare October 19, 2024 20:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow configurable stacktrace encoding #1371

Allow configurable stacktrace encoding #1371

arwineap commented Oct 9, 2023

CLAassistant commented Oct 9, 2023 •

edited

Loading

codecov bot commented Oct 10, 2023 •

edited

Loading

abhinav left a comment

arwineap commented Oct 10, 2023

abhinav left a comment

abhinav Oct 11, 2023

arwineap Oct 11, 2023

abhinav Oct 11, 2023

arwineap Oct 11, 2023

abhinav commented Oct 11, 2023

devdrops commented Aug 7, 2024 •

edited

Loading

arwineap commented Aug 8, 2024

win5do commented Dec 27, 2024

	nameEncoder := final.EncodeName

	// if no name encoder provided, fall back to FullNameEncoder for backwards
	// compatibility
	if nameEncoder == nil {
	nameEncoder = FullNameEncoder
	}

-		for i := range arr.elems {
-			if i > 0 {
-				line.AppendString(c.ConsoleSeparator)
-			}
-			fmt.Fprint(line, arr.elems[i])
-		}
+		if len(arr.elems) > 0 {
+			fmt.Fprint(line, arr.elems[0])
+		} else {
+			// User-provided encoder was a no-op.
+			// Fall back to full encoder.
+			line.AppendString(ent.Stack)
+		}

Allow configurable stacktrace encoding #1371

Are you sure you want to change the base?

Allow configurable stacktrace encoding #1371

Conversation

arwineap commented Oct 9, 2023

CLAassistant commented Oct 9, 2023 • edited Loading

codecov bot commented Oct 10, 2023 • edited Loading

Codecov Report

abhinav left a comment

Choose a reason for hiding this comment

arwineap commented Oct 10, 2023

abhinav left a comment

Choose a reason for hiding this comment

abhinav Oct 11, 2023

Choose a reason for hiding this comment

arwineap Oct 11, 2023

Choose a reason for hiding this comment

abhinav Oct 11, 2023

Choose a reason for hiding this comment

arwineap Oct 11, 2023

Choose a reason for hiding this comment

abhinav commented Oct 11, 2023

devdrops commented Aug 7, 2024 • edited Loading

arwineap commented Aug 8, 2024

win5do commented Dec 27, 2024

CLAassistant commented Oct 9, 2023 •

edited

Loading

codecov bot commented Oct 10, 2023 •

edited

Loading

devdrops commented Aug 7, 2024 •

edited

Loading