Debug.LogErrors are landing in the production environment, despite SentryOptionsConfigurationObject settings

[wyattPD]

Environment

How do you use Sentry?
Sentry SaaS (sentry.io) or self-hosted/on-premise (which version?)

We use Sentry SaaS.

Which version of the SDK?

2.4.0

How did you install the package? (Git-URL, Assetstore)
Git-URL

Which version of Unity?
2022.3.37f1

Is this happening in Unity (editor) or on a player like Android, iOS, Windows?
Android, iOS

Steps to Reproduce

1. Create a new SentryOptionsConfigurationObject via the Sentry SDK UI (this replaces the Build/Run time configuration objects)
2. This object sets the Environment field to pd_dev or pd_prod depending on the context of the build. We do not use the word production which is Sentry's default bucket if we left this blank.
3. This object also logs what environment it's in via a Debug.Log eg: "Hello I am in the [dev] environment"
4. Add a button in a scene that runs Debug.LogError("Hello, i am error!")
5. Add a button in the scene that runs UnityEngine.Diagnostics.Utils.ForceCrash(ForcedCrashCategory.Abort) (this will force the app to crash)
6. Build the game for iOS or Android.
7. Run the game on device, press each button

Expected Result

At build-time:

• I see the message telling me what environment I'm in. (eg: "Hello I am the [prod] environment")

At runtime, on device:

• I see the same log message (I'm using SRDebugger to see my console logs on-device).
• Any Debug.LogError, Exception, or Crash should show up under the pd_prod / pd_dev environment.

Actual Result

At build-time:

• ✅ I see the message telling me what environment I'm in. (eg: "Hello I am the [prod] environment")

At runtime, on device:

• 🟨 I do not see the log messages telling me what environment I'm in. Implying the SentryOptionsConfigurationObject did not execute at runtime.
• ✅ When the app crashes: The correct thing happens, the crash report shows up under the pd_dev / pd_prod environment corresponding to how the project was built.
• 🔴 When the app errors (either by throwing an exception or by Debug.LogError) the log message shows up in production -- as if I didn't set an environment at all!

Known workaround

I've found as a workaround I can setup a Scope with the environment and that behaves as expected. However, I can't setup the scope right away, so some errors are still leaking into production. When errors leak into production I can't tell if they were from prod or dev.

[bitsandfoxes]

Hey @wyattPD, thanks for the detailed issue description! The way you expect the SDK to behave is exactly how it should.
Unfortunately, with the current 3.0.0-beta.0 I'm unable to reproduce this behaviour any which way.

I do get the log during build

and I do get the log during runtime

And issues reported from either layer both have the environment set.
Native Crash:

C# Exception

Could you give the current version a try? Or maybe provide a minimal repro so I can debug this locally?

[wyattPD]

Thanks for the quick response @bitsandfoxes I'll see what I can do.

[wyattPD]

Hi again! I'm trying to use 3.0.0-beta.0 as you suggested and I'm hitting a snag. It looks like something in the SentryNativeBridge is not compiling for iOS.

Here's the chunk of the relevant log:

<REDACTED>/data/jenkins/workspace/<REDACTED>/iOS/Build/iOS/Libraries/io.sentry.unity/SentryNativeBridge.m:37:9: error: performSelector names a selector which retains the object
   37 |         performSelector:@selector(initWithDict:didFailWithError:)
      |         ^
In module 'Sentry' imported from <REDACTED>/data/jenkins/workspace/<REDACTED>/iOS/Build/iOS/Libraries/io.sentry.unity/SentryNativeBridge.m:1:
<REDACTED>/Library/Developer/Xcode/DerivedData/Unity-iPhone-cseycvbiwfelbyekvlwxliadxfvi/Build/Intermediates.noindex/ArchiveIntermediates/Unity-iPhone/BuildProductsPath/Release-iphoneos/Sentry.framework/PrivateHeaders/SentryOptions+HybridSDKs.h:11:1: note: method 'initWithDict:didFailWithError:' declared here
   11 | - (_Nullable instancetype)initWithDict:(NSDictionary<NSString *, id> *)options
      | ^
1 error generated.

I am unable to complete an iOS build to test 3.0.0-beta.0 because of this issue.
It appears Android does not have this issue so I'll test there and get back to you.

[ysle0]

@wyattPD
I fixed PerformSelector error in another way: using selector -> direct call
and it worked for me!

fyi I use 2022.3.41f1 unity3D

// original
//void SentryNativeBridgeStartWithOptions(const void *options)
//{
//    NSMutableDictionary *dictOptions = (__bridge_transfer NSMutableDictionary *)options;
//    id sentryOptions = [[SentryOptions alloc]
//        performSelector:@selector(initWithDict:didFailWithError:)
//        withObject:dictOptions withObject:nil];
//
//    [SentrySDK performSelector:@selector(startWithOptions:) withObject:sentryOptions];
//}

// fixed
void SentryNativeBridgeStartWithOptions(const void *options)
{
    NSMutableDictionary *dictOptions = (__bridge_transfer NSMutableDictionary *)options;
    NSError *error = nil;
    SentryOptions *sentryOptions = [[SentryOptions alloc] initWithDict:dictOptions didFailWithError:&error];

    if (error) {
        NSLog(@"Error initializing SentryOptions: %@", error);
        return;
    }

    [SentrySDK startWithOptions:sentryOptions];
}

[bitsandfoxes]

Thanks for that. The fix for the iOS native bridge issue is coming with #1963

[ysle0]

@bitsandfoxes could you mention my github id on that issue?
It can be my first trial for OSS and I want to be part of the contribution!
I hope this sounds sincerely not greedy for it!

[bitsandfoxes]

@bitsandfoxes could you mention my github id on that issue?
It can be my first trial for OSS and I want to be part of the contribution!
I hope this sounds sincerely not greedy for it!

Of course! Do you want to create your own PR instead? I'm more than happy to merge it.

[ysle0]

I feel honored to create my first PR with your help!
It's happening and I'm doing it!

[wyattPD]

Hi again! Thanks for the fast turnaround on the fix! How should I pull in this change to test my build? Is there a different branch I should point to than 3.0.0-beta.0?

[bitsandfoxes]

I'm working on getting a 3.0.0-beta.1 out today. I hope CI looks at me favourably.