Debugging swift testing tests fails to attach

[adam-fowler]

Describe the bug
If I try to debug a swift testing test the debugger fails to attach to the process.

A dialog opens with the following text

process exited with status -1 (attach failed (Not allowed to attach to process. Look in the console messages (Console.app), near the debugserver entries, when the attach failed. The subsystem that denied the attach permission will likely have logged an informative message about why it was denied.))

To Reproduce
Steps to reproduce the behavior:

1. Open project with swift-testing tests
2. Run debug tests

Expected behavior
You should be able to test swift-testing tests

Environment

• OS: macOS 15.0
• Swift version: Apple Swift version 6.0 (swiftlang-6.0.0.9.10 clang-1600.0.26.2)
• Visual Studio Code version: 1.94.2
• vscode-swift version: 1.11.3

This is the debug console output

Console is in 'commands' mode, prefix expressions with '?'.
Launching: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/libexec/swift/pm/swiftpm-testing-helper --test-bundle-path /Users/adamfowler/Developer/server/soto-project/soto-cognito-authentication-kit/.build/arm64-apple-macosx/debug/soto-cognito-authentication-kitPackageTests.xctest/Contents/MacOS/soto-cognito-authentication-kitPackageTests --testing-library swift-testing --enable-swift-testing --event-stream-version 0 --event-stream-output-path /var/folders/6v/zhyq15kx1fz1d9hjv1zq9n8c0000gn/T/vscodemkfifo-1729201780878 --filter SotoCognitoAuthenticationKitTests.* -Xswiftc -diagnostic-style=llvm

Minimal diagnostic bundle

[plemarquand]

I think this may be a duplicate of https://github.com/swiftlang/vscode-swift/issues/1089.

[adam-fowler]

How come we are using swiftpm-testing-helper to run the swift-testing executables? I thought you could run the swift testing executables directly. Did this change?

[plemarquand]

(I accidentally posted the response to this on the wrong issue (#1148), moving here)

Its there to support a single binary test product for projects that contain both XCTest and swift-testing tests. More information here: swiftlang/swift-package-manager#7766

More context on the swiftpm-testing-helper from @grynspan:

So, on Linux and Windows, the test product is a flat executable and runs directly. SwiftPM emits a main() function into the executable that calls into XCTest or into Swift Testing depending on what arguments are passed. There is no helper executable needed.

Now, one of our second-tier goals with Swift Testing was for it to share as much logic as possible across platforms. So on all platforms regardless of what kind of test product is produced, the main() function Swift Testing emits into the binary is identical.

However, on Darwin, the test product expected by XCTest is a loadable, but not executable, bundle. The main() function from such a bundle cannot be directly executed—it's just another exported symbol in a library, after all.

Xcode includes an executable, aptly named xctest, that knows how to open these bundles, find XCTest tests, and run them. (There are also other ways to run tests via Xcode but they're not relevant to SwiftPM.) XCTest doesn't use the main() function at all! It also doesn't know anything about Swift Testing or how to discover or run its tests.

Enter swiftpm-testing-helper: it loads an arbitrary loadable bundle/library, finds its main() function, and calls it. It does nothing else. The main() function, being emitted into the test product exactly the same for Darwin as for Linux and Windows, takes over and runs Swift Testing tests.

In a perfect world, Darwin would let us make .xctest bundles both loadable and executable, and we wouldn't need the helper, but for other unrelated reasons that would be a terrible and dangerous idea, so them's the breaks.