Add Swift Covers Native Methods for Mathy Godot Types, Add Testing Framework

[dyoustra]

Overview

We have been working on a general solution to issue #509 (Optimization: Move more vector operations to Swift). Although it’s not finished, we want to share our experiments to get some feedback.

We had two main goals:

• Write the Swift implementations as regular Swift source code in .swift files, not inside string literals embedded in Generator’s source code.
• Test each Swift implementation by comparing its output to the output of the Godot engine method it’s replacing.

This draft PR shows our ideas for implementing these two goals. It’s not intended for merging; there are failing tests and some incorrect implementations. It does show how our ideas let us easily write cover implementations and corresponding tests for all of the built-in geometric types, including vectors, transforms, basis, quaternion, plane, and the math utilities in GD.

Changes

• Generator reads the Swift source files in Sources/SwiftCovers and extracts method bodies from any extensions found in those files. Each method body is keyed by the extended type’s name and the name and type signature of the method. For example:

// Sources/SwiftCovers/Vector3.covers.swift

extension Vector3 {
    // ...

    public func cross(with: Vector3) -> Vector3 {
        return Vector3(
            x: (y * with.z) - (z * with.y),
            y: (z * with.x) - (x * with.z),
            z: (x * with.y) - (y * with.x)
        )
    }

    // ...
}

• When emitting methods, initializers, and operators for built-in types (like Vector3, Transform3D, etc.), Generator looks for an extracted method body with a matching key. If it finds one, it emits that method body as an alternative implementation of the method. This replaces the existing use of customBuiltinMethodImplementations and customBuiltinMethodImplementations. So the example above becomes the following:

// Generated Vector3.swift

public struct Vector3 {
    // ...

    public func cross(with: Vector3)-> Vector3 {
        #if CUSTOM_BUILTIN_IMPLEMENTATIONS
            do {
                return Vector3(
                    x: (y * with.z) - (z * with.y),
                    y: (z * with.x) - (x * with.z),
                    z: (x * with.y) - (y * with.x)
                )
            }
        #else // CUSTOM_BUILTIN_IMPLEMENTATIONS
        var result: Vector3 = Vector3()
        withUnsafePointer(to: with) { pArg0 in
            withUnsafePointer(to: UnsafeRawPointersN1(pArg0)) { pArgs in
                pArgs.withMemoryRebound(to: UnsafeRawPointer?.self, capacity: 1) { pArgs in
                    var mutSelfCopy = self
                    withUnsafeMutablePointer (to: &mutSelfCopy) { ptr in
                       Vector3.method_cross(ptr, pArgs, &result, 1)
                    }
                }
                
            }
            
        }
        
        return result
        #endif // CUSTOM_BUILTIN_IMPLEMENTATIONS
        
    }

    // ...
}

• If compiled with the condition TESTABLE_SWIFT_COVERS (set in Package.swift), Generator replaces the use of #if CUSTOM_BUILTIN_IMPLEMENTATIONS in the emitted methods with a normal if statement that checks a runtime-settable flag to decide whether to use the cover implementation or to call through to the engine.

public struct Vector3 {
    // ...

    public func cross(with: Vector3)-> Vector3 {
        if useSwiftCovers {
                do {
                    return Vector3(
                        x: (y * with.z) - (z * with.y),
                        y: (z * with.x) - (x * with.z),
                        z: (x * with.y) - (y * with.x)
                    )
                }
        } else {
            var result: Vector3 = Vector3()
            withUnsafePointer(to: with) { pArg0 in
                withUnsafePointer(to: UnsafeRawPointersN1(pArg0)) { pArgs in
                    pArgs.withMemoryRebound(to: UnsafeRawPointer?.self, capacity: 1) { pArgs in
                        var mutSelfCopy = self
                        withUnsafeMutablePointer (to: &mutSelfCopy) { ptr in
                           Vector3.method_cross(ptr, pArgs, &result, 1)
                        }
                    }
                    
                }
                
            }
            
            return result
        }
    }

    // ...
}

Each Swift cover method draws its implementation details from the Godot Engine code and the godot-cpp generated codebase.. Where helper methods for an extension are needed, they’ve been placed in SwiftCoverSupport.swift, including these:

• C-Conformant type conversions: The Swift and C compilers handle some signed Int32 operations differently when it comes to overflow, so we provide C-compatible versions
• Type-specific Tuple Conversions
• SwiftGodot Mathematical Constants
• Runtime Testing Flag
• And other type-specific helpers

Each file in SwiftCovers has an associated test file in Tests/SwiftGodotTests/BuiltIn. For each method that has a Swift cover, we generate (pseudo-)random inputs to feed to the method. We then call the method twice, once with the useSwiftCovers runtime flag set (to use the cover implementation) and again with the flag unset (to use the engine implementation). We then compare the outputs (with some tunable fuzziness for floating-point comparisons) to verify that each cover closely matches the engine.

We use a little combinator system named TinyGen (in TinyGen.swift) to generate the random inputs for testing. TinyGen is modeled on property-based testing frameworks like QuickCheck and Hedgehog, but without shrinking.
We use @TaskLocal properties for useSwiftCovers and for the floating-point fuzziness parameters Float.closeEnoughUlps and Double.closeEnoughUlps, because @TaskLocal lets us set up a global variable with an easy way to temporarily and dynamically override its value.

[migueldeicaza]

Thank you so much, this is wonderful!

I have one question about the covers, is there a reason to use covers and the system that goes with it injecting source code, rather than writing Swift extensions for those types? We could have the generator have a list of "Do not generate this code".

[mayoff]

I believe that is ultimately where SwiftGodot should end up.

The design we implemented here is a step in that direction. If we hand-write implementations for the geometric types, we need a way to test that those implementations are correct. What we've done here is treat the Godot engine as a test oracle. We can guarantee that we match its behavior as precisely as we want. We generate a bunch of random inputs for the tests, including weird edge cases with coordinate values like infinity, NaN, Int32.min, and Int32.max.

On the other hand, that does involve a bunch of infrastructure that we would ultimately want to discard, so it's entirely reasonable to keep this work in a separate fork where we can write and prove the Swift implementations, and them just copy them over to the main project without the test infrastructure.

[migueldeicaza]

I see, I see.

I love the direction then, nothing else to add at this point, and I appreciate the strong focus on testing and testing this.