Swift incremental compilation does not work with large modules #1291
Comments
|
I don't actually think that it works for any kind of module but I didn't find a good way to verify that assumption. |
mattrobmattrob
pushed a commit
that referenced
this issue
Jan 17, 2025
Since we extracting `swiftconstvalues` from #1170 But we forgot to update the path for swiftconstvalues in the incremental output file map. It leads to the incremental feature not working. For example, demo: https://github.com/vikage/DemoBazelSwiftIncremental When I change file B.swift (No other file uses struct B) but log shows build A.swift and C.swift as well I used `-driver-show-job-lifecycle` and `-driver-show-incremental` flags to show debug log ``` WARNING: Build option --swiftcopt has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed). INFO: Analyzed target //:Demo (99 packages loaded, 1088 targets configured). INFO: From Compiling Swift module //:Demo: remark: Incremental compilation: Read dependency graph '/private/var/tmp/_bazel_elliotvu/e512a8189164dd2ae5386a6bdb0cc34e/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/Demo.output_file_map.priors' remark: Incremental compilation: Enabling incremental cross-module building remark: Incremental compilation: May skip current input: {compile: A.swift.o <= A.swift} remark: Incremental compilation: May skip current input: {compile: B.swift.o <= B.swift} remark: Incremental compilation: May skip current input: {compile: C.swift.o <= C.swift} remark: Incremental compilation: Missing an output; will queue {compile: A.swift.o <= A.swift} remark: Incremental compilation: Missing an output; will queue {compile: B.swift.o <= B.swift} remark: Incremental compilation: Missing an output; will queue {compile: C.swift.o <= C.swift} remark: Incremental compilation: Queuing (initial): {compile: A.swift.o <= A.swift} remark: Incremental compilation: Queuing (initial): {compile: B.swift.o <= B.swift} remark: Incremental compilation: Queuing (initial): {compile: C.swift.o <= C.swift} remark: Found 3 batchable jobs remark: Forming into 1 batch remark: Adding {compile: A.swift} to batch 0 remark: Adding {compile: B.swift} to batch 0 remark: Adding {compile: C.swift} to batch 0 remark: Forming batch job from 3 constituents: A.swift, B.swift, C.swift remark: Starting Emitting module for Demo remark: Finished Emitting module for Demo remark: Starting Compiling A.swift, B.swift, C.swift remark: Finished Compiling A.swift, B.swift, C.swift remark: Incremental compilation: Reading dependencies from A.swift remark: Incremental compilation: Reading dependencies from B.swift remark: Incremental compilation: Reading dependencies from C.swift remark: Incremental compilation: Scheduling all post-compile jobs because something was compiled ``` These are logs after fix. As you can see, only B.swift will be compile ``` INFO: Analyzed target //:Demo (0 packages loaded, 0 targets configured). INFO: From Compiling Swift module //:Demo: remark: Incremental compilation: Read dependency graph '/private/var/tmp/_bazel_elliotvu/e512a8189164dd2ae5386a6bdb0cc34e/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/Demo.output_file_map.priors' remark: Incremental compilation: Enabling incremental cross-module building remark: Incremental compilation: May skip current input: {compile: A.swift.o <= A.swift} remark: Incremental compilation: Scheduling changed input {compile: B.swift.o <= B.swift} remark: Incremental compilation: May skip current input: {compile: C.swift.o <= C.swift} remark: Incremental compilation: Queuing (initial): {compile: B.swift.o <= B.swift} remark: Incremental compilation: not scheduling dependents of B.swift; unknown changes remark: Incremental compilation: Skipping input: {compile: A.swift.o <= A.swift} remark: Incremental compilation: Skipping input: {compile: C.swift.o <= C.swift} remark: Found 1 batchable job remark: Forming into 1 batch remark: Adding {compile: B.swift} to batch 0 remark: Forming batch job from 1 constituents: B.swift remark: Starting Emitting module for Demo remark: Finished Emitting module for Demo remark: Starting Compiling B.swift remark: Finished Compiling B.swift remark: Incremental compilation: Reading dependencies from B.swift remark: Incremental compilation: Fingerprint changed for existing interface of top-level name 'B' in B.swift remark: Incremental compilation: Fingerprint changed for existing implementation of top-level name 'B' in B.swift remark: Incremental compilation: Fingerprint changed for existing interface of type '4Demo1BV' in B.swift remark: Incremental compilation: Fingerprint changed for existing implementation of type '4Demo1BV' in B.swift remark: Incremental compilation: Fingerprint changed for existing interface of potential members of '4Demo1BV' in B.swift remark: Incremental compilation: Fingerprint changed for existing implementation of potential members of '4Demo1BV' in B.swift remark: Incremental compilation: Scheduling all post-compile jobs because something was compiled remark: Skipped Compiling A.swift remark: Skipped Compiling C.swift ``` The solution here should be to remap swiftconstvalues path to `_swift_incremental` directory Related issue: #1291
|
@adincebic This issue would be fixed on #1471 |
Oh wow! Thank you for your contribution. |
adincebic
pushed a commit
to adincebic/rules_swift
that referenced
this issue
Jan 17, 2025
Since we extracting `swiftconstvalues` from bazelbuild#1170 But we forgot to update the path for swiftconstvalues in the incremental output file map. It leads to the incremental feature not working. For example, demo: https://github.com/vikage/DemoBazelSwiftIncremental When I change file B.swift (No other file uses struct B) but log shows build A.swift and C.swift as well I used `-driver-show-job-lifecycle` and `-driver-show-incremental` flags to show debug log ``` WARNING: Build option --swiftcopt has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed). INFO: Analyzed target //:Demo (99 packages loaded, 1088 targets configured). INFO: From Compiling Swift module //:Demo: remark: Incremental compilation: Read dependency graph '/private/var/tmp/_bazel_elliotvu/e512a8189164dd2ae5386a6bdb0cc34e/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/Demo.output_file_map.priors' remark: Incremental compilation: Enabling incremental cross-module building remark: Incremental compilation: May skip current input: {compile: A.swift.o <= A.swift} remark: Incremental compilation: May skip current input: {compile: B.swift.o <= B.swift} remark: Incremental compilation: May skip current input: {compile: C.swift.o <= C.swift} remark: Incremental compilation: Missing an output; will queue {compile: A.swift.o <= A.swift} remark: Incremental compilation: Missing an output; will queue {compile: B.swift.o <= B.swift} remark: Incremental compilation: Missing an output; will queue {compile: C.swift.o <= C.swift} remark: Incremental compilation: Queuing (initial): {compile: A.swift.o <= A.swift} remark: Incremental compilation: Queuing (initial): {compile: B.swift.o <= B.swift} remark: Incremental compilation: Queuing (initial): {compile: C.swift.o <= C.swift} remark: Found 3 batchable jobs remark: Forming into 1 batch remark: Adding {compile: A.swift} to batch 0 remark: Adding {compile: B.swift} to batch 0 remark: Adding {compile: C.swift} to batch 0 remark: Forming batch job from 3 constituents: A.swift, B.swift, C.swift remark: Starting Emitting module for Demo remark: Finished Emitting module for Demo remark: Starting Compiling A.swift, B.swift, C.swift remark: Finished Compiling A.swift, B.swift, C.swift remark: Incremental compilation: Reading dependencies from A.swift remark: Incremental compilation: Reading dependencies from B.swift remark: Incremental compilation: Reading dependencies from C.swift remark: Incremental compilation: Scheduling all post-compile jobs because something was compiled ``` These are logs after fix. As you can see, only B.swift will be compile ``` INFO: Analyzed target //:Demo (0 packages loaded, 0 targets configured). INFO: From Compiling Swift module //:Demo: remark: Incremental compilation: Read dependency graph '/private/var/tmp/_bazel_elliotvu/e512a8189164dd2ae5386a6bdb0cc34e/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/Demo.output_file_map.priors' remark: Incremental compilation: Enabling incremental cross-module building remark: Incremental compilation: May skip current input: {compile: A.swift.o <= A.swift} remark: Incremental compilation: Scheduling changed input {compile: B.swift.o <= B.swift} remark: Incremental compilation: May skip current input: {compile: C.swift.o <= C.swift} remark: Incremental compilation: Queuing (initial): {compile: B.swift.o <= B.swift} remark: Incremental compilation: not scheduling dependents of B.swift; unknown changes remark: Incremental compilation: Skipping input: {compile: A.swift.o <= A.swift} remark: Incremental compilation: Skipping input: {compile: C.swift.o <= C.swift} remark: Found 1 batchable job remark: Forming into 1 batch remark: Adding {compile: B.swift} to batch 0 remark: Forming batch job from 1 constituents: B.swift remark: Starting Emitting module for Demo remark: Finished Emitting module for Demo remark: Starting Compiling B.swift remark: Finished Compiling B.swift remark: Incremental compilation: Reading dependencies from B.swift remark: Incremental compilation: Fingerprint changed for existing interface of top-level name 'B' in B.swift remark: Incremental compilation: Fingerprint changed for existing implementation of top-level name 'B' in B.swift remark: Incremental compilation: Fingerprint changed for existing interface of type '4Demo1BV' in B.swift remark: Incremental compilation: Fingerprint changed for existing implementation of type '4Demo1BV' in B.swift remark: Incremental compilation: Fingerprint changed for existing interface of potential members of '4Demo1BV' in B.swift remark: Incremental compilation: Fingerprint changed for existing implementation of potential members of '4Demo1BV' in B.swift remark: Incremental compilation: Scheduling all post-compile jobs because something was compiled remark: Skipped Compiling A.swift remark: Skipped Compiling C.swift ``` The solution here should be to remap swiftconstvalues path to `_swift_incremental` directory Related issue: bazelbuild#1291
brentleyjones
pushed a commit
that referenced
this issue
Jan 17, 2025
Since we extracting `swiftconstvalues` from #1170 But we forgot to update the path for swiftconstvalues in the incremental output file map. It leads to the incremental feature not working. For example, demo: https://github.com/vikage/DemoBazelSwiftIncremental When I change file B.swift (No other file uses struct B) but log shows build A.swift and C.swift as well I used `-driver-show-job-lifecycle` and `-driver-show-incremental` flags to show debug log ``` WARNING: Build option --swiftcopt has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed). INFO: Analyzed target //:Demo (99 packages loaded, 1088 targets configured). INFO: From Compiling Swift module //:Demo: remark: Incremental compilation: Read dependency graph '/private/var/tmp/_bazel_elliotvu/e512a8189164dd2ae5386a6bdb0cc34e/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/Demo.output_file_map.priors' remark: Incremental compilation: Enabling incremental cross-module building remark: Incremental compilation: May skip current input: {compile: A.swift.o <= A.swift} remark: Incremental compilation: May skip current input: {compile: B.swift.o <= B.swift} remark: Incremental compilation: May skip current input: {compile: C.swift.o <= C.swift} remark: Incremental compilation: Missing an output; will queue {compile: A.swift.o <= A.swift} remark: Incremental compilation: Missing an output; will queue {compile: B.swift.o <= B.swift} remark: Incremental compilation: Missing an output; will queue {compile: C.swift.o <= C.swift} remark: Incremental compilation: Queuing (initial): {compile: A.swift.o <= A.swift} remark: Incremental compilation: Queuing (initial): {compile: B.swift.o <= B.swift} remark: Incremental compilation: Queuing (initial): {compile: C.swift.o <= C.swift} remark: Found 3 batchable jobs remark: Forming into 1 batch remark: Adding {compile: A.swift} to batch 0 remark: Adding {compile: B.swift} to batch 0 remark: Adding {compile: C.swift} to batch 0 remark: Forming batch job from 3 constituents: A.swift, B.swift, C.swift remark: Starting Emitting module for Demo remark: Finished Emitting module for Demo remark: Starting Compiling A.swift, B.swift, C.swift remark: Finished Compiling A.swift, B.swift, C.swift remark: Incremental compilation: Reading dependencies from A.swift remark: Incremental compilation: Reading dependencies from B.swift remark: Incremental compilation: Reading dependencies from C.swift remark: Incremental compilation: Scheduling all post-compile jobs because something was compiled ``` These are logs after fix. As you can see, only B.swift will be compile ``` INFO: Analyzed target //:Demo (0 packages loaded, 0 targets configured). INFO: From Compiling Swift module //:Demo: remark: Incremental compilation: Read dependency graph '/private/var/tmp/_bazel_elliotvu/e512a8189164dd2ae5386a6bdb0cc34e/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/Demo.output_file_map.priors' remark: Incremental compilation: Enabling incremental cross-module building remark: Incremental compilation: May skip current input: {compile: A.swift.o <= A.swift} remark: Incremental compilation: Scheduling changed input {compile: B.swift.o <= B.swift} remark: Incremental compilation: May skip current input: {compile: C.swift.o <= C.swift} remark: Incremental compilation: Queuing (initial): {compile: B.swift.o <= B.swift} remark: Incremental compilation: not scheduling dependents of B.swift; unknown changes remark: Incremental compilation: Skipping input: {compile: A.swift.o <= A.swift} remark: Incremental compilation: Skipping input: {compile: C.swift.o <= C.swift} remark: Found 1 batchable job remark: Forming into 1 batch remark: Adding {compile: B.swift} to batch 0 remark: Forming batch job from 1 constituents: B.swift remark: Starting Emitting module for Demo remark: Finished Emitting module for Demo remark: Starting Compiling B.swift remark: Finished Compiling B.swift remark: Incremental compilation: Reading dependencies from B.swift remark: Incremental compilation: Fingerprint changed for existing interface of top-level name 'B' in B.swift remark: Incremental compilation: Fingerprint changed for existing implementation of top-level name 'B' in B.swift remark: Incremental compilation: Fingerprint changed for existing interface of type '4Demo1BV' in B.swift remark: Incremental compilation: Fingerprint changed for existing implementation of type '4Demo1BV' in B.swift remark: Incremental compilation: Fingerprint changed for existing interface of potential members of '4Demo1BV' in B.swift remark: Incremental compilation: Fingerprint changed for existing implementation of potential members of '4Demo1BV' in B.swift remark: Incremental compilation: Scheduling all post-compile jobs because something was compiled remark: Skipped Compiling A.swift remark: Skipped Compiling C.swift ``` The solution here should be to remap swiftconstvalues path to `_swift_incremental` directory Related issue: #1291 Co-authored-by: Thanh Vu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
After spending quite some time on this, I determined that Swift incremental compilation via Bazel and rules_swift does not work. I am aware that there has been some progress made on this in the past and I believe that incremental compilation does work for smaller modules but not for ones that contain a couple of thousands source files.
I created a small reproducer project and provided more details on what I tried in the
READMEfile.Find the repro here: https://github.com/adincebic/bazel-ios-swiftui-template/tree/ac/swift-incremental-compilation-not-working
The text was updated successfully, but these errors were encountered: