Start investigating the CI/CD

[waliid]

As a Pillarbox developer, I would like to understand what the best option is for our CI/CD.

Acceptance criteria

• Find at least one solution that is suitable for our project.

Tasks

• Understand what the AIS Team has achieved so far.
• Contact at least one member of the PlayNext team to see if they are interested in investigating Tart with us.
• Collect information about Tart and its environment (how it works).
• Configure a small project in a local development environment.
• Try to determine the costs associated with Tart.
• Identify the resources required with Tart.
• Determine how long Tart takes to:
  • Create a VM, start the VM and run a build.
  • Start a VM and run a build.
• ...

[waliid]

Firstly, I set up a CI on a single machine with a single GitHub runner, using a Tart virtual machine. The problem was that there was no parallelization, which meant that workflows took a long time to complete.

Then, I made the process more complex by adding a second runner on the same machine. The problem was that I had to manage concurrent writes for certain jobs, such as the iOS and tvOS tests, which write to the same folder (Derived Data).

Finally, I added two GitHub runners on another machine, so I now had two runners on the first machine and two on the second. The issue, was that my first job started a virtual machine on the first machine, but the second job could potentially use the second machine, which had no knowledge of the virtual machine, as a result, my workflow was failing.

I then tried to group my machines together using Orchard. The next issue, was that I couldn't use scp, because with Orchard only an SSH connection is possible. I got round this limitation by compressing the files and folders I wanted to send, sending them via SSH and then decompressing them on the fly. However, this solution proved to be very slow.

So I opted for another approach using labels. The first and second machines each have a specific label for the different files.

The problem with this solution is that once the first job starts on one machine, the entire workflow remains confined to that machine until the end, which prevents the resources of the other machines from being used.

[waliid]

Running tests simultaneously causes a failure because both jobs (test-ios and test-tvos) attempt to write to the same location.

To bypass this issue, one solution is to update our Fastfile to separate the derived data into two distinct directories as show below:

def run_package_tests(platform_id, scheme_name)
  output_path = "fastlane/test_output_#{platform_id}"
  derived_data_path = "#{Dir.home}/Library/Developer/Xcode/DerivedData/#{platform_id}"
  FileUtils.mkdir_p(output_path)
  FileUtils.mkdir_p(derived_data_path)
  run_tests(
    scheme: scheme_name,
    device: DEVICES[platform_id],
    package_path: '.',
    result_bundle: true,
    output_directory: output_path,
    derived_data_path: derived_data_path,
    number_of_retries: 3,
    fail_build: false,
    xcargs: '-testLanguage en -testRegion en_US'
  )
  trainer(
    path: output_path,
    output_remove_retry_attempts: true,
    fail_build: false
  )
end