From f55794c592debae6eae42929e13b9d7d2ca033f8 Mon Sep 17 00:00:00 2001 From: Marcus Date: Thu, 19 Dec 2024 21:28:50 +0000 Subject: [PATCH] Chromium blog post --- ...d_Chromium_with_Reclient_and_NativeLink.md | 326 ++++++++++++++++++ 1 file changed, 326 insertions(+) create mode 100644 web/platform/src/content/posts/Build_Chromium_with_Reclient_and_NativeLink.md diff --git a/web/platform/src/content/posts/Build_Chromium_with_Reclient_and_NativeLink.md b/web/platform/src/content/posts/Build_Chromium_with_Reclient_and_NativeLink.md new file mode 100644 index 000000000..c9b2a38bb --- /dev/null +++ b/web/platform/src/content/posts/Build_Chromium_with_Reclient_and_NativeLink.md @@ -0,0 +1,326 @@ +--- +title: "Build Chromium with Reclient and NativeLink" +tags: ["news", "blog-posts"] +image: https://www.gstatic.com/webp/gallery/4.sm.webp +slug: chromium +pubDate: 2024-12-04 +readTime: 95 minutes +--- + +# Build Chromium Builds and Tests on Ubuntu with NativeLink’s RBE and CAS + +Building and testing Chromium can be challenging due to the long, compute-heavy build times and the potential flakiness of automated tests in its vast and rapidly evolving codebase. Ensuring consistent, reliable test results across various environments demands robust tools that can handle frequent updates and complex dependencies. + +In this guide, we’ll demonstrate how to scale your Chromium build and test capabilities using **NativeLink**—with its Rust-based Bazel Remote Execution (RBE) and Content Addressable Storage (CAS) implementations—on Ubuntu. Follow along to streamline your build process and achieve reliable, reproducible results every time. + +--- + +## Table of Contents + +1. [Prerequisites](#prerequisites) +2. [Step 1: Launch an Ubuntu Instance](#step-1-launch-an-ubuntu-instance) +3. [Step 2: SSH into the Ubuntu Instance](#step-2-ssh-into-the-ubuntu-instance) +4. [Step 3: Set Up the Ubuntu Environment](#step-3-set-up-the-ubuntu-environment) +5. [Step 4: Scale Chromium Builds and Tests](#step-4-scale-chromium-builds-and-tests) +6. [Debugging Tips](#debugging-tips) + +--- + +## Prerequisites + +**What you need for this tutorial:** +- **AWS Console** (to launch and manage EC2 instances) +- **NativeLink** +- **Nix** (tested on version `2.22.0`) +- **Docker** (Ubuntu setup) + +**Important:** +If you have a Linux laptop and don't want to launch an EC2 instance, you can start from [Step 3](#step-3-set-up-the-ubuntu-environment). +Ensure you have the latest versions of NativeLink, Nix, and Docker. If you get stuck, please reach out in the [NativeLink Slack Community](https://join.slack.com/t/nativelink/shared_invite/zt-281qk1ho0-krT7HfTUIYfQMdwflRuq7A). + +--- + +## Step 1: Launch an Ubuntu Instance + +We’ll use Amazon EC2 to create a virtual machine running on AWS cloud. + +1. In the AWS console, navigate to **EC2 Dashboard** and click **Launch instance**. +2. **Name** the instance: `chromium-example`. +3. Under the **Quick Start** tab, choose **Ubuntu**. +4. Under **Architecture**, select **64-bit**. +5. For **Instance type**, choose **c6a.4xlarge**. +6. Under **Network settings**, select **Create security group** and allow **SSH traffic from Anywhere**. +7. Click **Create new key pair**, and set: + - Key pair name: `chromium-exp-ubuntu` + - Key pair type: `RSA` + - Private key file format: `.pem` + + After creation, the key pair `.pem` file will be downloaded. Keep track of its location. + +8. Under **Configure storage**, enter `500` for the storage size. +9. Click **Launch instance**. + +After launching, wait a few minutes for the instance state to be `Running`. Refresh the page as needed. + +--- + +## Step 2: SSH into the Ubuntu Instance + +1. Move the downloaded `.pem` key into `~/.ssh`: + ```shell + mv PATH/TO/chromium-exp-ubuntu.pem ~/.ssh + ls ~/.ssh + ``` + +2. If a Config file doesn't exist in `~/.ssh`, create one: + +```shell +touch ~/.ssh/config +``` + +3. Open the Config file and add the following (replace PublicIPv4DNS with your instance’s Public IPv4 DNS): + +```shell +Host chromium-exp-ubuntu + HostName PublicIPv4DNS + IdentityFile ~/.ssh/chromium-exp-ubuntu.pem + User ubuntu +``` + +![][image1] + +[image1]: + +4. Set permissions on the private key: + +```shell +chmod 600 ~/.ssh/chromium-exp-ubuntu.pem +``` + +5. Access the Ubuntu instance by SSH using the following command: + +```shell +ssh chromium-exp-ubuntu + +Write \`yes\` when prompted with: +Are you sure you want to continue connecting (yes/no/\[fingerprint\])? +``` + +BAM! We’re in! You should see something like this in your terminal: + +```shell +ubuntu@some-ip-address:~$ +``` + +## Step 3: Set Up the Ubuntu environment + +1. We’ll be using the multi-user installation of Nix. It offers several benefits, including shared resources which reduce redundancy and save disk space by allowing all users to access a common Nix store for packages and dependencies. It ensures a consistent environment where everyone accesses the same tool and software versions. Install Nix is your home directory, `/home/ubuntu`: + +```shell +sh <(curl -L https://nixos.org/nix/install) --daemon +``` + +Type `y` when prompted: +`Would you like to see a more detailed list of what I will do? [y/n]` + +Type `y` when prompted: +Ready to continue? [y/n] + +Type `y` when prompted: +Can I use `sudo`? [y/n] + +Type `y` when prompted: +Ready to continue? [y/n] + +Press `enter/return` when prompted: + +Press enter/return to acknowledge. + +2. Exit the Ubuntu shell session and then re-enter: + +Press Control+D to exit the Ubuntu shell. Afterward, SSH back into it with the following command: + +```shell +ssh chromium-exp-ubuntu +``` + +3. Create a nix profile. Type in the Ubuntu shell: + +```shell +mkdir -p ~/.config/nix +vim ~/.config/nix/nix.conf +``` + +Input this, and save the file: +```shell +experimental-features = nix-command flakes +``` + +4. Clone the NativeLink repository and navigate to the project root: + +```shell +git clone https://github.com/TraceMachina/nativelink.git +cd nativelink +``` + +5. Run `nix develop` command. It will take a few minutes for this to run, so don’t be alarmed: + +```shell +nix develop +``` + +Note, at the end of this process you should see output like: + +```shell +pre-commit-hooks.nix: updating /home/ubuntu/nativelink repo +pre-commit installed at .git/hooks/pre-commit +``` + +The `nix develop` command belongs to the Nix package manager and creates a deterministic, reproducible development environment as defined in a `flake.nix` file, isolating all required dependencies to ensure consistency across any system. This setup doesn't impact global system settings. + +6. Install Docker. You’ll need to follow [these commands](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) to set up Docker’s apt repository. The commands you’ll need are also listed below for handiness: + +```shell +# Add official Docker GPG key: + +sudo apt-get update \ +sudo apt-get install ca-certificates curl \ +sudo install -m 0755 -d /etc/apt/keyrings \ +sudo curl -fsSL \ https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc +``` + +After it completes. + +```shell +sudo chmod a+r /etc/apt/keyrings/docker.asc + +# Add the repository to Apt sources: +echo \ + "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ + $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ + sudo tee /etc/apt/sources.list.d/docker.list > /dev/null + + +sudo apt-get update + + +sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin +``` + +When prompted: Do you want to continue, type `Y`: + +Confirm everything is correctly installed by running the hello-world containerL + +```shell +sudo docker run hello-world +``` + +7. The docker group grants root-level privileges to the user. To avoid using `sudo` for every future docker command, type: + +```shell +sudo groupadd docker +sudo usermod -aG docker $USER +``` + +8. Exit the Ubuntu shell and then re-enter: +Press Control+D to exit the Ubuntu shell. Afterward, SSH back into it with the following command: + +```shell +ssh chromium-exp-ubuntu +``` + +9. Navigate to the `nativelink` root directory and create a deterministic, reproducible development environment : + +```shell +cd nativelink +nix develop +``` + +10. Now, you can run docker commands without `sudo`: + +```shell +docker ps +``` + +This wraps it up for the environment setup! Now let’s build the Chromium tests! + +## Step 4: Scale Chromium Builds and Tests + +1. Be sure to be in the NativeLink repository and have setup a deterministic, reproducible development environment: + +```shell +cd nativelink +nix develop +``` + +2. Navigate to the Chromium example: + +```shell +cd deployment-examples/chromium +``` + +3. Spin up the local development cluster: + +```shell +native up +``` +4. Run through the first script to build the remote execution containers and make them available to the cluster. It may take a few minutes: + +```shell +./01_operations.sh +``` + +You should see this output if successful: + +```shell +pipelinerun.tekton.dev/rebuild-nativelink-run-8svm4 condition met +``` + +5. Run through the second script to deploy NativeLink. It may take a few minutes: + +```shell +./02_application.sh +``` + +You should see this output if successful: + +```shell +deployment "nativelink-cas" successfully rolled out +deployment "nativelink-scheduler" successfully rolled out +deployment "nativelink-worker-chromium" successfully rolled out +``` +6. Run through the third script to build the Chromium tests using NativeLink RBE and CAS. It may take 15--30+ minutes: + +```shell +./03_build_chrome_tests.sh +``` + +When prompted Do you want to continue? (yes/no), type, `yes`. + +If the build is successful, you should see [199xx/199xx] builds complete. + +### Debugging Tips: +If you see the build terminate and incomplete, like [10042/19943], that means the build didn’t succeed due to a couple of possibilities: +CPU usage is high- you need to provision more memory + NativeLink CAS or Scheduler is in a poor state, or disk is full. + +You can check the CPU usage on the AWS Console and/or check the log: +```shell +cat /home/ubuntu/chromium/src/out/Default/.reproxy_tmp/logs/ +``` + +To fix this, you can create a new Ubuntu instance on AWS with more memory. Alternatively, you can terminate the cluster and rerun again in the same Ubuntu instance via `nativelink/deployment-examples/chromium` directory: + + +```shell +# teardown the local cluster: +native down + +# ensure there is no cluster running +docker ps + +# spin up the cluster +native up +``` + +More information about monitoring the logs can be found [here](https://www.nativelink.com/docs/nativelink-cloud/reclient/) in our less opinionated Chromium and Reclient Documentation.