Improve UX of the new command

[dgzlopes]

Feature Description

The k6 new command was introduced to:

• Simplify the process of creating new k6 scripts.
• Teach users about k6's capabilities (incl. Cloud integration and browser support).

After seeing it used in many places, I have detected some problems:

• It doesn't work well with the Cloud flows.
  • On our Cloud onboarding, in some cases, we tell the user to generate the script, open it, and change things manually.
    • 
  • In others, we ask the user to copy-paste a script directly instead of using this command (e.g., new Test Run page).
    • 
• The current example script tries to do too much without being able to be minimal enough or complete enough.
  • It is a protocol test with a massive commented config block that always includes Cloud, Browser, etc.
  • Even if you have Browser in the options block... there is no browser code in the test logic.

Suggested Solution (optional)

• Having two 'new' commands
  • k6 new
  • k6 cloud new
    • The Cloud block will appear in the config block of the generated test when you use this one.
    • Also, optionally, users can pass --project-id to override the dummy one.
      • This would solve the "manual changes" required step of Cloud onboarding. We can tell users to copy-paste:
        • k6 cloud new --project-id 123145
• Having templates instead of one massive example (note: I was inspired by how Svelte handles templates).
  • Usage: k6 cloud new --project-id 123145 --template=browser

Minimal (default) - Small and concise script. It is ideal for folks who need a starting point for a new project

import http from 'k6/http';
import { sleep, check } from 'k6';

export const options = {
  vus: 10,
  duration: '30s',
};

export default function() {
  let res = http.get('https://quickpizza.grafana.com');
  check(res, { "status is 200": (res) => res.status === 200 });
  sleep(1);
}

Protocol - More real script. It is ideal for folks learning, as a reference, or for demos.

import http from "k6/http";
import { check, sleep } from "k6";

const BASE_URL = __ENV.BASE_URL || 'https://quickpizza.grafana.com';

export const options = {
  vus: 10,
  stages: [
    { duration: "10s", target: 5 },
    { duration: "20s", target: 10 },
    { duration: "1s", target: 0 },
  ], thresholds: {
    http_req_failed: ["rate<0.01"],
    http_req_duration: ["p(95)<500", "p(99)<1000"],
  },
};

export function setup() {
  let res = http.get(BASE_URL);
  if (res.status !== 200) {
    throw new Error(
      `Got unexpected status code ${res.status} when trying to setup. Exiting.`
    );
  }
}

export default function() {
  let restrictions = {
    maxCaloriesPerSlice: 500,
    mustBeVegetarian: false,
    excludedIngredients: ["pepperoni"],
    excludedTools: ["knife"],
    maxNumberOfToppings: 6,
    minNumberOfToppings: 2
  };

  let res = http.post(BASE_URL + "/api/pizza", JSON.stringify(restrictions), {
    headers: {
      'Content-Type': 'application/json',
      'X-User-ID': 23423,
    },
  });

  check(res, { "status is 200": (res) => res.status === 200 });
  console.log(res.json().pizza.name + " (" + res.json().pizza.ingredients.length + " ingredients)");
  sleep(1);
}

Browser - Same as Protocol but with Browser.

import { browser } from "k6/browser";
import http from "k6/http";
import { sleep, check } from 'k6';

const BASE_URL = __ENV.BASE_URL || "https://quickpizza.grafana.com";

export const options = {
  scenarios: {
    ui: {
      executor: "shared-iterations",
      options: {
        browser: {
          type: "chromium",
        },
      },
    },
  }, 
};

export function setup() {
  let res = http.get(BASE_URL);
  if (res.status !== 200) {
    throw new Error(
      `Got unexpected status code ${res.status} when trying to setup. Exiting.`
    );
  }
}

export default async function() {
  let checkData;
  const page = await browser.newPage();
  try {
    await page.goto(BASE_URL);
    checkData = await page.locator("h1").textContent();
    check(page, {
      header: checkData === "Looking to break out of your pizza routine?",
    });
    await page.locator('//button[. = "Pizza, Please!"]').click();
    await page.waitForTimeout(500);
    await page.screenshot({ path: "screenshot.png" });
    checkData = await page.locator("div#recommendations").textContent();
    check(page, {
      recommendation: checkData !== "",
    });
  } finally {
    await page.close();
  }
  sleep(1);
}

(Future: We could let people leverage external templates)

Already existing or connected issues / PRs (optional)

#4153

[andrewslotin]

I really like the templates idea, but the fact that k6 cloud new does not create a new test in GCk6, but initializes a new test file locally instead might be confusing. The intended purpose of k6 cloud subcommand is to access and manage resources in Grafana Cloud k6. Would it make sense to change the existing k6 new command to accept optional --project-id flag instead and populate the options field in case it has been provided?

[joanlopez]

Generally speaking, I also think that a simple templates mechanism would be needed here. Indeed, I think we even discussed that specifically at time of releasing k6 new, as likely the next step to make that feature more powerful. And, in that case, I would see the "cloud" variant as yet another flavour, as "protocol", "browser", and others could be, either complementarily or mutually exclusively.

That said,

the fact that k6 cloud new does not create a new test in GCk6, but initializes a new test file locally instead might be confusing

I agree with @andrewslotin that k6 cloud new could be confusing, especially now that we have certain plans to start adding more subcommands to k6 cloud, and all of them involve interaction with cloud.

[dgzlopes]

I agree with @andrewslotin that k6 cloud new could be confusing, especially now that we have certain plans to start adding more subcommands to k6 cloud, and all of them involve interaction with cloud.

Yup, makes sense! Thoughts on this @oleiade?

And, in that case, I would see the "cloud" variant as yet another flavour, as "protocol", "browser", and others could be, either complementarily or mutually exclusively.

I'm not really sure about this. It seems a bit redundant to have a Cloud template when any of the existing templates can be converted (rather easily) to a Cloud test.

[joanlopez]

I'm not really sure about this. It seems a bit redundant to have a Cloud template when any of the existing templates can be converted (rather easily) to a Cloud test.

Sorry, I wasn't clear enough. I didn't want to mean exactly like browser or protocol, but some option you can set on the templating system. Like with X or Y, where one of these would be with cloud.

I explained it like this because I wanted to point to the fact that I see it as another option of the same templating system, and not a "completely separated" thing (like a different (sub)command).

In essence, how I envision it , I think it's similar to your draft PR, except for the sub-command and its naming.

[oleiade]

Hey folks 👋🏻

I agree that it might be better to restrict the proposal to modifying k6 new instead, and leave k6 cloud out of the picture. I think we had completely overlooked the subcommand (k6 cloud new) potential confusion/collision that @andrewslotin raised in our discussions @dgzlopes. Although I would expect cloud creation to be scoped under k6 cloud test create/list/delete, the argument about locallity k6 does stuff locally, k6 cloud does things in the cloud convinced me 👍🏻

Regarding templates, I gave that feedback to @dgzlopes prior to the issue that from a UX perspective I believe we should limit the amount of templates supported natively/out of the box by k6 to use-cases focused ones, like in this proposal: "as a user I want to write a browser test/functional test/protocol test", or "As a user, I want to run a test in the cloud, or in my local machine, or in kubernetes (insert anything relevant here"; essentially what and where, but limited to the minimum. We should also aim to support custom/user-defined templates, but that shouldn't block to move forward. Personally, I'd be really satisfied with the current state of the proposal already.

Side-thought: maybe a cheap first step towards templates would be to offer environment variables interpolation in the k6 new command. As-in: if you use environment variables in your script __ENV.SOME_VARIABLE, we'll interpolate it during the script generation?

Let's do it 🚀 🙇🏻

[dgzlopes]

Gotcha! I have updated the draft PR to only have a single command 👍 (& fixed a bunch of linting issues)

If the user adds --project-id the Cloud block will be appended with the right configuration.

Side-thought: maybe a cheap first step towards templates would be to offer environment variables interpolation in the k6 new command. As-in: if you use environment variables in your script __ENV.SOME_VARIABLE, we'll interpolate it during the script generation?

I think this would be very useful if we let users define their own templates b/c they can build conditional generation logic without us having to add support for "custom user-defined" flags/arguments.