diff --git a/docs/smartui-cli-upload.md b/docs/smartui-cli-upload.md index cd1f11dc..7a499f40 100644 --- a/docs/smartui-cli-upload.md +++ b/docs/smartui-cli-upload.md @@ -128,10 +128,10 @@ Please read the following table for more information about the options available | Config Key Shortcut | Configuration Key     | Description| Usage     | | ---------------------| ---------------------------------------------------------------------------|--------------| -------------------- | | -R | --ignoreResolutions | Ignores resolutions to compare only based on screenshot names | `npx smartui upload -R` | -| -F | --files extensions | Comma-separated list of allowed file extensions| `npx smartui upload -F jpg,png`| +| -F | --files `extensions` | Comma-separated list of allowed file extensions| `npx smartui upload -F jpg,png`| | -E | --removeExtensions | Strips file extensions from snapshot names| `npx smartui upload -E` | -| -i | --ignoreDir patterns | Comma-separated list of directories to ignore | `npx smartui upload -i` | - +| -i | --ignoreDir `patterns` | Comma-separated list of directories to ignore | `npx smartui upload -i dir1/dir2,dir3` | +| | --fetch-results | Live fetched comparison results in a json file | `npx smartui upload --fetch-results`| :::note You may use the `smartui upload --help` command in case you are facing issues during the execution of SmartUI Upload options in the CLI. @@ -164,6 +164,25 @@ The following are supported `CLI (Command Line Interface)` options for Visual Re You can see the Smart UI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. + +### Fetch results + +You can fetch build results by adding the `--fetch-results` flag to your test execution command. Here are different ways to use this feature: + +#### Default Usage +If no filename is specified, results will be stored in `results.json`: + +```bash +npx smartui upload --fetch-results +``` + +#### Custom Filename +Specify a custom filename for your results: + +```bash +npx smartui upload --fetch-results custom-results.json +``` + cmd For additional information about SmartUI APIs please explore the documentation [here](https://www.lambdatest.com/support/api-doc/) diff --git a/docs/smartui-cli.md b/docs/smartui-cli.md index a174354e..0e3ab118 100644 --- a/docs/smartui-cli.md +++ b/docs/smartui-cli.md @@ -78,7 +78,7 @@ The first step is to create a project with the application in which we will comb Install required NPM modules for `LambdaTest Smart UI CLI` in your **Frontend** project. ```bash -npm install -g @lambdatest/smartui-cli +npm install @lambdatest/smartui-cli ``` ### **Step 2:** Create URL file @@ -137,7 +137,7 @@ set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" You can now configure your project settings on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: ```bash -smartui config:create .smartui.json +npx smartui config:create .smartui.json ``` Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: @@ -217,7 +217,26 @@ Please read the following table for more information about the configuration fil You can now execute tests for `Visual Regression Testing` using the following options:. ```bash -smartui capture urls.json --config .smartui.json +npx smartui capture urls.json --config .smartui.json +``` + + +### Fetch results + +You can fetch build results by adding the `--fetch-results` flag to your test execution command. Here are different ways to use this feature: + +#### Default Usage +If no filename is specified, results will be stored in `results.json`: + +```bash +npx smartui capture urls.json --config .smartui.json --fetch-results +``` + +#### Custom Filename +Specify a custom filename for your results: + +```bash +npx smartui capture urls.json --config .smartui.json --fetch-results custom-results.json ``` ### Setup with Continuous Integration (CI) diff --git a/docs/smartui-sdk-fetch-results.md b/docs/smartui-sdk-fetch-results.md new file mode 100644 index 00000000..8a360b46 --- /dev/null +++ b/docs/smartui-sdk-fetch-results.md @@ -0,0 +1,252 @@ +--- +id: smartui-sdk-fetch-results +title: Fetching Results through SmartUI SDK +sidebar_label: Fetch Results +description: In this documentation, learn how to fetch live results for SmartUI tests +keywords: + - Visual Regression + - Visual Regression Testing Guide + - Visual Regression Test Automation + - Visual Regression Automation Testing + - Running Visual Regression Tests + - Visual Regression Testing Online + - Run Visual Regression + - Visual Regression Run Specific Test + - Visual Regression Testing Environment + - How to Run Visual Regression Tests + +url: https://www.lambdatest.com/support/docs/smartui-cli/ +slug: smartui-sdk-fetch-results/ +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; +import NewTag from '../src/component/newTag'; + +--- + + + +SmartUI CLI allows you to fetch detailed build results after executing your visual tests. This feature enables you to access comprehensive information about your build and screenshots in a JSON file, making it easier to integrate with your CI/CD pipelines and automation workflows. + + +## Prerequisites + +- Basic understanding of Command Line Interface +- Login to [LambdaTest SmartUI](https://smartui.lambdatest.com/) with your credentials. +- Ensure you are using `@lambdatest/smartui-cli` version 4.0.8 or higher. +- A properly configured SmartUI CLI project + +## Steps to Use + +### *Step 1:* Install SmartUI CLI + +If you haven't already installed SmartUI CLI, install it using npm: + +```bash +npm i @lambdatest/smartui-cli +``` + +### **Step 2:** Configure your Project Token + +Setup your project token show in the **SmartUI** app after, creating your project. + + + + +```bash +export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" +``` + + + + +```bash +set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" +``` + + + + +```bash +$Env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" +``` + + + +### **Step 3:** Execute Tests with Results Fetching + +You can fetch build results by adding the `--fetch-results` flag to your test execution command. Here are different ways to use this feature: + +#### Default Usage +If no filename is specified, results will be stored in `results.json`: + +```bash +npx smartui --config .smartui.json exec --fetch-results -- +``` +>**Example:** + For a `Node.js` test script : +>```bash +>npx smartui --config .smartui.json exec --fetch-results -- node test.js +>``` + +#### Custom Filename +Specify a custom filename for your results: + +```bash +npx smartui --config .smartui.json exec --fetch-results custom-results.json -- node test.js +``` + +### **Step 4:** Understanding the Results + +The fetched results JSON file contains detailed information about your build and screenshots. Here's what you'll find in the results file: + +```json +{ + "screenshots": { + "Screenshot-1": [ + { + "screenshot_name": "Screenshot-1", + "captured_image": "image_url", + "baseline_image": "image_url", + "compared_image": "image_url", + "browser_name": "edge", + "viewport": "1920", + "mismatch_percentage": 3.3, + "status": "Changes found" + }, + { + "screenshot_name": "Screenshot-1", + "captured_image": "image_url", + "baseline_image": "image_url", + "compared_image": "image_url", + "browser_name": "firefox", + "viewport": "1366", + "mismatch_percentage": 4.74, + "status": "Changes found" + }, + { + "screenshot_name": "Screenshot-1", + "captured_image": "image_url", + "baseline_image": "image_url", + "compared_image": "image_url", + "browser_name": "chrome", + "viewport": "1366", + "mismatch_percentage": 4.64, + "status": "Changes found" + }, + { + "screenshot_name": "Screenshot-1", + "captured_image": "image_url", + "baseline_image": "image_url", + "compared_image": "image_url", + "browser_name": "chrome", + "viewport": "1920", + "mismatch_percentage": 3.3, + "status": "Changes found" + }, + ], + "Screenshot-2": [ + { + "screenshot_name": "Screenshot-2", + "captured_image": "image_url", + "baseline_image": "image_url", + "compared_image": "image_url", + "browser_name": "edge", + "viewport": "1920", + "mismatch_percentage": 0.0, + "status": "Approved" + }, + { + "screenshot_name": "Screenshot-2", + "captured_image": "image_url", + "baseline_image": "image_url", + "compared_image": "image_url", + "browser_name": "firefox", + "viewport": "1366", + "mismatch_percentage": 4.74, + "status": "Changes found" + }, + { + "screenshot_name": "Screenshot-2", + "captured_image": "image_url", + "baseline_image": "image_url", + "compared_image": "image_url", + "browser_name": "chrome", + "viewport": "1366", + "mismatch_percentage": 4.64, + "status": "Changes found" + }, + { + "screenshot_name": "Screenshot-2", + "captured_image": "image_url", + "baseline_image": "image_url", + "compared_image": "image_url", + "browser_name": "chrome", + "viewport": "1920", + "mismatch_percentage": 3.3, + "status": "Changes found" + }, + ] + }, + "build": { + "build_id": "b420b7a9-77c6-****-****", + "baseline": false, + "build_type": "smartui-cli", + "build_status_ind": "completed", + "build_status": "pending-approval", + "commitId": "2b93***", + "branch": "main", + "commitAuthor": "John Doe", + "commitMessage": "Merge pull request from xyz/main" + }, + "project": { + "project_id": "1dfb7712-7f20-446f-***-***", + "name": "Project-Name", + "username": "johndoe", + "project_type": "smartui-cli", + "projectCategory": "web", + "platform": "cli" + } +} +``` + + diff --git a/sidebars.js b/sidebars.js index eb7e0c79..dcf9a8ee 100644 --- a/sidebars.js +++ b/sidebars.js @@ -2759,6 +2759,7 @@ module.exports = { "smartui-cli-git-branching-strategy", "smartui-cli-env-variables", "smartui-cli-responsive-dom", + "smartui-sdk-fetch-results", ], }, {