Skip to content

Commit

Permalink
Slack, Teams, and Data Extract API changes (#6031)
Browse files Browse the repository at this point in the history
Corresponding with our November releases, we have added the following capabilities and need the docs to match:

* Enterprise reporting offers filtering by Commit Branch in the UI
* Data Extract API allow for filtering by commit branch, has some data now available near real-time, and added spec path and test names to some endpoints as requested by customers.
* Slack integration has added the ability to configure when a message is sent based on Run Groups
* Microsoft Teams integration has added the ability to configure when a message is sent based on Run Groups and Tags.  The "beta" status of Teams integration is being removed and it is now official.

Where applicable, screen captures shown in our Docs will be updated in the December release because of more UI changes coming in the next few weeks.
  • Loading branch information
danjohansenconsulting authored Dec 10, 2024
1 parent daa3dad commit 0c4f784
Show file tree
Hide file tree
Showing 4 changed files with 79 additions and 17 deletions.
5 changes: 3 additions & 2 deletions docs/cloud/features/analytics/enterprise-reporting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,9 @@ The type of information and insights available on the various tabs are described

These are a few of the questions that can be answered by the cards shown on the **Overall Usage** tab of Enterprise
Reporting. Utilize the `Start Date` and `End Date` filters if you need to drill into a specific time period. The `Projects`
filter can be used to filter the data by one or multiple projects. The data on this tab correlates to what
Cypress Cloud [counts as a test result](/cloud/faq#What-counts-as-a-test-result).
filter can be used to filter the data by one or multiple projects. The `Branch` filter can be used to select one or more
branches that exist in your projects. The data on this tab correlates to what Cypress Cloud
[counts as a test result](/cloud/faq#What-counts-as-a-test-result).

<DocsImage
src="/img/cloud/features/analytics/enterprise-reporting-usage-report.png"
Expand Down
22 changes: 20 additions & 2 deletions docs/cloud/integrations/data-extract-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,9 @@ the ability to access your test data via API is already enabled.

<br />A couple of things to note:

- Data shown in Enterprise Reporting is not real-time. It is available as of midnight UTC of the current day.
I.e. - data will be populated through "end of day yesterday".
- Data shown in Enterprise Reporting is not real-time. For aggregated data it is available as of midnight UTC of the current day.
I.e. - data will be populated through "end of day yesterday". However, individual result data is near real-time and is available
30 minutes after the run has completed. Endpoints that return near real-time data are specified below.
- Historical data is available as specified by your data retention limit in Cypress Cloud.

## API endpoint details
Expand Down Expand Up @@ -80,6 +81,11 @@ The following query parameters are available for use:

For example, to request that the data be filtered for projects _Project e2e_ and _Phoenix_, use: `&projects=Project%20e2e&projects=Phoenix`

- branch. Optional, and accepts the name of your commit branch. The branch name should be URL encoded. You can pass as many branch
parameters as needed.

For example, to request that the data be filtered for branches _develop_ and _commitBranch1_, use: `&branch=develop&branch=commitBranch1`

- export_format. Specify the type of data format that will be returned in the response file. Valid options include: `csv`, `json`, `xlsx`

### Project list
Expand Down Expand Up @@ -422,6 +428,8 @@ This data allows you to extract all spec results regardless of spec result statu
returned is 500,000. Make multiple requests with different start_date and end_date parameter values if your volume of records will
exceed 500k for a single request.

Data is available near real-time, 30 minutes after the run has completed.

The report_id is: `spec-details`

Data returned includes:
Expand Down Expand Up @@ -455,6 +463,8 @@ This report allows you to extract all test result details for failed tests. The
returned is 500,000. Make multiple requests with different start_date and end_date parameter values if your volume of records will
exceed 500k for a single request.

Data is available near real-time, 30 minutes after the run has completed.

The report_id is: `failed-test-details`

Data returned includes:
Expand All @@ -472,6 +482,8 @@ Data returned includes:
- error_name (string array)
- error_message (string array)
- test_replay_url
- spec (path and filename)
- test_name (the test name matches what is shown in Cypress Cloud and a '///' delimiter is used between fragments of the test name)

Example:
https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=failed-test-details&export_format=csv&start_date=2024-01-01
Expand All @@ -482,6 +494,8 @@ This report allows you to extract all test result details regardless of test res
returned is 500,000. Make multiple requests with different start_date and end_date parameter values if your volume of records will
exceed 500k for a single request.

Data is available near real-time, 30 minutes after the run has completed.

The report_id is: `test-details`

Data returned includes:
Expand All @@ -500,6 +514,8 @@ Data returned includes:
- error_name (string array)
- error_message (string array)
- test_replay_url
- spec (path and filename)
- test_name (the test name matches what is shown in Cypress Cloud and a '///' delimiter is used between fragments of the test name)

Example:
https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=test-details&export_format=csv&start_date=2024-01-01
Expand Down Expand Up @@ -610,6 +626,8 @@ This report allows you to extract all test result details of flaky tests. The ma
returned is 500,000. Make multiple requests with different start_date and end_date parameter values if your volume of records will
exceed 500k for a single request.

Data is available near real-time, 30 minutes after the run has completed.

The report_id is: `flaky-test-details`

Data returned includes:
Expand Down
17 changes: 12 additions & 5 deletions docs/cloud/integrations/slack.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -118,16 +118,24 @@ To set notification preferences:
1. Navigate to Slack configuration page
2. Select the notification preferences for each channel

#### Filter notifications by tag
#### Filter notifications by tag or run group

When configuring at the project-level, you can filter notifications posted to Slack based on a
[tag](/app/references/command-line#cypress-run-tag-lt-tag-gt) the run was
[tag](/app/references/command-line#cypress-run-tag-lt-tag-gt) or
[group](/app/references/command-line#cypress-run-group-lt-name-gt) the run was
recorded with. This can be helpful to only post notifications on certain runs,
like those to a staging or release environment.

1. Navigate to the **Slack** configuration page
2. Under **Tags**, select the **Tag** dropdown for the desired integration and
select which tag to filter.
select which tag(s) to filter. If more than one tag is selected the Slack message will be sent
if any of the tags were associated with the run.
3. Under **Run Groups**, select the **Run Group** dropdown for the desired integration and
select which run group(s) to filter. If more than one run group is selected the Slack message will be
sent if any of the run groups were associated with the run.

If both a tag and run group are selected, only a run associated with both criteria will
trigger a message to be sent to Slack.

### Mute a channel

Expand All @@ -137,8 +145,7 @@ and resume notifications for a specific channel without losing the configuration
you've put in place.

1. Navigate to the **Slack** configuration page
2. Under **Actions**, select **Mute** for each Slack channel you want
muted.
2. Toggle the **Active** selector for each Slack channel you want muted.

### Remove a Slack channel

Expand Down
52 changes: 44 additions & 8 deletions docs/cloud/integrations/teams.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: 'Microsoft Teams Integration | Cypress Documentation'
title: 'Microsoft Teams Integration'
description: 'Setup Microsoft Teams notifications for your Cypress Cloud organization and projects.'
sidebar_position: 60
sidebar_label: Microsoft Teams
Expand All @@ -15,13 +15,6 @@ sidebar_label: Microsoft Teams

:::

:::caution

<Icon name="exclamation-triangle" /> Microsoft Teams integration is currently in
beta

:::

The Microsoft Teams integration allows you to see your Cypress test results
directly in your Microsoft Teams channels.

Expand Down Expand Up @@ -224,3 +217,46 @@ needs. You can have Cypress Cloud post run results for a specific project.
Once you've configured and added your webhooks, the installation is complete!
Cypress Cloud will post run results for **this** project to the specified
Microsoft Teams channel.

## Notification preferences

By default, Cypress Cloud will post a Teams message to each configured channel
only for failing runs. However, you can select any combination of the followings preferences:

- **Failed runs**: will notify on runs with a _failed_ status.
- **Passed runs**: will notify on runs with a _passed_ status.
- **Flaky tests**: will notify on runs that have an identified _flaky_ test.

To set notification preferences:

1. Navigate to Teams configuration page
2. Select the notification preferences for each channel

#### Filter notifications by tag or run group

When configuring at the project-level, you can filter notifications posted to Teams based on a
[tag](/app/references/command-line#cypress-run-tag-lt-tag-gt) or
[group](/app/references/command-line#cypress-run-group-lt-name-gt) the run was
recorded with. This can be helpful to only post notifications on certain runs,
like those to a staging or release environment.

1. Navigate to the **Teams** configuration page
2. Under **Tags**, select the **Tag** dropdown for the desired integration and
select which tag(s) to filter. If more than one tag is selected the Teams message will be sent
if any of the tags were associated with the run.
3. Under **Run Groups**, select the **Run Group** dropdown for the desired integration and
select which run group(s) to filter. If more than one run group is selected the Teams message will be
sent if any of the run groups were associated with the run.

If both a tag and run group are selected, only a run associated with both criteria will
trigger a message to be sent to Teams.

### Mute a channel

If you want Cypress Cloud to temporarily stop posting Teams messages to a
certain channel, you can **Mute** that channel. This allows you to easily pause
and resume notifications for a specific channel without losing the configuration
you've put in place.

1. Navigate to the **Teams** configuration page
2. Toggle the **Active** selector for each Teams channel you want muted.

0 comments on commit 0c4f784

Please sign in to comment.