Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Analytics docs refresh #798

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/admin/pricing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,4 @@ A billable active Code Search or Code Intelligence Platform user is one who is s

A user who has accessed a Sourcegraph product is counted as active once they complete an action that represents usage reflected above.

User counts can be found in the **Site admin** section of the Sourcegraph web application, and Cody-specific usage metrics can be found in the [Cody Analytics service](../cody/clients/enable-cody-enterprise#analytics).
User counts and more usage metrics can be found in [Sourcegraph Analytics](../analytics).
12 changes: 12 additions & 0 deletions docs/analytics/cloud.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Sourcegraph Analytics for Cloud customers

The following instructions are for Sourcegraph Cloud customers only. Customers that self-host can view our [self-hosted analytics enablement](./self-hosted).

## Enablement instructions

To enable Sourcegraph Analytics:

- Create an account on [Sourcegraph Accounts](https://accounts.sourcegraph.com/), or find the email address associated with your existing account.
- Get in touch with your Sourcegraph Technical Advisor or point of contact (or email us at [email protected] if you don't know your contact), provide them with the email address you used to register above, and ask for access to Sourcegraph Analytics.
- They will validate your account and link it to your Sourcegraph Enterprise instance's usage metrics.
- Sign in to [Sourcegraph Analytics](https://analytics.sourcegraph.com).
59 changes: 59 additions & 0 deletions docs/analytics/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Sourcegraph Analytics

Enterprise customers can use Sourcegraph Analytics to get a clear view of usage, engagement, performance, and impact on a self-serve basis.

## [For Sourcegraph Cloud customers](/analytics/cloud)

[Sourcegraph Cloud](/cloud) customers can use our managed [cloud analytics service](https://analytics.sourcegraph.com) for Cody and Code Search usage data.
- [Enablement instructions](/analytics/cloud#enablement)

![Sourcegraph-cloud-analytics](https://storage.googleapis.com/sourcegraph-assets/cloud-analytics.png)

## [For self-hosted customers](/analytics/self-hosted)

Self-hosted customers can use our self-hosted and locally deployed analytics service, built on Grafana, to see Cody and Code Search usage data.

- [Set up instructions](/analytics/self-hosted#set-up)
- [Updates](/analytics/self-hosted#updates)
- [Custom metrics](/analytics/self-hosted#custom-metrics)

![Sourcegraph-self-hosted-analytics](https://storage.googleapis.com/sourcegraph-assets/self-hosted-analytics.png)

## Metrics

We provide a set of metrics that can be used to monitor the usage, engagement, performance, and impact of Sourcegraph inside of your enterprise.

Learn more about how we think about [the ROI of Sourcegraph in our blog](https://sourcegraph.com/blog/measuring-the-impact-of-ai-coding-tools).

### Summary metrics

Summary metrics are not yet available.

### Code Search metrics

Code Search metrics are not yet available.

### Cody user metrics

| Metric | Description |
|--------|-------------|
| Authenticated users | All users who sign in to Cody in an editor or interact with Cody on the web, during the period selected. [Read more](/admin/pricing#authenticated-users). |
| Active users | All users who sign in to Cody in an editor or interact with Cody on the web, during the period selected. [Read more](/admin/pricing#billable-active-users). |
| Frequency of Usage | Count of users who used Cody n times over the past 30 days. |

### Cody completion metrics

| Metric | Description |
|--------|-------------|
| Hour saved | The number of hours saved by Cody users, assuming 2 minutes saved per completion. |
| Completions and completion users | The number of completions suggested and users who were presented with a completion suggestion. |
| Completion acceptance rate (CAR) | The percent of completions presented to a user for at least 750ms that were accepted. |
| Weighted completion acceptance rate (wCAR) | Similar to CAR, but weighted by the number of characters presented in the completion, and only counting those that were fully retained by the user for X minutes after accepating the completion. |
| Average completion latency (ms) | The average milliseconds of latency before a user is presented with a completion suggestion. |

### Cody chat metrics

| Metric | Description |
|--------|-------------|
| Hour saved | The number of hours saved by Cody users, assuming 5 minutes saved per completion. |
| Chats and chat users | The number of chats and users who chatted with Cody. |
15 changes: 15 additions & 0 deletions docs/analytics/self-hosted.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Sourcegraph Analytics for self-hosted customers

The following instructions are for Sourcegraph self-hosted customers only. Customers that use Sourcegraph cloud hosting can view our [cloud analytics enablement](./cloud).

## Set-up instructions

Coming soon.

## Updates

Coming soon.

## Custom metrics

Coming soon.
21 changes: 3 additions & 18 deletions docs/cody/clients/enable-cody-enterprise.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,25 +38,10 @@ Sourcegraph account admins have selective control over users' access to Cody Ent

### Analytics

<Callout type= "note">Cody Analytics are supported with VS Code IDE extension and on the latest versions of JetBrains IDE.</Callout>
See our [analytics documentation](../../analytics) for more details.

Cody Enterprise users get a clear view of usage analytics for their instance on a self-service basis. A separately managed cloud service for Cody analytics handles user auth, gets metrics data from Sourcegraph's BigQuery instance, and visualizes the metrics data.

The following metrics are available for Cody Enterprise users:

| **Metric Type** | **What is measured?** |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Active users | - Total active users <br /> - Average daily users <br /> - Average no. of days each user used Cody (of last 30 days) <br /> - Cody users by day (last 30 days) <br /> - Cody users by month (last two months) <br /> - Cody users by number of days used |
| Completions | - Total accepted completions <br /> - Minutes saved per completion <br /> - Hours saved by completions <br /> - Cody completions by day <br /> - Completions acceptance rate <br /> - Weighted completions acceptance rate <br /> - Average completion latency <br /> - Acceptance rate by language <br /> |
| Chat | - Total chat events <br /> - Minutes saved per chat <br /> - Hours saved by chats <br /> - Cody chats by day |
| Commands | - Total command events <br /> - Minutes saved per command <br /> - Hours saved by commands <br /> - Cody commands by day <br /> - Most used commands |

To enable Cody Analytics:

- Create an account on [Sourcegraph Accounts](https://accounts.sourcegraph.com/)
- A user already having an account on Sourcegraph.com gets automatically migrated to Sourcegraph Accounts. Users can sign in to Cody Analytics using their email and password
- Users without a Sourcegraph.com account, please get in touch with one of our teammates. They can help with both the account setup and assigning instances to specific users
- Map your user account to a Sourcegraph instance, and this gives you access to Cody's analytics
- [How to enable Sourcegraph Analytics for Cloud customers](../../analytics/cloud)
- [How to enable Sourcegraph Analytics for self-hosted customers](../../analytics/self-hosted)

### Multi-repository context

Expand Down
9 changes: 9 additions & 0 deletions src/data/navigation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,15 @@ export const navigation: NavigationItem[] = [
{ title: "Pricing", href: "/admin/pricing" },
],
},
{
title: "Sourcegraph Analytics",
href: "/analytics",
sections: [
{ title: "Metrics", href: "/analytics#metrics" },
{ title: "Cloud Analytics", href: "/analytics/cloud" },
{ title: "Self-hosted Analytics", href: "/analytics/self-hosted" },
],
},
{
title: "Sourcegraph Cloud",
href: "/cloud",
Expand Down
Loading