Skip to content
This repository has been archived by the owner on Sep 19, 2024. It is now read-only.

Commit

Permalink
Merge branch 'development' into html-comments
Browse files Browse the repository at this point in the history
  • Loading branch information
whilefoo authored Aug 22, 2023
2 parents bffef7d + fee0dd9 commit ee91997
Show file tree
Hide file tree
Showing 11 changed files with 302 additions and 79 deletions.
7 changes: 4 additions & 3 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ SUPABASE_KEY=
AUTO_PAY_MODE=
ANALYTICS_MODE=

# Use `trace` to get verbose logging or `info` to show less
LOG_LEVEL=debug
LOGDNA_INGESTION_KEY=
# Log environment
LOG_ENVIRONMENT=production # development to see logs in console
LOG_LEVEL=debug # 0: error 1: warn 2: info 3: http 4: verbose 5: debug 6: silly
LOG_RETRY=0 # 0 for no retry, more than 0 for the number of retries
66 changes: 49 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Ubiquity DAO's GitHub Bot to automate DevPool management.
git clone https://github.com/ubiquity/ubiquibot.git
cd ubiquibot
yarn
yarn tsc
yarn build
yarn start:watch
```

Expand Down Expand Up @@ -45,13 +45,14 @@ To test the bot, you can:

1. Create a new issue
2. Add a time label, ex: `Time: <1 Day`
3. At this point the bot should add a price label.
3. Add a priority label, ex: `Priority: 0 (Normal)`
4. At this point the bot should add a price label.

## Configuration

`chain-id` is ID of the EVM-compatible network that will be used for payouts.
`evm-network-id` is ID of the EVM-compatible network that will be used for payouts.

`base-multiplier` is a base number that will be used to calculate bounty price based on the following formula: `price = base-multiplier * time-label-weight * priority-label-weight / 10`
`price-multiplier` is a base number that will be used to calculate bounty price based on the following formula: `price = price-multiplier * time-label-weight * priority-label-weight * 100`

`time-labels` are labels for marking the time limit of the bounty:

Expand All @@ -64,15 +65,24 @@ To test the bot, you can:
- `name` is a human-readable name
- `weight` is a number that will be used to calculate the bounty price

`command-settings` are setting to enable or disable a command

- `name` is the name of the command
- `enabled` is a `true` or `false` value to enable or disable a command

`default-labels` are labels that are applied when an issue is created without any time or priority labels.

`auto-pay-mode` can be `true` or `false` that enables or disables automatic payout of bounties when the issue is closed.
`assistive-pricing` to create a new pricing label if it doesn't exist. Can be `true` or `false`.

`disable-analytics` can be `true` or `false` that disables or enables weekly analytics collection by Ubiquity.

`payment-permit-max-price` sets the max amount for automatic payout of bounties when the issue is closed.

`analytics-mode` can be `true` or `false` that enables or disables weekly analytics collection by Ubiquity.
`comment-incentives` can be `true` or `false` that enable or disable comment incentives. These are payments generated for comments in the issue by contributors, excluding the assignee.

`incentive-mode` can be `true` or `false` that enables or disables comment incentives. These are comments in the issue by either the creator of the bounty or other users.
`issue-creator-multiplier` is a number that defines a base multiplier for calculating incentive for the creator of the issue.

`issue-creator-multiplier` is a number that defines a base multiplier for calculating incentive reward for the creator of the issue.
`comment-element-pricing` defines how much is a part of the comment worth. For example `text: 0.1` means that any text in the comment will add 0.1

`incentives` defines incentive rewards:

Expand All @@ -81,7 +91,11 @@ To test the bot, you can:
- `totals`:
- `word` defines reward for each word in the comment

`max-concurrent-bounties` is the maximum number of bounties that can be assigned to a bounty hunter at once. This excludes bounties with pending pull request reviews.
`max-concurrent-assigns` is the maximum number of bounties that can be assigned to a bounty hunter at once. This excludes bounties with delayed or approved pull request reviews.

`register-wallet-with-verification` can be `true` or `false`. If enabled, it requires a signed message to set wallet address. This prevents users from setting wallet address from centralized exchanges, which would make payments impossible to claim.

`promotion-comment` is a message that is appended to the payment permit comment.

## How to run locally

Expand Down Expand Up @@ -113,7 +127,7 @@ DISQUALIFY_TIME="7 days" // 7 days

4. `yarn install`
5. Open 2 terminal instances:
- in one instance run `yarn tsc --watch` (compiles the Typescript code)
- in one instance run `yarn build --watch` (compiles the Typescript code)
- in another instance run `yarn start:watch` (runs the bot locally)
6. Open `localhost:3000` and follow instructions to add the bot to one of your repositories.

Expand All @@ -124,7 +138,8 @@ You can, for example:

1. Create a new issue
2. Add a time label, ex: `Time: <1 Day`
3. At this point the bot should add a price label, you should see event logs in one of your opened terminals
3. Add a priority label, ex: `Priority: 0 (Normal)`
4. At this point the bot should add a price label, you should see event logs in one of your opened terminals

## How it works

Expand All @@ -140,13 +155,31 @@ When using as a github app the flow is the following:
4. Event details are sent to your deployed bot instance (to a webhook URL that was set in github app's settings)
5. The bot handles the event

## Payments Permits in a local instance

For payment to work in your local instance, ubiquibot must be set up in a Github organization. It will not work for a ubiquibot instance set up in a personal account. Once, you have an ubiquibot instance working in an organization, follow the steps given below:

1. Create a new private repository in your Github organization with name `ubiquibot-config`
2. Add your ubiquibot app to `ubiquibot-config` repository.
3. Create a file `.github/ubiquibot-config.yml` in it. Fill the file with contents from [this file](https://github.com/ubiquity/ubiquibot/blob/development/.github/ubiquibot-config.yml).
4. Go to https://pay.ubq.fi/keygen and generate X25519 public/private key pair. Fill private key of your wallet's address in `PLAIN_TEXT` field and click `Encrypt`.
5. Copy the `CIPHER_TEXT` and append it to your repo `ubiquibot-config/.github/ubiquibot-config.yml` as

`private-key-encrypted: "PASTE_YOUR_CIPHER_TEXT_HERE"`

6. Copy the `X25519_PRIVATE_KEY` and append it in your local ubiquibot repository `.env` file as

`X25519_PRIVATE_KEY=PASTE_YOUR_X25519_PRIVATE_KEY_HERE`

## How to QA any additions to the bot

1. Fork the ubiquibot repo and install the [ubiquibot-qa app](https://github.com/apps/ubiquibot-qa) on the forked repository.
2. Enable github action running on the forked repo and allow `issues` on the settings tab.
3. Create a [QA issue](https://github.com/ubiquibot/staging/issues/21) similar to this where you show the feature working in the forked repo
4. Describe carefully the steps taken to get the feature working, this way our team can easily verify
5. Link that QA issue to the pull request as indicated on the template before requesting a review
Make sure you have your local instance of ubiquibot running.

1. Fork the ubiquibot repo and add your local instance of ubiquibot to the forked repository.
2. Enable Github action running on the forked repo and allow `issues` on the settings tab.
3. Create a [QA issue](https://github.com/ubiquibot/staging/issues/21) similar to this where you show the feature working in the forked repo.
4. Describe carefully the steps taken to get the feature working, this way our team can easily verify.
5. Link that QA issue to the pull request as indicated on the template before requesting a review.

## How to create a new release

Expand Down Expand Up @@ -196,7 +229,6 @@ We can't use a `jsonc` file due to limitations with Netlify. Here is a snippet o
/* https://github.com/syntax-tree/mdast#nodes */
"strong": 0 // Also includes italics, unfortunately https://github.com/syntax-tree/mdast#strong
/* https://github.com/syntax-tree/mdast#gfm */

}
}
```
2 changes: 0 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@
"format:check": "prettier -c src/**/*.ts",
"format": "prettier --write src",
"lint": "eslint --ext .ts ./src",
"prestart:watch": "ln -s src/assets/images lib/assets/images",
"start:serverless": "tsx src/adapters/github/github-actions.ts",
"start:watch": "nodemon --exec 'yarn start'",
"start": "probot run ./lib/src/index.js",
Expand All @@ -29,7 +28,6 @@
"@actions/core": "^1.10.0",
"@commitlint/cli": "^17.4.3",
"@commitlint/config-conventional": "^17.4.3",
"@logdna/logger": "^2.6.6",
"@netlify/functions": "^1.4.0",
"@probot/adapter-aws-lambda-serverless": "^3.0.2",
"@probot/adapter-github-actions": "^3.1.3",
Expand Down
1 change: 1 addition & 0 deletions src/adapters/supabase/helpers/index.ts
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
export * from "./client";
export * from "./log";
193 changes: 193 additions & 0 deletions src/adapters/supabase/helpers/log.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
import { getAdapters, getBotContext, Logger } from "../../../bindings";
import { Payload } from "../../../types";
import { getNumericLevel } from "../../../utils/helpers";
import { getOrgAndRepoFromPath } from "../../../utils/private";
interface Log {
repo: string | null;
org: string | null;
commentId: number | undefined;
issueNumber: number | undefined;
logMessage: string;
level: Level;
timestamp: string;
}

export enum Level {
ERROR = "error",
WARN = "warn",
INFO = "info",
HTTP = "http",
VERBOSE = "verbose",
DEBUG = "debug",
SILLY = "silly",
}

export class GitHubLogger implements Logger {
private supabase;
private maxLevel;
private app;
private logEnvironment;
private logQueue: Log[] = []; // Your log queue
private maxConcurrency = 6; // Maximum concurrent requests
private retryDelay = 1000; // Delay between retries in milliseconds
private throttleCount = 0;
private retryLimit = 0; // Retries disabled by default

constructor(app: string, logEnvironment: string, maxLevel: Level, retryLimit: number) {
this.app = app;
this.logEnvironment = logEnvironment;
this.maxLevel = getNumericLevel(maxLevel);
this.retryLimit = retryLimit;
this.supabase = getAdapters().supabase;
}

async sendLogsToSupabase({ repo, org, commentId, issueNumber, logMessage, level, timestamp }: Log) {
const { error } = await this.supabase.from("logs").insert([
{
repo_name: repo,
level: getNumericLevel(level),
org_name: org,
comment_id: commentId,
log_message: logMessage,
issue_number: issueNumber,
timestamp,
},
]);

if (error) {
console.error("Error logging to Supabase:", error.message);
return;
}
}

async processLogs(log: Log) {
try {
await this.sendLogsToSupabase(log);
} catch (error) {
console.error("Error sending log, retrying:", error);
return this.retryLimit > 0 ? await this.retryLog(log) : null;
}
}

async retryLog(log: Log, retryCount = 0) {
if (retryCount >= this.retryLimit) {
console.error("Max retry limit reached for log:", log);
return;
}

await new Promise((resolve) => setTimeout(resolve, this.retryDelay));

try {
await this.sendLogsToSupabase(log);
} catch (error) {
console.error("Error sending log (after retry):", error);
await this.retryLog(log, retryCount + 1);
}
}

async processLogQueue() {
while (this.logQueue.length > 0) {
const log = this.logQueue.shift();
if (!log) {
continue;
}
await this.processLogs(log);
}
}

async throttle() {
if (this.throttleCount >= this.maxConcurrency) {
return;
}

this.throttleCount++;
try {
await this.processLogQueue();
} finally {
this.throttleCount--;
if (this.logQueue.length > 0) {
await this.throttle();
}
}
}

async addToQueue(log: Log) {
this.logQueue.push(log);
if (this.throttleCount < this.maxConcurrency) {
await this.throttle();
}
}

private save(logMessage: string | object, level: Level, errorPayload?: string | object) {
if (getNumericLevel(level) > this.maxLevel) return; // only return errors lower than max level

const context = getBotContext();
const payload = context.payload as Payload;
const timestamp = new Date().toUTCString();

const { comment, issue, repository } = payload;
const commentId = comment?.id;
const issueNumber = issue?.number;
const repoFullName = repository?.full_name;

const { org, repo } = getOrgAndRepoFromPath(repoFullName);

if (!logMessage) return;

if (typeof logMessage === "object") {
// pass log as json stringified
logMessage = JSON.stringify(logMessage);
}

this.addToQueue({ repo, org, commentId, issueNumber, logMessage, level, timestamp })
.then(() => {
return;
})
.catch(() => {
console.log("Error adding logs to queue");
});

if (this.logEnvironment === "development") {
console.log(this.app, logMessage, errorPayload, level, repo, org, commentId, issueNumber);
}
}

info(message: string | object, errorPayload?: string | object) {
this.save(message, Level.INFO, errorPayload);
}

warn(message: string | object, errorPayload?: string | object) {
this.save(message, Level.WARN, errorPayload);
}

debug(message: string | object, errorPayload?: string | object) {
this.save(message, Level.DEBUG, errorPayload);
}

error(message: string | object, errorPayload?: string | object) {
this.save(message, Level.ERROR, errorPayload);
}

async get() {
try {
const { data, error } = await this.supabase.from("logs").select("*");

if (error) {
console.error("Error retrieving logs from Supabase:", error.message);
return [];
}

return data;
} catch (error) {
if (error instanceof Error) {
// 👉️ err is type Error here
console.error("An error occurred:", error.message);

return;
}

console.log("Unexpected error", error);
return [];
}
}
}
10 changes: 4 additions & 6 deletions src/bindings/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { getPayoutConfigByNetworkId } from "../helpers";
import { ajv } from "../utils";
import { Context } from "probot";
import { getScalarKey, getWideConfig } from "../utils/private";
import { Level } from "../adapters/supabase";

export const loadConfig = async (context: Context): Promise<BotConfig> => {
const {
Expand All @@ -32,8 +33,9 @@ export const loadConfig = async (context: Context): Promise<BotConfig> => {

const botConfig: BotConfig = {
log: {
level: process.env.LOG_LEVEL || "debug",
ingestionKey: process.env.LOGDNA_INGESTION_KEY ?? "",
logEnvironment: process.env.LOG_ENVIRONMENT || "production",
level: (process.env.LOG_LEVEL as Level) || Level.DEBUG,
retryLimit: Number(process.env.LOG_RETRY) || 0,
},
price: {
baseMultiplier,
Expand Down Expand Up @@ -84,10 +86,6 @@ export const loadConfig = async (context: Context): Promise<BotConfig> => {
},
};

if (botConfig.log.ingestionKey == "") {
throw new Error("LogDNA ingestion key missing");
}

if (botConfig.payout.privateKey == "") {
botConfig.mode.paymentPermitMaxPrice = 0;
}
Expand Down
Loading

0 comments on commit ee91997

Please sign in to comment.