Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add extended JSON file build output #1407

Merged
merged 8 commits into from
Jul 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@
!/scripts/find-troublesome-ancestors.ts
!/scripts/release.ts
!/scripts/update-drafts.ts
!/scripts/validate.ts
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,13 @@
"node": ">=18.19.0"
},
"scripts": {
"build:extended": "tsx scripts/build.ts extended-json",
"build": "tsx scripts/build.ts package",
"dist": "tsx scripts/dist.ts",
"feature-init": "tsx scripts/feature-init.ts",
"format": "npx prettier --write .",
"schema:write": "npm run schema -- --out ./schemas/data.schema.json",
"schema": "ts-json-schema-generator --tsconfig ./tsconfig.json --path ./types.ts",
"schema": "ts-json-schema-generator --tsconfig ./tsconfig.json --path ./types.ts --type=WebFeaturesData",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bit is pretty serious:

I figured out what that top-level $ref was for. Without it, it was just a list of definitions without asserting anything using those definitions. Without this option, any object will pass validation against our schema. 🙄

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh... let's put it back!

"test:caniuse": "tsx scripts/caniuse.ts",
"test:coverage": "npm run --workspaces test:coverage",
"test:dist": "tsx scripts/dist.ts --check",
Expand Down
63 changes: 63 additions & 0 deletions schemas/data.schema.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"$ref": "#/definitions/WebFeaturesData",
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"FeatureData": {
Expand Down Expand Up @@ -111,6 +112,68 @@
"description": "Date the feature achieved Baseline low status",
"type": "string"
},
"by_compat_key": {
"additionalProperties": {
"additionalProperties": false,
"properties": {
"baseline": {
"description": "Whether the feature is Baseline (low substatus), Baseline (high substatus), or not (false)",
"enum": [
"high",
"low",
false
],
"type": [
"string",
"boolean"
]
},
"baseline_high_date": {
"description": "Date the feature achieved Baseline high status",
"type": "string"
},
"baseline_low_date": {
"description": "Date the feature achieved Baseline low status",
"type": "string"
},
"support": {
"additionalProperties": false,
"description": "Browser versions that most-recently introduced the feature",
"properties": {
"chrome": {
"type": "string"
},
"chrome_android": {
"type": "string"
},
"edge": {
"type": "string"
},
"firefox": {
"type": "string"
},
"firefox_android": {
"type": "string"
},
"safari": {
"type": "string"
},
"safari_ios": {
"type": "string"
}
},
"type": "object"
}
},
"required": [
"baseline",
"support"
],
"type": "object"
},
"description": "Statuses for each key in the feature's compat_features list, if applicable. Not available to the npm release of web-features.",
"type": "object"
},
"support": {
"additionalProperties": false,
"description": "Browser versions that most-recently introduced the feature",
Expand Down
66 changes: 63 additions & 3 deletions scripts/build.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,21 @@
import { basename } from "node:path";
import { DefinedError } from "ajv";
import { getStatus } from "compute-baseline";
import stringify from "fast-json-stable-stringify";
import { execSync } from "node:child_process";
import fs from "node:fs";
import stringify from "fast-json-stable-stringify";
import { basename } from "node:path";
import winston from "winston";
import yargs from "yargs";
import * as data from "../index.js";
import { validate } from "./validate.js";

const logger = winston.createLogger({
format: winston.format.combine(
winston.format.colorize(),
winston.format.simple(),
),
transports: new winston.transports.Console(),
});

const rootDir = new URL("..", import.meta.url);

Expand All @@ -14,14 +26,23 @@ yargs(process.argv.slice(2))
describe: "Generate the web-features npm package",
handler: buildPackage,
})
.command({
command: "extended-json",
describe: "Generate a web-features JSON file with BCD per-key statuses",
handler: buildExtendedJSON,
})
.parseSync();

function buildPackage() {
const packageDir = new URL("./packages/web-features/", rootDir);
const filesToCopy = ["LICENSE.txt", "types.ts", "schemas/data.schema.json"];

if (!valid(data)) {
logger.error("Data failed schema validation. No package built.");
process.exit(1);
}

const json = stringify(data);
// TODO: Validate the resulting JSON against a schema.
Comment on lines +40 to -24
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I had to figure out how to validate the built JSON against the schema for the extended JSON, I did the same for the packaged one.

const path = new URL("data.json", packageDir);
fs.writeFileSync(path, json);
for (const file of filesToCopy) {
Expand All @@ -39,3 +60,42 @@ function buildPackage() {
encoding: "utf-8",
});
}

function buildExtendedJSON() {
for (const [id, featureData] of Object.entries(data.features)) {
if (
Array.isArray(featureData.compat_features) &&
featureData.compat_features.length &&
featureData.status
) {
featureData.status.by_compat_key = {};
for (const key of featureData.compat_features) {
featureData.status.by_compat_key[key] = getStatus(id, key);
}
}
}

if (!valid(data)) {
logger.error("Data failed schema validation. No JSON file written.");
process.exit(1);
}

fs.writeFileSync(
new URL("./web-features.extended.json", rootDir),
stringify(data),
);
}

function valid(data: any): boolean {
const valid = validate(data);
if (!valid) {
// TODO: turn on strictNullChecks, fix all the errors, and replace this with:
// const errors = validate.errors;
const errors = (valid as any).errors as DefinedError[];
Comment on lines +91 to +94
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit ugly because ajv can't produce (useful) types without strictNullChecks set to true in tsconfig.json. Fixable, but it requires touching a dozen files not germane to this PR.

for (const error of errors) {
logger.error(`${error.instancePath}: ${error.message}`);
}
return false;
}
return true;
}
21 changes: 8 additions & 13 deletions scripts/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,9 @@ import fs from "node:fs";
import path from "node:path";
import url from 'url';

import Ajv from 'ajv';
import addFormats from 'ajv-formats';

import { DefinedError } from "ajv";
import * as data from '../index.js';

import schema from '../schemas/data.schema.json' assert { type: 'json' };
import { validate } from "./validate.js";

let status: 0 | 1 = 0;

Expand All @@ -25,21 +22,19 @@ function checkSchemaConsistency(): void {
}
}

function validate() {
const ajv = new Ajv({allErrors: true});
addFormats(ajv);

const validate = ajv.compile(schema);

function valid() {
const valid = validate(data);
if (!valid) {
for (const error of validate.errors) {
// TODO: turn on strictNullChecks, fix all the errors, and replace this with:
// const errors = validate.errors;
const errors = (validate as any).errors as DefinedError[];
for (const error of errors) {
console.error(`${error.instancePath}: ${error.message}`);
}
status = 1;
}
}

checkSchemaConsistency();
validate();
valid();
process.exit(status);
21 changes: 21 additions & 0 deletions scripts/validate.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import Ajv from "ajv";
import addFormats from "ajv-formats";
import assert from "node:assert/strict";

import * as schema from "../schemas/data.schema.json" with { type: "json" };

export function validate(data: any) {
const ajv = new Ajv({ allErrors: true, allowUnionTypes: true });
Comment on lines +7 to +8
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I consolidated compiling the schema here, so that we can have one place with settings (and to check that we don't do something foolish like validate nothing at all).

addFormats(ajv);
// TODO: turn on strictNullChecks, fix all the errors, and replace this with:
// const validator = ajv.compile<WebFeaturesData>(schema);
const validator = ajv.compile(schema);

assert.equal(
validator({}),
false,
"Failed confidence check: schema validates empty object",
);

return validator;
}
7 changes: 6 additions & 1 deletion types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ type browserIdentifier = "chrome" | "chrome_android" | "edge" | "firefox" | "fir

type BaselineHighLow = "high" | "low";

interface SupportStatus {
interface Status {
/** Whether the feature is Baseline (low substatus), Baseline (high substatus), or not (false) */
baseline: BaselineHighLow | false;
/** Date the feature achieved Baseline low status */
Expand All @@ -45,6 +45,11 @@ interface SupportStatus {
};
}

interface SupportStatus extends Status {
/** Statuses for each key in the feature's compat_features list, if applicable. Not available to the npm release of web-features. */
by_compat_key?: Record<string, Status>
}

/** Specification URL
* @format uri
*/
Expand Down