Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split some URL tests into separate files to allow for better ShadowRealm coverage #904

Open
ptomato opened this issue Nov 12, 2024 · 0 comments
Labels
test-change-proposal Proposal to add or remove tests for an interop area

Comments

@ptomato
Copy link

ptomato commented Nov 12, 2024

Test List

https://wpt.fyi/results/url?sha=f382c0ce93&max-count=2&product=chrome&product=firefox&product=safari&view=interop&q=IdnaTestV2.any.html%20or%20IdnaTestV2.window.html%20or%20urlencoded-parser.any.html%20or%20urlencoded-parser.any.worker.html%20or%20urlencoded-parser-request-response.any.html%20or%20failure.html%20or%20url-constructor-base-failure.any.html

Rationale

As part of the TC39 ShadowRealm proposal we are trying to increase WPT coverage for the various Web APIs exposed in ShadowRealm. This includes running existing tests for APIs such as URL in ShadowRealm scopes. However, in order to run as many test files as possible in ShadowRealm scopes, it's necessary to split some of the files that previously contained a mix of tests both suitable and unsuitable for executing in ShadowRealm scopes. See web-platform-tests/wpt#41985 for more details.

Here are the splits and renames made in that PR that are relevant to Interop:

  • urlencoded-parser.any.js → split out urlencoded-parser-request-response.any.js
  • failure.html → split out url-constructor-base-failure.any.js
  • IdnaTestV2.window.html → renamed to IdnaTestV2.any.html

Concretely, we propose to add the following tests to the Interop 2023 URL label:

  • IdnaTestV2.any.html
  • url-constructor-base-failure.any.html
  • urlencoded-parser-request-response.any.html

and remove:

  • IdnaTestV2.window.html

(See web-platform-tests/wpt-metadata#6930 for how this would be implemented in wpt-metadata.)

Because subtests have different weights in the Interop score depending on how many other subtests there are in the file, this has an effect on the scores, although it is small. Here is a wpt.fyi link (courtesy of @gsnedders) that I believe shows the effect on the scores. From the numbers there:

  • Chrome: 95.39% → 95.55%
  • Firefox: 99.28% → 99.10%
  • Safari: 99.78% → 99.80%
@ptomato ptomato added the test-change-proposal Proposal to add or remove tests for an interop area label Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test-change-proposal Proposal to add or remove tests for an interop area
Projects
None yet
Development

No branches or pull requests

1 participant