Remove `endpoint.write_to_disk()` in favor of standardized `endpoint.output()` #75059

wbinnssmith · 2025-01-18T01:17:32Z

This:

Makes the Endpoint trait not manage any file writing on its own, instead returning a structure containing OutputAssets, the paths where the OutputAssets can be found, and the associated Project.
Allows for OutputAssets to be written at a future point in Turbopack
Implements endpoint_write_to_disk for compatibility in dev. This continues to be used for prod builds, but in the future it won’t.

In a future PR, production builds will dedupe and unify all output assets, writing them to disk at once, rather than exposing individual endpoint writes to JS to occur sequentially.

Test Plan: CI

…output()` This: - Makes the Endpoint trait not manage any file writing on its own, instead returning a structure containing OutputAssets, the paths where the OutputAssets can be found, and the associated Project. - Allows for OutputAssets to be written at a future point in Turbopack - Implements `endpoint_write_to_disk` for compatibility in dev. This continues to be used for prod builds, but in the future it won’t. In a future PR, production builds will dedupe and unify all output assets, writing them to disk at once, rather than exposing individual endpoint writes to JS to occur sequentially. Test Plan: CI

wbinnssmith · 2025-01-18T01:17:44Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Remove endpoint.write_to_disk() in favor of standardized endpoint.output() #75059 👈 (View in Graphite)
refactor(turbopack): Only perform strongly consistent reads/resolves on VcOperation #75016 : 1 other dependent PR (#75051 )
perf(turbo-tasks): Optionally schedule ResolveNative/ResolveTrait tasks as local tasks #69126
refactor(turbopack/next-api): Make VcArc use OperationVc #74479
fix(turbopack-swc-utils): Don't panic if issues aren't emitted, use #[must_use] annotation #74906
canary

This stack of pull requests is managed by Graphite. Learn more about stacking.

ijjk · 2025-01-18T01:29:00Z

Failing test suites

Commit: efad310

pnpm test-dev-turbo test/development/acceptance/ReactRefreshRequire.test.ts (turbopack)

ReactRefreshRequire > propagates hot update to all inverse dependencies

Expand output

● ReactRefreshRequire › propagates hot update to all inverse dependencies

expect(received).toEqual(expected) // deep equality

Expected: ["init LeafV1", "init MiddleAV1", "init MiddleBV1", "init MiddleCV1", "init RootV1"]
Received: undefined

  243 |     )
  244 |
> 245 |     expect(await session.evaluate(() => (window as any).log)).toEqual([
      |                                                               ^
  246 |       'init LeafV1',
  247 |       'init MiddleAV1',
  248 |       'init MiddleBV1',

  at Object.toEqual (development/acceptance/ReactRefreshRequire.test.ts:245:63)

Read more about building and testing Next.js in contributing.md.

pnpm test test/integration/app-document-remove-hmr/test/index.test.js (turbopack)

_app removal HMR > should HMR when _app is removed

Expand output

● _app removal HMR › should HMR when _app is removed

TIMED OUT: success

<head><meta charset="utf-8" data-next-head=""><meta name="viewport" content="width=device-width" data-next-head=""><noscript data-n-css=""></noscript><script src="/_next/static/chunks/packages_next_dist_compiled_d4dc90._.js" defer=""></script><script src="/_next/static/chunks/packages_next_dist_shared_lib_cd2c78._.js" defer=""></script><script src="/_next/static/chunks/packages_next_dist_client_9e6763._.js" defer=""></script><script src="/_next/static/chunks/packages_next_dist_04dd8f._.js" defer=""></script><script src="/_next/static/chunks/packages_next_app_1d3324.js" defer=""></script><script src="/_next/static/chunks/test_integration_app-document-remove-hmr_pages__app_6c205e._.js" defer=""></script><script src="/_next/static/chunks/%5Broot%20of%20the%20server%5D__f97c22._.js" defer=""></script><script src="/_next/static/chunks/node_modules__pnpm_9f95c7._.js" defer=""></script><script src="/_next/static/chunks/test_integration_app-document-remove-hmr_pages__app_5771e1._.js" defer=""></script><script src="/_next/static/chunks/test_integration_app-document-remove-hmr_pages__app_735c88._.js" defer=""></script><script src="/_next/static/chunks/packages_next_dist_29c433._.js" defer=""></script><script src="/_next/static/chunks/test_integration_app-document-remove-hmr_pages_c8bddd._.js" defer=""></script><script src="/_next/static/chunks/%5Broot%20of%20the%20server%5D__7c0a5e._.js" defer=""></script><script src="/_next/static/chunks/node_modules__pnpm_ab5364._.js" defer=""></script><script src="/_next/static/chunks/test_integration_app-document-remove-hmr_pages_index_5771e1._.js" defer=""></script><script src="/_next/static/chunks/test_integration_app-document-remove-hmr_pages_index_3946b9._.js" defer=""></script><script src="/_next/static/development/_ssgManifest.js" defer=""></script><script src="/_next/static/development/_buildManifest.js" defer=""></script><noscript id="__next_css__DO_NOT_USE__"></noscript></head><body><p>custom _document</p><div id="__next"><p>index page updated</p></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/","query":{},"buildId":"development","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script><div id="__next-build-indicator" style="position: fixed; bottom: 10px; right: 20px; width: 0px; height: 0px; z-index: 99999;"></div><next-route-announcer><p aria-live="assertive" id="__next-route-announcer__" role="alert" style="border: 0px; clip: rect(0px, 0px, 0px, 0px); height: 1px; margin: -1px; overflow: hidden; padding: 0px; position: absolute; top: 0px; width: 1px; white-space: nowrap; overflow-wrap: normal;"></p></next-route-announcer></body>

undefined

  731 |
  732 |   if (hardError) {
> 733 |     throw new Error('TIMED OUT: ' + regex + '\n\n' + content + '\n\n' + lastErr)
      |           ^
  734 |   }
  735 |   return false
  736 | }

  at check (lib/next-test-utils.ts:733:11)
  at Object.<anonymous> (integration/app-document-remove-hmr/test/index.test.js:59:7)

Read more about building and testing Next.js in contributing.md.

pnpm test-dev-turbo test/development/acceptance-app/ReactRefreshRequire.test.ts (turbopack)

ReactRefreshRequire app > re-runs accepted modules
ReactRefreshRequire app > propagates a hot update to closest accepted module
ReactRefreshRequire app > propagates hot update to all inverse dependencies

Expand output

● ReactRefreshRequire app › re-runs accepted modules

expect(received).toEqual(expected) // deep equality

Expected: ["init FooV1", "init BarV1"]
Received: undefined

  35 |       `require('./foo'); export default function Noop() { return null; };`
  36 |     )
> 37 |     expect(await session.evaluate(() => (window as any).log)).toEqual([
     |                                                               ^
  38 |       'init FooV1',
  39 |       'init BarV1',
  40 |     ])

  at Object.toEqual (development/acceptance-app/ReactRefreshRequire.test.ts:37:63)

● ReactRefreshRequire app › propagates a hot update to closest accepted module

expect(received).toEqual(expected) // deep equality

Expected: ["init FooV1", "init BarV1"]
Received: undefined

   96 |     )
   97 |
>  98 |     expect(await session.evaluate(() => (window as any).log)).toEqual([
      |                                                               ^
   99 |       'init FooV1',
  100 |       'init BarV1',
  101 |     ])

  at Object.toEqual (development/acceptance-app/ReactRefreshRequire.test.ts:98:63)

● ReactRefreshRequire app › propagates hot update to all inverse dependencies

expect(received).toEqual(expected) // deep equality

Expected: ["init LeafV1", "init MiddleAV1", "init MiddleBV1", "init MiddleCV1", "init RootV1"]
Received: undefined

  244 |     )
  245 |
> 246 |     expect(await session.evaluate(() => (window as any).log)).toEqual([
      |                                                               ^
  247 |       'init LeafV1',
  248 |       'init MiddleAV1',
  249 |       'init MiddleBV1',

  at Object.toEqual (development/acceptance-app/ReactRefreshRequire.test.ts:246:63)

Read more about building and testing Next.js in contributing.md.

pnpm test-dev-turbo test/development/acceptance/error-recovery.test.ts (turbopack)

ReactRefreshLogBox turbo > stuck error

Expand output

● ReactRefreshLogBox turbo › stuck error

Application is in inconsistent state: timeout.

  93 |             }
  94 |             if (status !== 'pending') {
> 95 |               throw new Error(
     |                     ^
  96 |                 `Application is in inconsistent state: ${status}.`
  97 |               )
  98 |             }

  at Object.patch (lib/development-sandbox.ts:95:21)
  at Object.<anonymous> (development/acceptance/error-recovery.test.ts:361:5)

Read more about building and testing Next.js in contributing.md.

ijjk · 2025-01-18T01:33:21Z

Stats from current PR

Default Build (Increase detected ⚠️)

General

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
buildDuration	16.9s	15.4s	N/A
buildDurationCached	14.4s	12.3s	N/A
nodeModulesSize	418 MB	418 MB	N/A
nextStartRea..uration (ms)	418ms	420ms	N/A

Client Bundles (main, webpack)

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
5306-HASH.js gzip	54 kB	54 kB	N/A
8276.HASH.js gzip	169 B	168 B	N/A
8377-HASH.js gzip	5.44 kB	5.44 kB	N/A
bccd1874-HASH.js gzip	52.9 kB	52.9 kB	✓
framework-HASH.js gzip	57.5 kB	57.5 kB	N/A
main-app-HASH.js gzip	241 B	242 B	N/A
main-HASH.js gzip	34.4 kB	34.4 kB	N/A
webpack-HASH.js gzip	1.71 kB	1.71 kB	N/A
Overall change	52.9 kB	52.9 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
polyfills-HASH.js gzip	39.4 kB	39.4 kB	✓
Overall change	39.4 kB	39.4 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
_app-HASH.js gzip	193 B	193 B	✓
_error-HASH.js gzip	193 B	193 B	✓
amp-HASH.js gzip	512 B	510 B	N/A
css-HASH.js gzip	343 B	342 B	N/A
dynamic-HASH.js gzip	1.84 kB	1.84 kB	✓
edge-ssr-HASH.js gzip	265 B	265 B	✓
head-HASH.js gzip	363 B	362 B	N/A
hooks-HASH.js gzip	393 B	392 B	N/A
image-HASH.js gzip	4.57 kB	4.57 kB	N/A
index-HASH.js gzip	268 B	268 B	✓
link-HASH.js gzip	2.35 kB	2.34 kB	N/A
routerDirect..HASH.js gzip	328 B	328 B	✓
script-HASH.js gzip	397 B	397 B	✓
withRouter-HASH.js gzip	323 B	326 B	N/A
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	3.59 kB	3.59 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
_buildManifest.js gzip	749 B	747 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
index.html gzip	523 B	523 B	✓
link.html gzip	539 B	538 B	N/A
withRouter.html gzip	520 B	520 B	✓
Overall change	1.04 kB	1.04 kB	✓

Edge SSR bundle Size

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
edge-ssr.js gzip	129 kB	129 kB	N/A
page.js gzip	208 kB	208 kB	N/A
Overall change	0 B	0 B	✓

Middleware size

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
middleware-b..fest.js gzip	671 B	667 B	N/A
middleware-r..fest.js gzip	155 B	156 B	N/A
middleware.js gzip	31.3 kB	31.3 kB	N/A
edge-runtime..pack.js gzip	844 B	844 B	✓
Overall change	844 B	844 B	✓

Next Runtimes

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
274-experime...dev.js gzip	322 B	322 B	✓
274.runtime.dev.js gzip	314 B	314 B	✓
app-page-exp...dev.js gzip	374 kB	374 kB	✓
app-page-exp..prod.js gzip	130 kB	130 kB	✓
app-page-tur..prod.js gzip	143 kB	143 kB	✓
app-page-tur..prod.js gzip	139 kB	139 kB	✓
app-page.run...dev.js gzip	362 kB	362 kB	✓
app-page.run..prod.js gzip	126 kB	126 kB	✓
app-route-ex...dev.js gzip	37.6 kB	37.6 kB	✓
app-route-ex..prod.js gzip	25.6 kB	25.6 kB	✓
app-route-tu..prod.js gzip	25.6 kB	25.6 kB	✓
app-route-tu..prod.js gzip	25.4 kB	25.4 kB	✓
app-route.ru...dev.js gzip	39.2 kB	39.2 kB	✓
app-route.ru..prod.js gzip	25.4 kB	25.4 kB	✓
pages-api-tu..prod.js gzip	9.69 kB	9.69 kB	✓
pages-api.ru...dev.js gzip	11.6 kB	11.6 kB	✓
pages-api.ru..prod.js gzip	9.68 kB	9.68 kB	✓
pages-turbo...prod.js gzip	21.9 kB	21.9 kB	✓
pages.runtim...dev.js gzip	27.7 kB	27.7 kB	✓
pages.runtim..prod.js gzip	21.9 kB	21.9 kB	✓
server.runti..prod.js gzip	916 kB	916 kB	✓
Overall change	2.47 MB	2.47 MB	✓

build cache Overall increase ⚠️

	vercel/next.js canary	vercel/next.js wbinnssmith/endpoint-output-assets	Change
0.pack gzip	2.1 MB	2.1 MB	N/A
index.pack gzip	74.4 kB	76.1 kB	⚠️ +1.78 kB
Overall change	74.4 kB	76.1 kB	⚠️ +1.78 kB

Diff details

Diff for main-HASH.js

Diff too large to display

Commit: efad310

ijjk added created-by: Turbopack team PRs by the Turbopack team. Turbopack Related to Turbopack with Next.js. labels Jan 18, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove `endpoint.write_to_disk()` in favor of standardized `endpoint.output()` #75059

Remove `endpoint.write_to_disk()` in favor of standardized `endpoint.output()` #75059

wbinnssmith commented Jan 18, 2025 •

edited

Loading

wbinnssmith commented Jan 18, 2025

ijjk commented Jan 18, 2025 •

edited

Loading

ijjk commented Jan 18, 2025

Remove endpoint.write_to_disk() in favor of standardized endpoint.output() #75059

Are you sure you want to change the base?

Remove endpoint.write_to_disk() in favor of standardized endpoint.output() #75059

Conversation

wbinnssmith commented Jan 18, 2025 • edited Loading

wbinnssmith commented Jan 18, 2025

ijjk commented Jan 18, 2025 • edited Loading

Failing test suites

ijjk commented Jan 18, 2025

Stats from current PR

Remove `endpoint.write_to_disk()` in favor of standardized `endpoint.output()` #75059

Remove `endpoint.write_to_disk()` in favor of standardized `endpoint.output()` #75059

wbinnssmith commented Jan 18, 2025 •

edited

Loading

ijjk commented Jan 18, 2025 •

edited

Loading