Remote play

[fengb]

We need a way to drive a web interface independently from the Fundude core. Ideally this would happen transparently — e.g. only exists when requesting serial connection.

UX architecture

1. User boots up a game (Tetris)
2. Request remote connect to host
3. Game state transfers to host machine. Local “game” is a view into remote state.
4. Upon request disconnect, send state back to local.

Possible solutions

1. Reuse the binary for transferring states: Save/load state #41
2. Standardized data interface between web and core.
   • How do we trigger this transparently? I think the quick solution is to copy the state on every sync.
   • How should debug data (e.g. video cache) be transferred?

Background

The serial port is a synchronous device that requires the remote data to exist. Based on some quick hacks, I’ve discovered that ROM programming expects data to be available immediately, possibly some wiggle room of ~10ms, but definitely not going to work across browser latency (1-2 frames ~30ms), let alone actual network latency (100+ ms).

This means getting serial link to work properly requires the Fundude instances to be synced together. This is only really feasible if both are running in the same wasm instance... which means we need to somehow drive the web interface separately from the core.

[fengb]

Potential new design:

1. Server and client each will run a copy per game state
2. Client inputs directly update local state and sends inputs to the server
3. For client inputs, server rewinds time to match input time
4. Whenever server has input updates, it will send a binary copy to the client
5. Whenever client receives a binary update, it'll restore and fast forward to "now"

This seems pretty complicated, but it relies on 2 simple ideas:

1. User inputs rarely changed
2. Game state only diverges from user inputs

Separate execution instances is needed for low latency, while rewinding time should enable seamless reconciliation. And this makes frontend integration pretty easy: it's already done!

Thanks to @marijnfs for the inspiration on Youtube comments.