Roadmap

[dgalaktionov]

According to what I've read, one of the top priorities for ST is to replace the network library. If so, it would be useful to define the features that are still missing for a minimum viable product so potential contributors may know where to help.

So here I'll detail a draft roadmap, but I'm by no means an expert in videogame networking and have a very limited understanding of how the communication in ST works right now, so I've probably made some wrong assumptions about how ST works and what do you need. I'd appreciate if one of the devs gave me feedback so we can replace the old lib ASAP.

Server and Client

There is already a Server class but it would be nice to make it a bit more complete and easy to extend, with some customizable listeners. At least these three:

• OnClientConnected, which is called when some new connection appears and gets past the protocol negotiation and whatnot.
• OnPacketReceived, which is where we expect to get the actual data, with a valid header (see below).
• OnClientDisconnected, called when the client is not among us anymore, either because of disconnection or an error, such as a timeout.

Then there would be a Client, with at least the equivalent of the first two callbacks. The connection negotiation would be done in the default implementation, for convenience.

Header

Is it correct to assume that the header defined in Connection.h is going to be used just for the initial negotiation? For the actual data packets I'd propose a minimalist header similar to other libs:

|  SEQ (4 bytes)   |   ACK (4 bytes)  |

Just like in TCP, SEQ is a packet ID that we increment and ACK is the last SEQ we received from the other side. I think those two are important as they would allow us to implement reliable messages and save bandwidth by avoiding sending duplicate data (see below).

Game state

I would like to think that ST has some state to synchronize between players, such as players position, orientation, equipped items, current animation... Normally the server simulates the game and the only thing the clients have to send are their commands, but this doesn't appear to be the case, so the clients must send their own state and the server must echo it. I'm guessing that the cell host also has to send NPC and item states.

I really like how it is managed by the Quake III snapshot system and I think it may work in our case too, including the deltas and the "introspection". We would have a base Entity class with an owner attribute (so we avoid sending echoing back its own snapshot to a player) with a serialization and deserialization function. The server would keep the 64 last sent snapshots of each entity and send the delta to each client corresponding to their ACK. To make matters simpler, the server would have one global SEQ and increase it only after sending a packet to each client. That way we don't have to have to keep 64 snapshots per entity per client.

Of course we'd be using the allocators to implement some entity pool, with the "free list" trick. My guess is that we'd need several pools as some entities will be smaller than others (players vs items), and some may be very short lived (arrows?).

Not sure how to handle the removal of entities yet. I'm leaned to approach it with bitmaps but maybe there is a better solution I'm not seeing due to my professional bias :-)

Data types

So a packet will contain the header and then a bunch of entities and maybe other messages, each one preceded by a header of 8 bits or however much we need. I propose to start with the following serialization convenience types/functions:

• Fixed-length integer of n bits, with parametrizable n.
• Variable-length integer, either elias delta codes or VByte (don't have the link, but it's basically UTF8 for numbers) for integers that are expected to be small most of the time.
• Floats of 16 and 32 bits.
• Strings for player names?

Reliable messages

I'm guessing that you have some messages that do not represent state but we want to be sure they make it, such as when a player disconnects or changes cell. Do you also need to guarantee the order of these messages? Because if not we can just have an "important messages" queue per client, try to send them in the next packet and keep trying until we get an ACK.

Bandwidth considerations

As much as possible, we want to make sure we don't send duplicated data to a client. From the client perspective, they would only have to send updates on the entities they own, which would mean that they'd probably end up using less upstream than downstream bandwidth, which is good (except for the cell host, which will have to suck up a lot of upstream).

Rate limiting on both the server and client sounds reasonable. Is there also a simple way in ST to know if something is happening to the player (like if they move or lose health)? Because in that case we could do with a low rate for clients but send extra packets whenever something happens (as with input commands in other games).

Memory considerations

We should get a rough estimation of how much preallocating 64 (or any reasonable number of) snapshots for every possible player and entity at the start of the server would cost us in MB. If it's prohibitive then we'd have to look for a more dynamic solution...

[maximegmd]

Hey,

Regarding headers, we will probably go for something similar to what is done in netcode.io and reliable.io with a bitfield to ACK 32 packets in a single header (you can check the dev's website at https://gafferongames.com/ ).

Game states and data types are not in the scope of this library, this is just a generic game netcode library to allow clients to send arbitrary payload of any size, reliable or unreliable. We have a custom codegen system for packets but are currently in the process of moving to protobuf.

Rate limiting is already implemented and we only send what changes.

[dgalaktionov]

Wow good thing I didn't jump to implementing all those features we don't need :-)

So that means it's almost ready to be used? Because server and client abstractions with extendable listeners are just a convenience.

On a side note: I have been trying to access https://gafferongames.com/ for a couple of days now because it had some interesting resources but it looks down every time...

[wopss]

Use the archive (https://web.archive.org/web/*/https://gafferongames.com/).