Investigate adding FEC

[hamishcoleman]

FEC (Forward Error Correction) is a way to increase reliability when some data is lost.

If n3n users are experiencing packet loss, the addition of FEC could assist with that. However, FEC is not a simple cure-all - it has a cost of increased complexity and increased packet sizes (or increased number of packets - or probably both) and increased memory use (for caching the all the packets in a bundle before reassembly) and increased latency (often, a FEC algo cannot return the original packets until all the packets in the bundle have arrived). All this has meant that in most expected internet protocol situations, a simple packet retransmit is more reliable than the total overhead of adding FEC.

Since the internet is intended to be a low-loss system, before adding something like FEC, we should investigate what is causing these packet losses, and how to flexibly handle that.

For instance, the FEC convolution algorithm used is tuned to the expected percentage data loss - so either a flexible suite of algos is needed, or an acceptable failure rate determined.

• How often are packet losses occurring?
• Do we have metrics to capture this?
• Can we find a protocol edge case that is causing drops?
• Do we know what percentage loss is expected?
• Can we detect this loss at runtime?
• Are there some simple FEC algos that we can add?
• What are the tradeoffs inherent in any algos proposed to add?

[eebssk1]

Since the internet is intended to be a low-loss system, before adding something like FEC, we should investigate what is causing these packet losses, and how to flexibly handle that.
This feature might be interested to mobile users, ISPs that do random drop for udp packets/lossy backbone interconnect.

Currently the two widely used tunnel are both based on packet redundancy and busrting.
How KCP works is clearly described in its readme.
While Hysteria is just bursting more packet on loss(up to bandwidth limit) to battle against ISPs, which does not suit for n2n obviously. See (Chinese) https://v2.hysteria.network/zh/docs/misc/Hysteria-Brutal/ or https://github.com/apernet/tcp-brutal.

[Logan007]

Few thoughts:

I think we would need packet numbers, just as we hacked into header encryption. First, to keep packets in order (for proper check-summing as well as detecting loss or other issues). Second, to get a basis for metrics.

We could try to limit it to a reasonable number (we would need to anyway because buffer space might be limited) and to hack it somehow into existing protocol fields (no idea yet) to maintain compatibility. Do we have a nibble available in flags?

Then, an additional packet type "CHECK" might be required to be sent after every other n packets (with n being dynamically determined according to check results). It would include the checksum/convolution data and perhaps some control information (like "new n" or "my n and your n" or such).

To keep things easy in the beginning, we should only include data PACKET type?

[eebssk1]

Few thoughts:

I think we would need packet numbers, just as we hacked into header encryption. First, to keep packets in order (for proper check-summing as well as detecting loss or other issues). Second, to get a basis for metrics.

We could try to limit it to a reasonable number (we would need to anyway because buffer space might be limited) and to hack it somehow into existing protocol fields (no idea yet) to maintain compatibility. Do we have a nibble available in flags?

Then, an additional packet type "CHECK" might be required to be sent after every other n packets (with n being dynamically determined according to check results). It would include the checksum/convolution data and perhaps some control information (like "new n" or "my n and your n" or such).

To keep things easy in the beginning, we should only include data PACKET type?

Does this approach add latency between packets? Since latency is critical in some applications like VoIP,might need to keep it as low as possible(or as a configurable option?).
Also might not try error correction too many times or too late,as this might make audio in VoIP more weird than without correction(unless the software already have some sort of compensation).

[Logan007]

Does this approach add latency between packets?

Depends... not necessarily though. If you only use it to detect packet loss, it will not add any delay.

And if you use it to make up for packet loss, you will get delay because we will have to wait a bit to determine if a packet loss occurred (to take action) or not. That requires also time measurements at sender (to make sure to send out checksum packet at stream's end even if not a multiple of n) as well as at receiving end.

So, implementation could go in steps. First just a detection step (to inform user about possible packet loss) before going into correction.

How about using upper nibble of TTL (or version field) for a 4 bit packet number, used in circular way 0 ... F, 0 ... F along with a corresponding 16 times max-packet-size sized buffer (per known peer)?

[eebssk1]

For reference.
This is a udp to udp FEC relay.
https://github.com/wangyu-/UDPspeeder
You can run it to relay the packet between supernode.
Suitable for those who setup supernode on oversea/not_same_region,and only pSp is possible.