Improve speed of check-for-updates

[edsko]

This involves:

• Building the tar index incrementally (Rebuild tar-index incrementally #22)
• Avoiding building a tar index in cabal, and instead use the one from the hackage-security library (Use hackage-security library to extract .cabal files from the index #29)
• Make sure we can do compress our range requests (Compression for range requests? #101)
• Sort out issues of range requests and the CDN (Range requests interact badly with the CDN #104, and also checkForUpdates interacts badly with the CDN #106)

In an ideal scenario, we would avoid reading the existing tarfile altogether, and literally just append to it; after all, even just copying a 200 MB file is not immediate. This would also mean we would need to

• Compute the hash incrementally
• Give up on updating the file atomically (and might therefore need to introduce some kind of locking policy, perhaps with an auxiliary file).

[edsko]

Issues with the CDN are outside the scope of the security project proper. Have opened separate ticket about incrementally computing the hash (#130), but it's not a priority for now. Closing this.