Support checksum in AssetDownloadManager

[cbalioglu]

As of today, the InProcAssetDownloadManager does not have any checks to make sure that downloaded assets (e.g. model checkpoints, datasets, tokenizers) have not been tampered with. Ideally we would like to have an optional "checksum" field in asset cards that the download manager can use to verify the integrity of a downloaded file.

from fairseq2.assets import default_asset_store, InProcDownloadManager

card = default_asset_store.retrieve_card("mistral_7b")

checkpoint_uri = card.field("checkpoint").as_uri()

# We should check if the card has a checksum field; if not, `get_as_` returns None.
checksum = card.field("checksum").get_as_(str)

download_manager = InProcDownloadManager()

download_manager.download_checkpoint(checkpoint_uri, model_name="mistral")

# The API of the download manager should be extended with an additional "checksum" parameter.
# download_manager.download_checkpoint(checkpoint_uri, checksum, model_name="mistral")