[BH2024] Update Bioimaging Archive retrieval tool

[B0r1sD]

Related to Project 17: Development of FAIR image analysis workflows & training in Galaxy
Fixes beatrizserrano/galaxy-image-community#11

[kostrykin]

By the way if you write

Fix beatrizserrano/galaxy-image-community#11

into the text of your PR, it will be linked automatically by GitHub :)

[B0r1sD]

Unfortunately, I discovered even the Bioimage -, Core accession numbers do not follow one clear pattern for their FTP link. See ftp://ftp.ebi.ac.uk/biostudies/fire/S-BIAD/S-BIAD0-99/S-BIAD24 that is not following the S-BIAD/000/S-BIAD000 pattern that was assumed previously. IMO this tool should just ask in one line for the whole ftp link that users can find in the 'ftp' button in the study they want in their Galaxy history. This way it will be correct all the time and usable for the whole BIA.
I can quickly prune this wrapper to do exactly that (as the previous version would just fail on these 'exceptions') and then focus on getting a Galaxy button on BIA.

[B0r1sD]

The tool also keeps failing on the datatype .lif which is in a lot of BIA studies. The tool fetches whole studies atm so not only the images (hence the name is a bit misleading?).

Leia Image Format (LIF) images are JPEG images with metadata

If we want the Galaxy button in BIA, I think it is a datatype that should be added to Galaxy. But I've heard this is quite cumbersome for the developers and I also have no idea if there are more datatypes in a similar siutation.

We could introduce a filter to just download .tif/.tiff from studies but that probably drops metadata from the studies. I am leaning towards not spending more time on this refactoring as it exposes too many flaws - the proper solution (the Galaxy button in BIA) has been initatiated but there is no estimation on when it will be there so this tool has to do for now.
Maybe just updating the help text with the explanation of this is enough for now?... Let me know what you think.

[kostrykin]

Thanks so much for all your efforts, @B0r1sD.

Your points are perfectly reasonable. Asking the user for the whole FTP link sounds good. On the other hand, if the user already has the FTP link, couldn't the user then just use Galaxy's "Upload data" button and fetch the data via FTP from there? So would there be any point in using this tool? As far as I remember from talking to @sunyi000, the whole point of the tool was to get those FTP links, but maybe I'm missing something.

Personally, I think, we should generally consider one step after another. It will be a great improvement when the Galaxy button in BIA will be there. With the experiences and feedback we will hopefully gather then, we might consider adding support for further datatypes, LIF or beyond, but first we need the experiences in how far it's actually required.

Maybe just updating the help text with the explanation of this is enough for now?

Agreed. I'd be interested in the stance of @sunyi000 who originally wrote this tool.

[B0r1sD]

Exactly @kostrykin, but I think the original author didn't anticipate the heterogenity of the accession paths as most of them don't follow the expected pattern that would be scriptable. So the best course of action to keep this tool is to clearly state what it can and can not do.
Unfortunately, I've just received some feedback from Teresa at BIA about that button:

Thanks so much for sending us such a detailed explanation of the implementation. Unfortunately, this would be non-trivial for us because the implementation would need to be done by another EBI service (BioStudies) whose backend we use. We are working on a BIA image-centric API but it’s not quite ready yet. For the moment, it would be easier if those using Galaxy either use the BioStudies API, FTP, or direct HTTP links.

I believe that was the (alpha) API we discussed was not yet ready to handle data requests, but I could be wrong? As for the metadata, it does seems like it would follow a clear logical of https://www.ebi.ac.uk/biostudies/api/v1/studies/ + accession_number e.g. https://www.ebi.ac.uk/biostudies/api/v1/studies/S-BIAD24/ or https://www.ebi.ac.uk/biostudies/api/v1/studies/S-JCBD-201709074.

So I guess we wait on their BIA API they are working on, and in the meanwhile address the tool as written above. Input from original author is of course always welcome!