Update Anndata to 0.10.9 #6309

pavanvidem · 2024-09-06T14:12:37Z

FOR CONTRIBUTOR:

I have read the CONTRIBUTING.md document and this tool is appropriate for the tools-iuc repo.
License permits unrestricted use (educational + commercial)
This PR adds a new tool or tool collection
This PR updates an existing tool or tool collection
This PR does something else (explain below)

This update adds some new user-requested features from #6025

bgruening · 2024-09-06T14:49:02Z

If you have time you should really change the tests from sim_size to some better more strict asserts.

bgruening · 2024-09-06T14:46:17Z

tools/anndata/macros.xml

    <token name="@VERSION_SUFFIX@">0</token>
    <xml name="requirements">
        <requirements>
            <requirement type="package" version="@TOOL_VERSION@">anndata</requirement>
            <requirement type="package" version="3.0.6">loompy</requirement>
+            <requirement type="package" version="1.26.4">numpy</requirement>


Should the anndata package be fixed instead? https://github.com/conda-forge/anndata-feedstock/blob/main/recipe/meta.yaml#L28

Turned out it was issue was only with the loompy package. I moved all the loompy functions to a modify_loom tool and added the numpy requirement there so that all the anndata functions can use numpy 2.

tools/anndata/manipulate.xml

bgruening · 2024-09-06T14:47:18Z

tools/anndata/manipulate.xml

+            <when value="flag_genes">
+                <repeat name="gene_flags" title="Flag genes that start with these names">
+                    <param name="startswith" type="text" label="Text that you expect the genes to be flagged to start with" help="for example, 'MT-' for mito genes"/>
+                    <param name="col_name" type="text" label="Name of the column in var.names where this boolean flag is stored" help="for example, name this column as 'mito' for mitochondrial genes."/>


Suggested change

<param name="col_name" type="text" label="Name of the column in var.names where this boolean flag is stored" help="for example, name this column as 'mito' for mitochondrial genes."/>

<param name="col_name" type="text" label="Name of the column in var.names where this boolean flag is stored" help="For example, name this column as 'mito' for mitochondrial genes."/>

bgruening · 2024-09-06T14:47:39Z

tools/anndata/manipulate.xml

+            </when>
+            <when value="flag_genes">
+                <repeat name="gene_flags" title="Flag genes that start with these names">
+                    <param name="startswith" type="text" label="Text that you expect the genes to be flagged to start with" help="for example, 'MT-' for mito genes"/>


Suggested change

<param name="startswith" type="text" label="Text that you expect the genes to be flagged to start with" help="for example, 'MT-' for mito genes"/>

<param name="startswith" type="text" label="Text that you expect the genes to be flagged to start with" help="For example, 'MT-' for mito genes"/>

Should we add a validator here ... eg. no whitespace?

pavanvidem · 2024-09-14T08:07:20Z

If you have time you should really change the tests from sim_size to some better more strict asserts.

Done for all h5 files

bgruening · 2024-09-14T12:12:35Z

tools/anndata/import.xml

+#if $in.adata_format == 'mtx'
+    && rm -rf mtx
 #end if


Suggested change

#if $in.adata_format == 'mtx'

&& rm -rf mtx

#end if

Galaxy will clean-up, you don't need to do that.

bgruening · 2024-09-14T12:14:18Z

tools/anndata/import.xml

 #end if
+adata.write('anndata.h5ad', compression='gzip')
+print(adata)


Suggested change

print(adata)

print is needed here. I assert the matrix dimensions in the stdout.

bgruening · 2024-09-14T12:15:05Z

tools/anndata/import.xml

-                    <when value="umi_tools">
-                        <param name="input" type="data" format="tabular" label="condensed count matrix from UMI tools"/>
+                    <when value="v3_10x">
+                        <param name="features" type="data" format="tabular" label="Features"/>


Please add to help, how does such a file need to look like?

bgruening · 2024-09-14T12:16:19Z

tools/anndata/import.xml

            </assert_stdout>
-            <output name="anndata" value="import.tsv.h5ad" ftype="h5ad" compare="sim_size"/>


you are not testing the output file?

there are no annotations in the anndata to assert h5 keys. That is why I added the print(data) so that we can at least compare the matrix dimensions with assert_stdout.

bgruening · 2024-09-14T12:16:36Z

tools/anndata/import.xml

            </assert_stdout>
-            <output name="anndata" value="import.csv.h5ad" ftype="h5ad" compare="sim_size"/>


You are not testing the output file?

bgruening · 2024-09-14T12:16:49Z

tools/anndata/import.xml

@@ -226,8 +204,10 @@ adata.write('anndata.h5ad')
                <conditional name="tenx">
                    <param name="use" value="no"/>
                </conditional>
-                </conditional>
-            <output name="anndata" value="import.mtx.no_10x.h5ad" ftype="h5ad" compare="sim_size"/>


Some here ...

bgruening · 2024-09-14T12:18:04Z

tools/anndata/manipulate.xml

-adata.write('anndata.h5ad')
+#if $manipulate.function != 'split_on_obs'
+adata.write('anndata.h5ad', compression='gzip')
+print(adata)


Suggested change

print(adata)

bgruening · 2024-09-14T12:21:50Z

tools/anndata/export.xml

@@ -1,90 +1,33 @@
-<tool id="anndata_export" name="Export AnnData and loom files" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@">
-    <description>Interconvert AnnData and Loom formats</description>
+<tool id="anndata_export" name="Export AnnData" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@">


please add, make sure, a profile version is in all tools

Suggested change

<tool id="anndata_export" name="Export AnnData" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@">

<tool id="anndata_export" name="Export AnnData" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@PROFILE_VERSION@">

set profile to 21.09 (same as scanpy) to support other instances. See here: #6001

bgruening · 2024-09-14T19:24:42Z

Thanks @pavanvidem! Good luck with the trainings!

mvdbeek · 2024-09-14T19:36:02Z

Attention: deployment skipped!

https://github.com/galaxyproject/tools-iuc/actions/runs/10864948585

Update Anndata to 0.10.9

fc03c16

bgruening reviewed Sep 6, 2024

View reviewed changes

pavanvidem added 12 commits September 9, 2024 11:20

replace h5ad size comparison with h5 asserts

4de98de

fix linting and remove h5ad test data

e0a651c

add tests for newly added functions

0bf651c

add gzip compression for all h5ad files

9a531cc

add help text about newly added functions and update tests

0343a4c

fix tests

bd992d4

remove flagged.h5ad

fe812e7

move all loompy functions to modify_loom

011951c

trying to fix tests

53470b2

fix anndata to loom

48b01d4

fix

30a636f

move all loom functions to modify_loom.xml and fix tests

959a8d1

pavanvidem marked this pull request as ready for review September 13, 2024 20:01

change output loom name to fix the test

4196c66

re-add all the tests to modify_loom

320fb61

bgruening reviewed Sep 14, 2024

View reviewed changes

pavanvidem added 2 commits September 14, 2024 15:27

review suggestions

0b4f26b

fix the order of elements in collection

953f226

bgruening merged commit 67b3808 into galaxyproject:main Sep 14, 2024
14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Anndata to 0.10.9 #6309

Update Anndata to 0.10.9 #6309

pavanvidem commented Sep 6, 2024

bgruening commented Sep 6, 2024

bgruening Sep 6, 2024

pavanvidem Sep 14, 2024 •

edited

Loading

bgruening Sep 6, 2024

bgruening Sep 6, 2024

bgruening Sep 6, 2024

pavanvidem Sep 14, 2024

pavanvidem commented Sep 14, 2024

bgruening Sep 14, 2024

bgruening Sep 14, 2024

pavanvidem Sep 14, 2024 •

edited

Loading

bgruening Sep 14, 2024

bgruening Sep 14, 2024

pavanvidem Sep 14, 2024

bgruening Sep 14, 2024

bgruening Sep 14, 2024

bgruening Sep 14, 2024

bgruening Sep 14, 2024

pavanvidem Sep 14, 2024

bgruening commented Sep 14, 2024

mvdbeek commented Sep 14, 2024

	<param name="col_name" type="text" label="Name of the column in var.names where this boolean flag is stored" help="for example, name this column as 'mito' for mitochondrial genes."/>
	<param name="col_name" type="text" label="Name of the column in var.names where this boolean flag is stored" help="For example, name this column as 'mito' for mitochondrial genes."/>

	<param name="startswith" type="text" label="Text that you expect the genes to be flagged to start with" help="for example, 'MT-' for mito genes"/>
	<param name="startswith" type="text" label="Text that you expect the genes to be flagged to start with" help="For example, 'MT-' for mito genes"/>

		</assert_stdout>
		<output name="anndata" value="import.tsv.h5ad" ftype="h5ad" compare="sim_size"/>

		</assert_stdout>
		<output name="anndata" value="import.csv.h5ad" ftype="h5ad" compare="sim_size"/>

	<tool id="anndata_export" name="Export AnnData" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@">
	<tool id="anndata_export" name="Export AnnData" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@PROFILE_VERSION@">

Update Anndata to 0.10.9 #6309

Update Anndata to 0.10.9 #6309

Conversation

pavanvidem commented Sep 6, 2024

bgruening commented Sep 6, 2024

Choose a reason for hiding this comment

pavanvidem Sep 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pavanvidem commented Sep 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pavanvidem Sep 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bgruening commented Sep 14, 2024

mvdbeek commented Sep 14, 2024

pavanvidem Sep 14, 2024 •

edited

Loading

pavanvidem Sep 14, 2024 •

edited

Loading