Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add zenodo api url linting, fix broken zenodo links #4432

Merged
merged 5 commits into from
Oct 18, 2023
Merged

add zenodo api url linting, fix broken zenodo links #4432

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2882163
add zenodo api url linting
hexylena Oct 16, 2023
fad0e0d
Update the broken links
hexylena Oct 17, 2023
9ba4422
fix linting
hexylena Oct 17, 2023
ff38da9
fix heading
hexylena Oct 17, 2023
19b22b6
Merge branch 'main' into zebu-bicolor
hexylena Oct 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion bin/lint.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -684,6 +684,21 @@ def self.check_bolded_heading(contents)
end
end

def self.zenodo_api(contents)
find_matching_texts(contents, /(zenodo\.org\/api\/files\/)/)
.map do |idx, _text, selected|
ReviewDogEmitter.error(
path: @path,
idx: idx,
match_start: selected.begin(1),
match_end: selected.end(1) + 1,
replacement: nil,
message: 'The Zenodo.org/api URLs are not stable, you must use a URL of the format zenodo.org/record/..., apologies we cannot fix automatically.',
code: 'GTN:032'
)
end
end

def self.fix_md(contents)
[
*fix_notoc(contents),
Expand All @@ -707,7 +722,8 @@ def self.fix_md(contents)
*check_useless_box_prefix(contents),
*check_bad_heading_order(contents),
*check_bolded_heading(contents),
*snippets_too_close_together(contents)
*snippets_too_close_together(contents),
*zenodo_api(contents),
]
end

Expand Down
6 changes: 3 additions & 3 deletions topics/admin/tutorials/data-library/tutorial.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,18 +206,18 @@ We've included this file in the example data libraries repository we cloned at t
> description: some data
> synopsis: samples collected from somewhere
> items:
> - url: https://zenodo.org/api/files/287966da-5411-4f79-8cfb-0ffa84d0d6cc/wildtype.fna
> - url: https://zenodo.org/records/582600/files/wildtype.fna
> src: url
> ext: fasta
> info: https://doi.org/10.5281/zenodo.582600
> - name: A directory
> description: Exome sequencing means that all protein-coding genes in a genome are
> items:
> - url: https://zenodo.org/api/files/287966da-5411-4f79-8cfb-0ffa84d0d6cc/mutant_R1.fastq
> - url: https://zenodo.org/records/582600/files/mutant_R1.fastq
> src: url
> ext: fastqsanger
> info: https://doi.org/10.5281/zenodo.582600
> - url: https://zenodo.org/api/files/287966da-5411-4f79-8cfb-0ffa84d0d6cc/mutant_R2.fastq
> - url: https://zenodo.org/records/582600/files/mutant_R2.fastq
> src: url
> ext: fastqsanger
> info: https://doi.org/10.5281/zenodo.582600
Expand Down
10 changes: 5 additions & 5 deletions topics/assembly/tutorials/flye-assembly/tutorial.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: tutorial_hands_on

title: 'Genome assembly using PacBio data'
zenodo_link: 'https://zenodo.org/record/5702408#.YZUb5uvjIiU'
zenodo_link: 'https://zenodo.org/record/5702408'
tags:
- assembly
- pacbio
Expand Down Expand Up @@ -58,12 +58,12 @@ We will use long reads sequencing data: CLR (continuous long reads) from PacBio
> <hands-on-title>Data upload from Zenodo</hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo](https://zenodo.org/record/5702408)
> 2. Import the files from [Zenodo]({{ page.zenodo_link }})
>
> ```
> https://zenodo.org/api/files/d010d8f1-a1fd-4366-991f-916c2f0c55db/SRR8534473_subreads.fastq.gz
> https://zenodo.org/api/files/d010d8f1-a1fd-4366-991f-916c2f0c55db/SRR8534474_subreads.fastq.gz
> https://zenodo.org/api/files/d010d8f1-a1fd-4366-991f-916c2f0c55db/SRR8534475_subreads.fastq.gz
> https://zenodo.org/records/5702408/files/SRR8534473_subreads.fastq.gz
> https://zenodo.org/records/5702408/files/SRR8534474_subreads.fastq.gz
> https://zenodo.org/records/5702408/files/SRR8534475_subreads.fastq.gz
> ```
>
> {% snippet faqs/galaxy/datasets_import_via_link.md %}
Expand Down
2 changes: 1 addition & 1 deletion topics/contributing/tutorials/create-new-tutorial-technical/tutorial.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ items:
description: latest
items:
- info: https://doi.org/10.5281/zenodo....
url: https://zenodo.org/api/files/URL/to/the/input/file
url: https://zenodo.org/records/URL/files/path/to/input
ext: galaxy-datatype
src: url
```
Expand Down
52 changes: 26 additions & 26 deletions topics/galaxy-interface/tutorials/upload-rules/tutorial.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ redirect_from:
- /topics/galaxy-data-manipulation/tutorials/upload-rules/tutorial

title: "Rule Based Uploader"
zenodo_link: ""
zenodo_link: "https://zenodo.org/records/3263975"
level: Intermediate
questions:
- "How to use the rule based uploader to create complex collections"
Expand Down Expand Up @@ -86,12 +86,12 @@ You can select all the data below and copy them to your clipboard.

```
study_accession sample_accession experiment_accession fastq_ftp
PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000770.fastqsanger.gz
PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000771.fastqsanger.gz
PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000772.fastqsanger.gz
PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000773.fastqsanger.gz
PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000774.fastqsanger.gz
PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000775.fastqsanger.gz
PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/records/3263975/files/DRR000770.fastqsanger.gz
PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/records/3263975/files/DRR000771.fastqsanger.gz
PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/records/3263975/files/DRR000772.fastqsanger.gz
PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/records/3263975/files/DRR000773.fastqsanger.gz
PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/records/3263975/files/DRR000774.fastqsanger.gz
PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/records/3263975/files/DRR000775.fastqsanger.gz
```
{:#example-1-metadata}

Expand All @@ -111,12 +111,12 @@ PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/api/files/c07c0fbd-c578-4c8
>
> ```
> study_accession sample_accession experiment_accession fastq_ftp
> PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000770.fastqsanger.gz
> PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000771.fastqsanger.gz
> PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000772.fastqsanger.gz
> PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000773.fastqsanger.gz
> PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000774.fastqsanger.gz
> PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000775.fastqsanger.gz
> PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/records/3263975/files/DRR000770.fastqsanger.gz
> PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/records/3263975/files/DRR000771.fastqsanger.gz
> PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/records/3263975/files/DRR000772.fastqsanger.gz
> PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/records/3263975/files/DRR000773.fastqsanger.gz
> PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/records/3263975/files/DRR000774.fastqsanger.gz
> PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/records/3263975/files/DRR000775.fastqsanger.gz
> ```
>
> ![screenshot](../../images/rules/rules_example_1_2_paste.png)
Expand Down Expand Up @@ -191,12 +191,12 @@ This example will demonstrate using such history datasets as the source for coll
>
> ```
> study_accession sample_accession experiment_accession fastq_ftp
> PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000770.fastqsanger.gz
> PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000771.fastqsanger.gz
> PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000772.fastqsanger.gz
> PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000773.fastqsanger.gz
> PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000774.fastqsanger.gz
> PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000775.fastqsanger.gz
> PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/records/3263975/files/DRR000770.fastqsanger.gz
> PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/records/3263975/files/DRR000771.fastqsanger.gz
> PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/records/3263975/files/DRR000772.fastqsanger.gz
> PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/records/3263975/files/DRR000773.fastqsanger.gz
> PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/records/3263975/files/DRR000774.fastqsanger.gz
> PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/records/3263975/files/DRR000775.fastqsanger.gz
> ```
>
> 2. **Open** the `Rule-based` upload tab again, but this time:
Expand Down Expand Up @@ -248,13 +248,13 @@ For this next example we will again use ENA data, this time corresponding to the
>
> ```
> study_accession sample_accession experiment_accession fastq_ftp
> PRJDB3920 SAMD00034150 DRX036147 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036147_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036147_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036148 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036148_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036148_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036149 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036149_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036149_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036150 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036150_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036150_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036151 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036151_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036151_2.fastq.gz
> PRJDB3920 SAMD00034153 DRX036152 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036152_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036152_2.fastq.gz
> PRJDB3920 SAMD00034152 DRX036164 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036164_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036164_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036147 https://zenodo.org/records/3263975/files/DRX036147_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036147_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036148 https://zenodo.org/records/3263975/files/DRX036148_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036148_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036149 https://zenodo.org/records/3263975/files/DRX036149_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036149_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036150 https://zenodo.org/records/3263975/files/DRX036150_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036150_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036151 https://zenodo.org/records/3263975/files/DRX036151_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036151_2.fastq.gz
> PRJDB3920 SAMD00034153 DRX036152 https://zenodo.org/records/3263975/files/DRX036152_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036152_2.fastq.gz
> PRJDB3920 SAMD00034152 DRX036164 https://zenodo.org/records/3263975/files/DRX036164_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036164_2.fastq.gz
> ```
>
>
Expand Down
6 changes: 3 additions & 3 deletions topics/genome-annotation/tutorials/apollo-euk/tutorial.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,9 +110,9 @@ In this tutorial we use the same data as in the [Funannotate](../funannotate/tut
> -> `{{ page.title }}`):
>
> ```
> https://zenodo.org/api/files/042ff43f-cd53-45ec-90cf-781fe53e3771/genome.fasta
> https://zenodo.org/api/files/042ff43f-cd53-45ec-90cf-781fe53e3771/annotation.gff3
> https://zenodo.org/api/files/042ff43f-cd53-45ec-90cf-781fe53e3771/rnaseq.bam
> https://zenodo.org/record/3270822/files/genome.fasta
> https://zenodo.org/record/3270822/files/annotation.gff3
> https://zenodo.org/record/3270822/files/rnaseq.bam
> ```
>
> {% snippet faqs/galaxy/datasets_import_via_link.md %}
Expand Down
32 changes: 16 additions & 16 deletions topics/genome-annotation/tutorials/apollo/tutorial.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: tutorial_hands_on

title: Refining Genome Annotations with Apollo (prokaryotes)
zenodo_link: https://zenodo.org/record/3270822
zenodo_link: https://zenodo.org/record/4889110
tags:
- prokaryote
questions:
Expand Down Expand Up @@ -106,21 +106,21 @@ In this tutorial we have obtained some data from NCBI related to [*Escherichia c
> 3. Copy & Paste the following table into the Rule-based uploader textbox:
>
> ```
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/augustus.gff3 Augustus gff3
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/blastp_vs_swissprot_2018-01-22.blastxml Blastp vs swissprot blastxml
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/BWA-MEM_K12_Coverage.bigwig BWA-MEM K12 Coverage bigwig
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/BWA-MEM_K12_Mapping.bam BWA-MEM K12 Mapping bam
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/BWA-MEM_O104_Coverage.bigwig BWA-MEM O104 Coverage bigwig
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/BWA-MEM_O104_Mapping.bam BWA-MEM O104 Mapping bam
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/E._coli_str_K-12_substr_MG1655_100kb_subset.fasta Genome fasta
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/K12_Variants.vcf K12 Variants vcf
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/NCBI_AnnotWriter_Genes.gff3 NCBI AnnotWriter Genes gff3
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/O104_H4_LASTZ_Alignment.bed O104 H4 LASTZ Alignment bed
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/O104_Variants.vcf O104 Variants vcf
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/TopHat_SRR1927169_rep1.bam TopHat SRR1927169 rep1 bam
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/TopHat_SRR1927169_rep1_Coverage.bigwig TopHat SRR1927169 rep1 Coverage bigwig
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/TopHat_SRR1927170_rep2.bam TopHat SRR1927170 rep2 bam
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/TopHat_SRR1927170_rep2_Coverage.bigwig TopHat SRR1927170 rep2 Coverage bigwig
> https://zenodo.org/records/4889110/files/augustus.gff3 Augustus gff3
> https://zenodo.org/records/4889110/files/blastp_vs_swissprot_2018-01-22.blastxml Blastp vs swissprot blastxml
> https://zenodo.org/records/4889110/files/BWA-MEM_K12_Coverage.bigwig BWA-MEM K12 Coverage bigwig
> https://zenodo.org/records/4889110/files/BWA-MEM_K12_Mapping.bam BWA-MEM K12 Mapping bam
> https://zenodo.org/records/4889110/files/BWA-MEM_O104_Coverage.bigwig BWA-MEM O104 Coverage bigwig
> https://zenodo.org/records/4889110/files/BWA-MEM_O104_Mapping.bam BWA-MEM O104 Mapping bam
> https://zenodo.org/records/4889110/files/E._coli_str_K-12_substr_MG1655_100kb_subset.fasta Genome fasta
> https://zenodo.org/records/4889110/files/K12_Variants.vcf K12 Variants vcf
> https://zenodo.org/records/4889110/files/NCBI_AnnotWriter_Genes.gff3 NCBI AnnotWriter Genes gff3
> https://zenodo.org/records/4889110/files/O104_H4_LASTZ_Alignment.bed O104 H4 LASTZ Alignment bed
> https://zenodo.org/records/4889110/files/O104_Variants.vcf O104 Variants vcf
> https://zenodo.org/records/4889110/files/TopHat_SRR1927169_rep1.bam TopHat SRR1927169 rep1 bam
> https://zenodo.org/records/4889110/files/TopHat_SRR1927169_rep1_Coverage.bigwig TopHat SRR1927169 rep1 Coverage bigwig
> https://zenodo.org/records/4889110/files/TopHat_SRR1927170_rep2.bam TopHat SRR1927170 rep2 bam
> https://zenodo.org/records/4889110/files/TopHat_SRR1927170_rep2_Coverage.bigwig TopHat SRR1927170 rep2 Coverage bigwig
> ```
> 4. Click **Build**
>
Expand Down
Loading