Installation and download of sample data problem

[pktrigg]

Good Morning,
I have pulled ginan from git and am running through the compile and setup process you lay out in the readme.md file.

I am at the point of downloading the sameple data from AWS and have bumped into a problem with lots of missing files. Is it possible to get this sorted out so I can complete the installation and setup?

with the 404 I dont really have a lot of choice other than request the files be put into the correct folder:-)

I get many hundreds of these...

`guardian@GG-DT-07:/mnt/c/ggtools/ginan/ginan/inputData/data$ ./getData.sh
Reading from file list: data.list
Downloading from server: https://peanpod.s3.ap-southeast-2.amazonaws.com/aux/data
from https://peanpod.s3.ap-southeast-2.amazonaws.com/aux/data
--2024-07-17 12:17:12-- https://peanpod.s3.ap-southeast-2.amazonaws.com/aux/data/AGGO00ARG_R_20191990000_01D_30S_MO.rnx%0D.gz
Resolving peanpod.s3.ap-southeast-2.amazonaws.com (peanpod.s3.ap-southeast-2.amazonaws.com)... 3.5.168.14, 52.95.129.230, 52.95.130.134, ...
Connecting to peanpod.s3.ap-southeast-2.amazonaws.com (peanpod.s3.ap-southeast-2.amazonaws.com)|3.5.168.14|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-07-17 12:17:12 ERROR 404: Not Found.

.gz: No such file or directory01D_30S_MO.rnx
from https://peanpod.s3.ap-southeast-2.amazonaws.com/aux/data
--2024-07-17 12:17:12-- https://peanpod.s3.ap-southeast-2.amazonaws.com/aux/data/ALIC00AUS_R_20191990000_01D_30S_MO.rnx%0D.gz
Resolving peanpod.s3.ap-southeast-2.amazonaws.com (peanpod.s3.ap-southeast-2.amazonaws.com)... 3.5.168.14, 52.95.129.230, 52.95.130.134, ...
Connecting to peanpod.s3.ap-southeast-2.amazonaws.com (peanpod.s3.ap-southeast-2.amazonaws.com)|3.5.168.14|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-07-17 12:17:13 ERROR 404: Not Found.

.gz: No such file or directory01D_30S_MO.rnx
from https://peanpod.s3.ap-southeast-2.amazonaws.com/aux/data
--2024-07-17 12:17:13-- https://peanpod.s3.ap-southeast-2.amazonaws.com/aux/data/ANKR00TUR_R_20191990000_01D_30S_MO.rnx%0D.gz
Resolving peanpod.s3.ap-southeast-2.amazonaws.com (peanpod.s3.ap-southeast-2.amazonaws.com)... 3.5.168.14, 52.95.129.230, 52.95.130.134, ...
Connecting to peanpod.s3.ap-southeast-2.amazonaws.com (peanpod.s3.ap-southeast-2.amazonaws.com)|3.5.168.14|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-07-17 12:17:13 ERROR 404: Not Found.

.gz: No such file or directory01D_30S_MO.rnx
from https://peanpod.s3.ap-southeast-2.amazonaws.com/aux/data
--2024-07-17 12:17:13-- https://peanpod.s3.ap-southeast-2.amazonaws.com/aux/data/AREG00PER_R_20191990000_01D_30S_MO.rnx%0D.gz
Resolving peanpod.s3.ap-southeast-2.amazonaws.com (peanpod.s3.ap-southeast-2.amazonaws.com)... 3.5.168.14, 52.95.129.230, 52.95.130.134, ...
Connecting to peanpod.s3.ap-southeast-2.amazonaws.com (peanpod.s3.ap-southeast-2.amazonaws.com)|3.5.168.14|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-07-17 12:17:13 ERROR 404: Not Found.`

[seballgeyer]

Hello,

Thank you for reaching out and providing detailed information about the issue you're encountering while downloading the sample data from AWS.

The %0D character in the URL is the root cause of the issue. This character is introduced by Windows-style line endings in the file list. If you remove %0D from the URL, it should work correctly. To help us resolve this, could you please provide the following information:

• Operating System: What operating system are you using to run the script? (e.g., Windows, macOS, Linux)

In the meantime, you can take the following step to potentially resolve the issue:

• Convert Line Endings: If you are using Windows or if the file list has Windows-style line endings, you can convert the line endings to Unix-style using a tool like dos2unix.

Thank you for your patience and cooperation.

Best regards,

Sebastien

[pktrigg]

Hi Thanks so much for getting back to me. I am running ubuntu 22.04.4 LTS within windows subsystem for linux on a windows 11 machine. I am using bash as my shell so did not think there would be this issue. When you say to run dos2unix to sort out the LF issue, which files are you suggesting I run it on? I am not clear where the files are to patch up! At present I was using something like this… sed -i -e 's/\r$//' ../../scripts/demoData/setFiles.sh Paul Kennedy Mobile: +61 439 518 265 From: Sebastien ***@***.***> Sent: Wednesday, July 17, 2024 1:03 PM To: GeoscienceAustralia/ginan ***@***.***> Cc: Paul Kennedy ***@***.***>; Author ***@***.***> Subject: Re: [GeoscienceAustralia/ginan] Installation and download of sample data problem (Discussion #96) Hello, Thank you for reaching out and providing detailed information about the issue you're encountering while downloading the sample data from AWS. The %0D character in the URL is the root cause of the issue. This character is introduced by Windows-style line endings in the file list. If you remove %0D from the URL, it should work correctly. To help us resolve this, could you please provide the following information: * Operating System: What operating system are you using to run the script? (e.g., Windows, macOS, Linux) In the meantime, you can take the following step to potentially resolve the issue: * Convert Line Endings: If you are using Windows or if the file list has Windows-style line endings, you can convert the line endings to Unix-style using a tool like dos2unix. Thank you for your patience and cooperation. Best regards, Sebastien — Reply to this email directly, view it on GitHub<#96 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACDTJMJVRUAMHE3PJE6GOFTZMX3IPAVCNFSM6AAAAABK7Z5C7KVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMBWHEYDKNY>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[seballgeyer]

Hi Paul.,

Sorry, I did forget the most important part, which files?

• inputData/data/data.list
• inputData/products/products.list

Best regards,
Sebastien

[pktrigg]

hi, all sorted. this problem has now been fixed. simple unix/windows CR/LF issue.