Process to create Pre-signed URL

[mahimatics]

AWS SDK Exploration

• Figure out how to use the AWS SDK for Java with credentials (maybe without the IAM stuff to begin with)- https://docs.aws.amazon.com/AmazonS3/latest/userguide/example_s3_Scenario_PresignedUrl_section.html
  • https://docs.aws.amazon.com/sdkref/latest/guide/jvm-system-properties.html
  • https://docs.aws.amazon.com/sdkref/latest/guide/feature-static-credentials.html
• Try to list data in the bucket - https://www.baeldung.com/java-aws-s3-list-bucket-objects
• Try to create a pre-signed URL (for uploading data, PUT)
• Try to upload some data using the pre-signed URL

Process Code

To start

• Inputs
  • Bucket name
  • Key name (name under which the file gets uploaded into S3)
• Based on input, create the URL and end process
• Keep this process as async
  • executeBlocking may not be needed for this process

Make the process sync

• Check feasibility

[mahimatics]

• Need to explore either creating /results or sync process
• Change process input to accept bucket, region, rID
• Check the expiration time

[ThorodanBrom]

• Given resource ID
  • Ownership
  • Get the name of the resource and resourceGroup ID as well
  • Call item API again with the resourceGroup and get the name
  • New object key name will be RG-name/RI-name.extension
• Check if the key already exists - using HEAD
• New input called fileType
  • GeoPackage - .gpkg

[mahimatics]

Size testing yet to be done. Sync process code is ready and under review.

[mahimatics]

Once the connection is initiated, S3 allows the completion of the file upload. So the process dev is completed. Deploy it to dev and give it for integration with UI.

[mahimatics]

Draft PR for MinIO support done
File upload size seems to be an issue on MinIO
Upload file only if it doesn't exist

[ThorodanBrom]

• Added fix to get new presigned URL for a particular resource+file type as long as the resource was not successfully onboarded Fix: S3PreSignedURLProcess #209