Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add configuration option(s) for instructing the S3 API to not upload files to iRODS #137

Open
korydraughn opened this issue Nov 12, 2024 · 0 comments
Labels
enhancement New feature or request performance
Milestone

Comments

@korydraughn
Copy link
Collaborator

Enhancement / Performance

Disabling uploads to the iRODS server would allow us to measure the upload speed from clients to the S3 API.

Storing the uploaded data locally would be optional as well.

For example, the config file could grow two new options:

{
    // Controls whether data is uploaded to iRODS.
    // Defaults to true. When set to false, data is
    // NOT uploaded to iRODS.
    "upload_data_to_irods": true,

    // Controls whether intermediate data (e.g. part files, etc)
    // is stored in a local directory. This only applies
    // when "upload_data_to_irods" is false.
    "store_intermediate_data": true,

    // The directory where intermediate data is stored.
    // This only applies when "store_intermediate_data"
    // is true.
    "intermediate_data_directory": "/tmp/irods_s3_api"
}

We'd need to think about how the intermediate_data_directory property would play with the multipart_upload_part_files_directory property.

@korydraughn korydraughn added enhancement New feature or request performance labels Nov 12, 2024
@korydraughn korydraughn added this to the 0.4.0 milestone Nov 12, 2024
@korydraughn korydraughn changed the title Add configuration option for instructing the S3 API to not upload files to iRODS Add configuration option(s) for instructing the S3 API to not upload files to iRODS Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance
Projects
None yet
Development

No branches or pull requests

1 participant