Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JuiceFS gateway support for object metadata #5367

Open
stevapple opened this issue Dec 11, 2024 · 7 comments
Open

JuiceFS gateway support for object metadata #5367

stevapple opened this issue Dec 11, 2024 · 7 comments
Labels
kind/feature New feature or request

Comments

@stevapple
Copy link

What would you like to be added:
Add support for storing, fetching and modifying object metadata in JuiceFS gateway.

Why is this needed:
Object metadata is used for statistics and will be user-facing (as HTTP headers), compared to user tags. This is extremely useful when we want to expose some custom metadata to downstream users (eg. web clients).

@stevapple stevapple added the kind/feature New feature or request label Dec 11, 2024
@davies
Copy link
Contributor

davies commented Dec 16, 2024

Can you provide a more detailed example on how this feature will be used?

@stevapple
Copy link
Author

@davies Object metadata is tightly associated with the object, and is user-facing. If we want to allow end-users to provide and get metadata through a public URL (pre-signed or anonymous), this is the only choice. We’re especially using this feature to add MIME and other useful information for media/file management backed by S3.

@prOOrc
Copy link
Contributor

prOOrc commented Dec 18, 2024

We have encountered an issue because of rclone tries to store mtime in the object metadata.

@zhijian-pro
Copy link
Contributor

@prOOrc Can you provide the specific details for reproducing the error, we will not store the metadata, but it should not cause an error.

@zhijian-pro
Copy link
Contributor

@stevapple We have no support plan for object metadata at this time.
Supporting the object metadata can lead to degraded performance in getObject operations.

@prOOrc
Copy link
Contributor

prOOrc commented Dec 26, 2024

@zhijian-pro Yes, it doesn't cause an error. But rclone tries to preserve the mtime in metadata. We have to ignore the modification time during file synchronization from/to sources that support the mtime.
I added metadata support in my fork. It can be enabled optionally.

@davies
Copy link
Contributor

davies commented Dec 26, 2024

@prOOrc Can you send a PR for it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants