Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zip64 format #8

Open
jimmywarting opened this issue Jul 3, 2019 · 8 comments
Open

zip64 format #8

jimmywarting opened this issue Jul 3, 2019 · 8 comments
Labels
enhancement New feature or request
Milestone

Comments

@jimmywarting
Copy link
Contributor

Write zip's as zip64 format to support larger files then 4gb

@jimmywarting jimmywarting added this to the Backlog milestone Jul 3, 2019
@jimmywarting jimmywarting added the enhancement New feature or request label Jul 3, 2019
@jimmywarting
Copy link
Contributor Author

note to self:

  • Mac's default archiver (and others) don't support zip64 so it would be best if each entry is only marked as zip64 when needed
  • the local entry can't contain file size or any information if it is a zip32 or zip64 entry since we don't know how big it is in advance. this information needs to be written to both (zip32 & zip64) central directory at the EOF
  • it would probably also be best if we avoided to write the zip64 central dir too if the zip don't get flagged as zip64

@vincaslt
Copy link
Contributor

Could it be the reason why zip file downloaded on windows can't be extracted with default Extract all... command on windows? 7zip works fine though.

@jimmywarting
Copy link
Contributor Author

possible

@christianwengert
Copy link

christianwengert commented Dec 17, 2020

I started to look into this too. FYI I found this graphical overview of zip32 and zip64, which seems to make it easier to understand.

would love to see this in streamsaver or this library!

https://blog.yaakov.online/zip64-go-big-or-go-home/

@Touffy
Copy link

Touffy commented Mar 3, 2021

Some feedback from implementing this in client-zip : the above article was helpful to some extent but not entirely appropriate for a streaming implementation. What seems to work is to

  • not include the Zip64 extra field at all in the file header
  • still set sizes to zero in the file header, since we're streaming

and then, if after streaming the file data you find it's bigger than 4GB, or the total current size of the stream is bigger than 4GB,

  • use 64-bit sizes in the data descriptor
  • write the Zip64 extra field in the central repository entry for that file

finally, if any file was larger than 4GB or the central repository offset is larger than 4G

  • write the Zip64 end of central repository and its locator

The one thing I couldn't figure out is the "version needed to read" field in file header. Since we're streaming, we can't know in advance whether the file will be readable by just version 2 or at least 4.5. So I write 4.5 all the time. The ZIP readers I've tried my files on don't mind.

@FallingHazard
Copy link

Was there any progress on this?

@Mekacher-Anis
Copy link

Bump ⬆️
first, thank you for the awesome library !!
has there been any progress on this ? I think this is why the native windows zip archiver can't read/extract the files.
When trying to extract the files you get this
image

@Mekacher-Anis
Copy link

Bump ⬆️ first, thank you for the awesome library !! has there been any progress on this ? I think this is why the native windows zip archiver can't read/extract the files. When trying to extract the files you get this image

Never mind, it's just a stupid mistake on my side, I was prepending / to the file name and that doesn't work with the native archive utility of windows and macos

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

6 participants