Corrupt zip creation

[dralletje]

Hello,

When creating a zip with append and then pipe'in it into the unzip module (https://www.npmjs.org/package/unzip) gives this error: "invalid signature: 0xb87c42c0"
With the hex part changing every time.
Also my unzip program occasionally refuses to open it, also with an error.

I have the following, pretty simple, code to generate the zip (It is coffeescript but I think you'll understand):

archiver = require 'archiver'
archive = archiver 'zip'

archive.on 'error', (err) ->
  console.log "Archiverrrr:\n", err.stack

archive.append 'File content', name: 'file.txt'
archive.finalize()

When I pipe that into a fs.createWriteStream and try to open it or I pipe it into unzip.Parse from the unzip module, it gives a error. Archiver itself does not give an error.
This issues arises with both the npm and the latest git build.

Thanks for reading!

[silverwind]

Could you show the code where you pipe it into unzip?

[dralletje]

Yeah sure :-)

archive.pipe(unzip.Parse())

[ctalkington]

what platform / node versions?

[ctalkington]

you may want to delete the modules and clear cache and install again and let me know what versions of modules its pulling in

[dralletje]

I am on Mac osx yosemite and I tested it on debian 6 and 7 too.
All using node versions 0.10.31 and 0.11.13 (Latest stable and latest latest).

After clearing the cache it took version 0.11.0: [email protected] node_modules/archiver
Still no luck, but thanks for helping, already :-)

[ctalkington]

its hard to say given the little pieces of info given. would need full compiled jsjs that is being run to try and debug it.

[ctalkington]

if someone wants to provide a valid repo that I can test with. I can look further into this. there have been changes made to compress-commons to do with extended attributes too. though doesn't seem related here.

[silverwind]

Try this repo.

fs.js goes through a file write/read, which works.
direct.js does piping, which fails with a whole lot of invalid signature errors.

Maybe something about that stream which isn't compatible with unzip?

[ctalkington]

@silverwind thanks. made a comment on the fs.js one. looking into how unzip handles streams now.

[ctalkington]

this looks to be related to compressed data only, forcing the file entry to stored allows it to go through without errors.

[ctalkington]

it could also be related to the use of data descriptors (something that happens with streams and compressed data)

i say this because it seems to parse out the stream (looking for data descriptor) then prepend the data back to itself

[ctalkington]

im going to say this is an unzip bug. i also believe that he misses a few cases such as only checking for data descriptor if compressed (dont believe spec sets any limit to that end)

[silverwind]

Yes, I suspect it to be on unzip's side too. A simple test would be to find another unzipping library that supports streams, but I've yet to find one that does.

Here's the part where unzip checks the signature. I'm not familiar with these signatures, do you see anything obvious?

[ctalkington]

to me it seemed like something when matching for size unknown (ie data descriptors) that the prepending the data back on top caused _readRecord to be used multiple times.

[silverwind]

The fix contained in EvanOxfeld/node-unzip#64 confirms this as an unzip issue, you can close this one.

[ctalkington]

thanks @silverwind