Skip to content

Commit

Permalink
Bump requirements (#36)
Browse files Browse the repository at this point in the history
  • Loading branch information
bicycle1885 authored Oct 23, 2017
1 parent 93332f5 commit 2bc177c
Show file tree
Hide file tree
Showing 6 changed files with 62 additions and 62 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,22 +42,22 @@ erat ex bibendum ipsum, sed varius ipsum ipsum vitae dui.

# Streaming API.
stream = IOBuffer(text)
stream = TranscodingStream(GzipCompression(), stream)
stream = TranscodingStream(GzipDecompression(), stream)
stream = TranscodingStream(GzipCompressor(), stream)
stream = TranscodingStream(GzipDecompressor(), stream)
for line in eachline(stream)
println(line)
end
close(stream)

# Array API.
array = Vector{UInt8}(text)
array = transcode(GzipCompression, array)
array = transcode(GzipDecompression, array)
array = transcode(GzipCompressor, array)
array = transcode(GzipDecompressor, array)
@assert text == String(array)
```

Each codec has an alias to its transcoding stream type for ease of use. For
example, `GzipCompressionStream{S} = TranscodingStream{GzipCompression,S} where
example, `GzipCompressorStream{S} = TranscodingStream{GzipCompressor,S} where
S<:IO`.

## Codec packages
Expand Down
38 changes: 19 additions & 19 deletions docs/src/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ Read lines from a gzip-compressed file
--------------------------------------

The following snippet is an example of using CodecZlib.jl, which exports
`GzipDecompressionStream{S}` as an alias of
`TranscodingStream{GzipDecompression,S} where S<:IO`:
`GzipDecompressorStream{S}` as an alias of
`TranscodingStream{GzipDecompressor,S} where S<:IO`:
```julia
using CodecZlib
stream = GzipDecompressionStream(open("data.txt.gz"))
stream = GzipDecompressorStream(open("data.txt.gz"))
for line in eachline(stream)
# do something...
end
Expand All @@ -21,7 +21,7 @@ Note that the last `close` call will close the file as well. Alternatively,
end:
```julia
using CodecZlib
open(GzipDecompressionStream, "data.txt.gz") do stream
open(GzipDecompressorStream, "data.txt.gz") do stream
for line in eachline(stream)
# do something...
end
Expand All @@ -36,7 +36,7 @@ The input is not limited to usual files. You can read data from a pipe
```julia
using CodecZlib
pipe, proc = open(`cat some.data.gz`)
stream = GzipDecompressionStream(pipe)
stream = GzipDecompressorStream(pipe)
for line in eachline(stream)
# do something...
end
Expand All @@ -51,7 +51,7 @@ Writing compressed data is easy. One thing you need to keep in mind is to call
```julia
using CodecZstd
mat = randn(100, 100)
stream = ZstdCompressionStream(open("data.mat.zst", "w"))
stream = ZstdCompressorStream(open("data.mat.zst", "w"))
writedlm(stream, mat)
close(stream)
```
Expand All @@ -60,7 +60,7 @@ Of course, `open(<stream type>, ...) do ... end` works well:
```julia
using CodecZstd
mat = randn(100, 100)
open(ZstdCompressionStream, "data.mat.zst", "w") do stream
open(ZstdCompressorStream, "data.mat.zst", "w") do stream
writedlm(stream, mat)
end
```
Expand All @@ -77,7 +77,7 @@ transcoding stream as follows:
using CodecZstd
using TranscodingStreams
buf = IOBuffer()
stream = ZstdCompressionStream(buf)
stream = ZstdCompressorStream(buf)
write(stream, "foobarbaz"^100, TranscodingStreams.TOKEN_END)
flush(stream)
compressed = take!(buf)
Expand All @@ -88,17 +88,17 @@ Use a noop codec
----------------

Sometimes, the `Noop` codec, which does nothing, may be useful. The following
example creates a decompression stream based on the extension of a filepath:
example creates a decompressor stream based on the extension of a filepath:
```julia
using CodecZlib
using CodecBzip2
using TranscodingStreams

function makestream(filepath)
if endswith(filepath, ".gz")
codec = GzipDecompression()
codec = GzipDecompressor()
elseif endswith(filepath, ".bz2")
codec = Bzip2Decompression()
codec = Bzip2Decompressor()
else
codec = Noop()
end
Expand All @@ -123,7 +123,7 @@ using CodecZstd
input = open("data.txt.gz", "r")
output = open("data.txt.zst", "w")

stream = GzipDecompressionStream(ZstdCompressionStream(output))
stream = GzipDecompressorStream(ZstdCompressorStream(output))
write(stream, input)
close(stream)
```
Expand All @@ -137,13 +137,13 @@ Stop decoding on the end of a block

Most codecs support decoding concatenated data blocks. For example, if you
concatenate two gzip files into a file and read it using
`GzipDecompressionStream`, you will see the byte stream of concatenation of two
`GzipDecompressorStream`, you will see the byte stream of concatenation of two
files. If you need the first part of the file, you can set `stop_on_end` to
`true` to stop transcoding at the end of the first block:
```julia
using CodecZlib
# cat foo.txt.gz bar.txt.gz > foobar.txt.gz
stream = GzipDecompressionStream(open("foobar.txt.gz"), stop_on_end=true)
stream = GzipDecompressorStream(open("foobar.txt.gz"), stop_on_end=true)
read(stream) #> the content of foo.txt
eof(stream) #> true
```
Expand All @@ -156,8 +156,8 @@ problem of overreading is resolved:
using CodecZlib
using TranscodingStreams
stream = NoopStream(open("foobar.txt.gz"))
read(GzipDecompressionStream(stream, stop_on_end=true)) #> the content of foo.txt
read(GzipDecompressionStream(stream, stop_on_end=true)) #> the content of bar.txt
read(GzipDecompressorStream(stream, stop_on_end=true)) #> the content of foo.txt
read(GzipDecompressorStream(stream, stop_on_end=true)) #> the content of bar.txt
```

Check I/O statistics
Expand All @@ -181,7 +181,7 @@ function decompress(input, output)
println(STDERR)
end

input = GzipDecompressionStream(open("foobar.txt.gz"))
input = GzipDecompressorStream(open("foobar.txt.gz"))
output = IOBuffer()
decompress(input, output)
```
Expand All @@ -197,7 +197,7 @@ in one shot. `transcode` takes a codec object as its first argument and a data
vector as its second argument:
```julia
using CodecZlib
decompressed = transcode(ZlibDecompression, b"x\x9cKL*JLNLI\x04R\x00\x19\xf2\x04U")
decompressed = transcode(ZlibDecompressor, b"x\x9cKL*JLNLI\x04R\x00\x19\xf2\x04U")
String(decompressed)
```

Expand All @@ -211,7 +211,7 @@ data)` method that reuses the allocated object as follows:
```julia
using CodecZstd
strings = ["foo", "bar", "baz"]
codec = ZstdCompression()
codec = ZstdCompressor()
try
for s in strings
data = transcode(codec, s)
Expand Down
56 changes: 28 additions & 28 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Introduction
the actual type should be written as `TranscodingStream{C<:Codec,S<:IO}`. This
type wraps an underlying I/O stream `S` by a codec `C`. The codec defines
transformation (or transcoding) of the stream. For example, when `C` is a
lossless decompression type and `S` is a file, `TranscodingStream{C,S}` behaves
lossless decompressor type and `S` is a file, `TranscodingStream{C,S}` behaves
like a data stream that incrementally decompresses data from the file.

Codecs are defined in other packages listed below:
Expand All @@ -40,74 +40,74 @@ Codecs are defined in other packages listed below:
<td rowspan="6"><a href="https://github.com/bicycle1885/CodecZlib.jl">CodecZlib.jl</a></td>
<td rowspan="6"><a href="http://zlib.net/">zlib</a></td>
<td rowspan="2"><a href="https://tools.ietf.org/html/rfc1952">RFC1952</a></td>
<td><code>GzipCompression</code></td>
<td><code>GzipCompressionStream</code></td>
<td><code>GzipCompressor</code></td>
<td><code>GzipCompressorStream</code></td>
<td>Compress data in gzip (.gz) format.</td>
</tr>
<tr>
<td><code>GzipDecompression</code></td>
<td><code>GzipDecompressionStream</code></td>
<td><code>GzipDecompressor</code></td>
<td><code>GzipDecompressorStream</code></td>
<td>Decompress data in gzip (.gz) format.</td>
</tr>
<tr>
<td rowspan="2"><a href="https://tools.ietf.org/html/rfc1950">RFC1950</a></td>
<td><code>ZlibCompression</code></td>
<td><code>ZlibCompressionStream</code></td>
<td><code>ZlibCompressor</code></td>
<td><code>ZlibCompressorStream</code></td>
<td>Compress data in zlib format.</td>
</tr>
<tr>
<td><code>ZlibDecompression</code></td>
<td><code>ZlibDecompressionStream</code></td>
<td><code>ZlibDecompressor</code></td>
<td><code>ZlibDecompressorStream</code></td>
<td>Decompress data in zlib format.</td>
</tr>
<tr>
<td rowspan="2"><a href="https://tools.ietf.org/html/rfc1951">RFC1951</a></td>
<td><code>DeflateCompression</code></td>
<td><code>DeflateCompressionStream</code></td>
<td><code>DeflateCompressor</code></td>
<td><code>DeflateCompressorStream</code></td>
<td>Compress data in deflate format.</td>
</tr>
<tr>
<td><code>DeflateDecompression</code></td>
<td><code>DeflateDecompressionStream</code></td>
<td><code>DeflateDecompressor</code></td>
<td><code>DeflateDecompressorStream</code></td>
<td>Decompress data in deflate format.</td>
</tr>
<tr>
<td rowspan="2"><a href="https://github.com/bicycle1885/CodecBzip2.jl">CodecBzip2.jl</a></td>
<td rowspan="2"><a href="http://www.bzip.org/">bzip2</a></td>
<td rowspan="2"></td>
<td><code>Bzip2Compression</code></td>
<td><code>Bzip2CompressionStream</code></td>
<td><code>Bzip2Compressor</code></td>
<td><code>Bzip2CompressorStream</code></td>
<td>Compress data in bzip2 (.bz2) format.</td>
</tr>
<tr>
<td><code>Bzip2Decompression</code></td>
<td><code>Bzip2DecompressionStream</code></td>
<td><code>Bzip2Decompressor</code></td>
<td><code>Bzip2DecompressorStream</code></td>
<td>Decompress data in bzip2 (.bz2) format.</td>
</tr>
<tr>
<td rowspan="2"><a href="https://github.com/bicycle1885/CodecXz.jl">CodecXz.jl</a></td>
<td rowspan="2"><a href="https://tukaani.org/xz/">xz</a></td>
<td rowspan="2"><a href="https://tukaani.org/xz/xz-file-format.txt">The .xz File Format</a></td>
<td><code>XzCompression</code></td>
<td><code>XzCompressionStream</code></td>
<td><code>XzCompressor</code></td>
<td><code>XzCompressorStream</code></td>
<td>Compress data in xz (.xz) format.</td>
</tr>
<tr>
<td><code>XzDecompression</code></td>
<td><code>XzDecompressionStream</code></td>
<td><code>XzDecompressor</code></td>
<td><code>XzDecompressorStream</code></td>
<td>Decompress data in xz (.xz) format.</td>
</tr>
<tr>
<td rowspan="2"><a href="https://github.com/bicycle1885/CodecZstd.jl">CodecZstd.jl</a></td>
<td rowspan="2"><a href="http://facebook.github.io/zstd/">zstd</a></td>
<td rowspan="2"><a href="https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md">Zstandard Compression Format</a></td>
<td><code>ZstdCompression</code></td>
<td><code>ZstdCompressionStream</code></td>
<td rowspan="2"><a href="https://github.com/facebook/zstd/blob/dev/doc/zstd_compressor_format.md">Zstandard Compressor Format</a></td>
<td><code>ZstdCompressor</code></td>
<td><code>ZstdCompressorStream</code></td>
<td>Compress data in zstd (.zst) format.</td>
</tr>
<tr>
<td><code>ZstdDecompression</code></td>
<td><code>ZstdDecompressionStream</code></td>
<td><code>ZstdDecompressor</code></td>
<td><code>ZstdDecompressorStream</code></td>
<td>Decompress data in zstd (.zst) format.</td>
</tr>
<tr>
Expand Down Expand Up @@ -148,7 +148,7 @@ Codecs are defined in other packages listed below:

Install packages you need by calling `Pkg.add(<package name>)` in a Julia
session. For example, if you want to read gzip-compressed files, call
`Pkg.add("CodecZlib")` to use `GzipDecompression` or `GzipDecompressionStream`.
`Pkg.add("CodecZlib")` to use `GzipDecompressor` or `GzipDecompressorStream`.
By convention, codec types have a name that matches `.*(Co|Deco)mpression` and
I/O types have a codec name with `Stream` suffix. All codecs are a subtype
`TranscodingStreams.Codec` and streams are a subtype of `Base.IO`. An important
Expand All @@ -163,7 +163,7 @@ Error handling
--------------

You may encounter an error while processing data with this package. For example,
your compressed data may be corrupted or truncated and the decompression codec
your compressed data may be corrupted or truncated and the decompressor codec
cannot handle it properly. In this case, the codec informs the stream of the
error and the stream goes to an unrecoverable mode. In this mode, the only
possible operations are `isopen` and `close`. Other operations, such as `read`
Expand Down
4 changes: 2 additions & 2 deletions src/stream.jl
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,8 @@ julia> using CodecZlib
julia> file = open(Pkg.dir("TranscodingStreams", "test", "abra.gzip"));
julia> stream = TranscodingStream(GzipDecompression(), file)
TranscodingStreams.TranscodingStream{CodecZlib.GzipDecompression,IOStream}(<mode=idle>)
julia> stream = TranscodingStream(GzipDecompressor(), file)
TranscodingStreams.TranscodingStream{CodecZlib.GzipDecompressor,IOStream}(<mode=idle>)
julia> readstring(stream)
"abracadabra"
Expand Down
8 changes: 4 additions & 4 deletions src/transcode.jl
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ julia> using CodecZlib
julia> data = b"abracadabra";
julia> compressed = transcode(ZlibCompression, data);
julia> compressed = transcode(ZlibCompressor, data);
julia> decompressed = transcode(ZlibDecompression, compressed);
julia> decompressed = transcode(ZlibDecompressor, compressed);
julia> String(decompressed)
"abracadabra"
Expand Down Expand Up @@ -52,9 +52,9 @@ julia> using CodecZlib
julia> data = b"abracadabra";
julia> compressed = transcode(ZlibCompression(), data);
julia> compressed = transcode(ZlibCompressor, data);
julia> decompressed = transcode(ZlibDecompression(), compressed);
julia> decompressed = transcode(ZlibDecompressor, compressed);
julia> String(decompressed)
"abracadabra"
Expand Down
8 changes: 4 additions & 4 deletions test/REQUIRE
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
CodecZlib 0.3
CodecBzip2 0.3
CodecXz 0.3
CodecZstd 0.3
CodecZlib 0.4
CodecBzip2 0.4
CodecXz 0.4
CodecZstd 0.4
CodecBase 0.1

0 comments on commit 2bc177c

Please sign in to comment.