Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#132 allow users to optionally provide an output buffer when calling transcode #136

Merged
merged 26 commits into from
Apr 9, 2023
Merged

#132 allow users to optionally provide an output buffer when calling transcode #136

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
f5b64a6
allow users to optionally provide an output buffer when calling trans…
baumgold Mar 15, 2023
9e5bb40
add tests and refactor docstring
baumgold Mar 23, 2023
090cb9c
nicer formating
baumgold Mar 25, 2023
b6d5f9d
new copydata! method
baumgold Mar 25, 2023
2ce1efb
reassociate docstring with correct method
baumgold Mar 29, 2023
49b5b82
Expand ByteData
baumgold Apr 3, 2023
78d2b3b
Expand ByteData
baumgold Apr 3, 2023
20e82d9
Expand ByteData
baumgold Apr 3, 2023
c010983
clarify formatting
baumgold Apr 3, 2023
829c655
formatting
baumgold Apr 3, 2023
3d17ac8
Generic args
baumgold Apr 4, 2023
1bf636d
Generic args
baumgold Apr 4, 2023
cd753f6
small fixes
baumgold Apr 4, 2023
0d1e934
simplify
baumgold Apr 4, 2023
3b90dd6
fix buffer
baumgold Apr 4, 2023
5ec6bf3
fix
baumgold Apr 4, 2023
a11043e
formatting
baumgold Apr 4, 2023
e926f4f
fix buffer
baumgold Apr 4, 2023
c5cd743
Address PR comment
JoaoAparicio Apr 7, 2023
8931cb4
Address PR comment
JoaoAparicio Apr 7, 2023
b743bcd
Address PR review
JoaoAparicio Apr 7, 2023
72e6d0a
Merge pull request #1 from JoaoAparicio/outputbuffer
baumgold Apr 8, 2023
8134ab1
dont use isnothing
baumgold Apr 8, 2023
93c28d4
add doc-string
baumgold Apr 8, 2023
b3f444b
replace keepbytes with marginpos
baumgold Apr 8, 2023
9a4b867
fix typeo
baumgold Apr 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions src/buffer.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,11 @@ function copydata!(buf::Buffer, data::Ptr{UInt8}, nbytes::Integer)
return buf
end

# Copy data from `data` to `buf`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we make this a docstring?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want this? All these things are internal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my mind, it's an internal thing. If you're a user, you don't care about internals. If you care about internals you might as well read the code comments.

That said, if you feel strongly about this I can make them docstrings :-)

function copydata!(buf::Buffer, data::Buffer, nbytes::Integer = length(data))
return copydata!(buf, bufferptr(data), nbytes)
end

# Copy data from `buf` to `data`.
function copydata!(data::Ptr{UInt8}, buf::Buffer, nbytes::Integer)
# NOTE: It's caller's responsibility to ensure that the buffer has at least
Expand Down
7 changes: 3 additions & 4 deletions src/noop.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,10 +120,9 @@ function Base.transcode(::Type{Noop}, data::ByteData)
return transcode(Noop(), data)
end

function Base.transcode(::Noop, data::ByteData)
# Copy data because the caller may expect the return object is not the same
# as from the input.
return Vector{UInt8}(data)
function Base.transcode(codec::Noop, input::Buffer, output::Buffer = Buffer(Vector{UInt8}()))
copydata!(output, input)
return output.data
end


Expand Down
35 changes: 28 additions & 7 deletions src/transcode.jl
View file
Open in desktop
baumgold marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# =========

"""
transcode(::Type{C}, data::Vector{UInt8})::Vector{UInt8} where C<:Codec
transcode(::Type{C}, data::ByteData)::ByteData where {C<:Codec}
baumgold marked this conversation as resolved.
Show resolved Hide resolved

Transcode `data` by applying a codec `C()`.

Expand All @@ -27,7 +27,7 @@ julia> String(decompressed)

```
"""
function Base.transcode(::Type{C}, data::ByteData) where C<:Codec
function Base.transcode(::Type{C}, data::ByteData) where {C<:Codec}
baumgold marked this conversation as resolved.
Show resolved Hide resolved
codec = C()
initialize(codec)
try
Expand All @@ -38,10 +38,12 @@ function Base.transcode(::Type{C}, data::ByteData) where C<:Codec
end

"""
transcode(codec::Codec, data::Vector{UInt8})::Vector{UInt8}
transcode(codec::Codec, data::Union{ByteData, Buffer}[, output::Union{ByteData, Buffer}])::ByteData
baumgold marked this conversation as resolved.
Show resolved Hide resolved

Transcode `data` by applying `codec`.

If `output` is unspecified, then this method will allocate it.

Note that this method does not initialize or finalize `codec`. This is
efficient when you transcode a number of pieces of data, but you need to call
[`TranscodingStreams.initialize`](@ref) and
Expand All @@ -59,7 +61,9 @@ julia> codec = ZlibCompressor();

julia> TranscodingStreams.initialize(codec)

julia> compressed = transcode(codec, data);
julia> compressed = Vector{UInt8}()

julia> transcode(codec, data, compressed);

julia> TranscodingStreams.finalize(codec)

Expand All @@ -76,9 +80,18 @@ julia> String(decompressed)

```
"""
baumgold marked this conversation as resolved.
Show resolved Hide resolved
function Base.transcode(codec::Codec, data::ByteData)
input = Buffer(data)
output = Buffer(initial_output_size(codec, buffermem(input)))
_default_output_buffer(codec, input) = Buffer(
initial_output_size(
codec,
buffermem(input)
)
)

function Base.transcode(
baumgold marked this conversation as resolved.
Show resolved Hide resolved
codec::Codec,
input::Buffer,
output::Buffer = _default_output_buffer(codec, input) ,
)
error = Error()
code = startproc(codec, :write, error)
if code === :error
Expand Down Expand Up @@ -121,6 +134,14 @@ function Base.transcode(codec::Codec, data::ByteData)
throw(error[])
end

Base.transcode(codec::Codec, data::ByteData, output::ByteData) =
transcode(codec, data, Buffer(output))

Base.transcode(codec::Codec, data::ByteData) = transcode(codec, Buffer(data))
baumgold marked this conversation as resolved.
Show resolved Hide resolved

Base.transcode(codec::Codec, data::ByteData, output::Buffer) =
baumgold marked this conversation as resolved.
Show resolved Hide resolved
transcode(codec, Buffer(data), output)
baumgold marked this conversation as resolved.
Show resolved Hide resolved

# Return the initial output buffer size.
function initial_output_size(codec::Codec, input::Memory)
return max(
Expand Down
17 changes: 17 additions & 0 deletions test/codecnoop.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,9 +192,26 @@
data = b""
@test transcode(Noop(), data) == data
@test transcode(Noop(), data) !== data
@test transcode(Noop(), data, Vector{UInt8}()) == data
@test transcode(Noop(), data, TranscodingStreams.Buffer(Vector{UInt8}())) == data
@test transcode(Noop(), data, Vector{UInt8}()) !== data
@test transcode(Noop(), data, TranscodingStreams.Buffer(Vector{UInt8}())) !== data
output = Vector{UInt8}()
@test transcode(Noop(), data, output) === output
output = TranscodingStreams.Buffer(Vector{UInt8}())
@test transcode(Noop(), data, output) === output.data

data = b"foo"
@test transcode(Noop(), data) == data
@test transcode(Noop(), data) !== data
@test transcode(Noop(), data, Vector{UInt8}()) == data
@test transcode(Noop(), data, TranscodingStreams.Buffer(Vector{UInt8}())) == data
@test transcode(Noop(), data, Vector{UInt8}()) !== data
@test transcode(Noop(), data, TranscodingStreams.Buffer(Vector{UInt8}())) !== data
output = Vector{UInt8}()
@test transcode(Noop(), data, output) === output
output = TranscodingStreams.Buffer(Vector{UInt8}())
@test transcode(Noop(), data, output) === output.data

TranscodingStreams.test_roundtrip_transcode(Noop, Noop)
TranscodingStreams.test_roundtrip_read(NoopStream, NoopStream)
Expand Down