From 46eea6205c646658e9158113637572509a9ae9bb Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Thu, 2 Nov 2023 12:39:25 +0000 Subject: [PATCH] build based on ec7a59b --- previews/PR1116/client/index.html | 2 +- previews/PR1116/examples/index.html | 2 +- previews/PR1116/index.html | 2 +- previews/PR1116/reference/index.html | 42 +++++++++++++-------------- previews/PR1116/search/index.html | 2 +- previews/PR1116/server/index.html | 2 +- previews/PR1116/websockets/index.html | 2 +- 7 files changed, 27 insertions(+), 27 deletions(-) diff --git a/previews/PR1116/client/index.html b/previews/PR1116/client/index.html index 21aaae645..f8e3c45b0 100644 --- a/previews/PR1116/client/index.html +++ b/previews/PR1116/client/index.html @@ -43,4 +43,4 @@ HTTP.pushlayer!(Auth.auth_layer) # Now can use normal HTTP.jl methods and auth_layer will be included -HTTP.get(url; authcreds=creds)

For more ideas or examples on how client-side layers work, it can be useful to see how HTTP.request is built on layers internally, in the /src/clientlayers source code directory.

+HTTP.get(url; authcreds=creds)

For more ideas or examples on how client-side layers work, it can be useful to see how HTTP.request is built on layers internally, in the /src/clientlayers source code directory.

diff --git a/previews/PR1116/examples/index.html b/previews/PR1116/examples/index.html index c5eddac7b..c77cc898c 100644 --- a/previews/PR1116/examples/index.html +++ b/previews/PR1116/examples/index.html @@ -454,4 +454,4 @@ close(server) - + diff --git a/previews/PR1116/index.html b/previews/PR1116/index.html index 5f15ab44a..558752609 100644 --- a/previews/PR1116/index.html +++ b/previews/PR1116/index.html @@ -58,4 +58,4 @@ # simple echo server send(ws, msg) end -end

Further Documentation

Check out the client, server, and websocket-specific documentation pages for more in-depth discussions and examples for the many configurations available.

Migrating Legacy Code to 1.0

The 1.0 release is finally here! It's been a lot of work over the course of about 9 months combing through every part of the codebase to try and modernize APIs, fix long-standing issues, and bring the level of functionality up to par with other language http implementations. Along the way, some breaking changes were made, but with the aim that the package will now be committed to current published APIs for a long time to come. With the amount of increased functionality and fixes, we hope it provides enough incentive to make the update; as always, if you run into issues upgrading or feel something didn't get polished or fixed quite right, don't hesitate to open an issue so we can help.

The sections below outline a mix of breaking changes that were made, in addition to some of the new features in 1.0 with the aim to help those updating legacy codebases.

Struct Changes

In addition, in the face of redirects or retried requests, note the response_stream will not be written to until the final response is received.

Keyword Argument Changes

Other Largish Changes

"Handlers" framework overhaul

The server-side Handlers framework has been changed to a more modern and flexible framework, including the Handler and Middleware interfaces. It's similar in ways to the old interfaces, but in our opinion, simpler and more straightforward with the clear distinction/pattern between what a Handler does vs. a Middlware.

In that vein, HTTP.Handlers.handle has been removed. HTTP.serve expects a single request or stream Handler function, which should be of the form f(::Request)::Response for the request case, or f(::Stream)::Nothing for streams.

There are also plans to either include some common useful middleware functions in HTTP.jl directly, or a sister package specifically for collecting useful middlewares people can reuse.

WebSockets overhaul

The WebSockets code was some of the oldest and least maintained code in HTTP.jl. It was debated removing it entirely, but there aren't really other modern implementations that are well-maintained. So the WebSockets code was overhauled, modernized, and is now tested against the industry standard autobahn test suite (yay for 3rd party verification!). The API changed as well; while WebSockets.open and WebSockets.listen have stayed the same, the WebSocket object itself now doesn't subtype IO and has a restricted interface like:

HTTP.Router reimplementation

While clever, the old HTTP.Router implementation relied on having routes registered "statically", which can be really inconvenient for any cases where the routes are generated programmatically or need to be set/updated dynamically.

The new HTTP.Router implementation uses a text-matching based trie data structure on incoming request path segments to find the right matching handler to process the request. It also supports parsing and storing path variables, like /api/{id} or double wildcards for matching trailing path segments, like /api/**.

HTTP.Router now also supports complete unrestricted route registration via HTTP.register!.

Internal client-side layers overhaul

While grandiose in vision, the old type-based "layers" framework relied heavily on type parameter abuse for generating a large "stack" of layers to handle different parts of each HTTP.request. The new framework actually matches very closely with the server-side Handler and Middleware interfaces, and can be found in more detail under the Client-side Middleware (Layers) section of the docs. The new implementation, while hopefully bringing greater consistency between client-side and server-side frameworks, is much simpler and forced a large cleanup of state-handling in the HTTP.request process for the better.

In addition to the changing of all the client-side layer definitions, HTTP.stack now behaves slightly different in returning the new "layer" chain for HTTP.request, while also accepting custom request/stream layers is provided. A new HTTP.@client macro is provided for convenience in the case that users want to write a custom client-side middleware/layer and wrap its usage in an HTTP.jl-like client.

There also existed a few internal methods previously for manipulating the global stack of client-side layers (insert, insert_default!, etc.). These have been removed and replaced with a more formal (and documented) API via HTTP.pushlayer! and HTTP.poplayer!. These can be used to globally manipulate the client-side stack of layers for any HTTP.request that is made.

+end

Further Documentation

Check out the client, server, and websocket-specific documentation pages for more in-depth discussions and examples for the many configurations available.

Migrating Legacy Code to 1.0

The 1.0 release is finally here! It's been a lot of work over the course of about 9 months combing through every part of the codebase to try and modernize APIs, fix long-standing issues, and bring the level of functionality up to par with other language http implementations. Along the way, some breaking changes were made, but with the aim that the package will now be committed to current published APIs for a long time to come. With the amount of increased functionality and fixes, we hope it provides enough incentive to make the update; as always, if you run into issues upgrading or feel something didn't get polished or fixed quite right, don't hesitate to open an issue so we can help.

The sections below outline a mix of breaking changes that were made, in addition to some of the new features in 1.0 with the aim to help those updating legacy codebases.

Struct Changes

In addition, in the face of redirects or retried requests, note the response_stream will not be written to until the final response is received.

Keyword Argument Changes

Other Largish Changes

"Handlers" framework overhaul

The server-side Handlers framework has been changed to a more modern and flexible framework, including the Handler and Middleware interfaces. It's similar in ways to the old interfaces, but in our opinion, simpler and more straightforward with the clear distinction/pattern between what a Handler does vs. a Middlware.

In that vein, HTTP.Handlers.handle has been removed. HTTP.serve expects a single request or stream Handler function, which should be of the form f(::Request)::Response for the request case, or f(::Stream)::Nothing for streams.

There are also plans to either include some common useful middleware functions in HTTP.jl directly, or a sister package specifically for collecting useful middlewares people can reuse.

WebSockets overhaul

The WebSockets code was some of the oldest and least maintained code in HTTP.jl. It was debated removing it entirely, but there aren't really other modern implementations that are well-maintained. So the WebSockets code was overhauled, modernized, and is now tested against the industry standard autobahn test suite (yay for 3rd party verification!). The API changed as well; while WebSockets.open and WebSockets.listen have stayed the same, the WebSocket object itself now doesn't subtype IO and has a restricted interface like:

HTTP.Router reimplementation

While clever, the old HTTP.Router implementation relied on having routes registered "statically", which can be really inconvenient for any cases where the routes are generated programmatically or need to be set/updated dynamically.

The new HTTP.Router implementation uses a text-matching based trie data structure on incoming request path segments to find the right matching handler to process the request. It also supports parsing and storing path variables, like /api/{id} or double wildcards for matching trailing path segments, like /api/**.

HTTP.Router now also supports complete unrestricted route registration via HTTP.register!.

Internal client-side layers overhaul

While grandiose in vision, the old type-based "layers" framework relied heavily on type parameter abuse for generating a large "stack" of layers to handle different parts of each HTTP.request. The new framework actually matches very closely with the server-side Handler and Middleware interfaces, and can be found in more detail under the Client-side Middleware (Layers) section of the docs. The new implementation, while hopefully bringing greater consistency between client-side and server-side frameworks, is much simpler and forced a large cleanup of state-handling in the HTTP.request process for the better.

In addition to the changing of all the client-side layer definitions, HTTP.stack now behaves slightly different in returning the new "layer" chain for HTTP.request, while also accepting custom request/stream layers is provided. A new HTTP.@client macro is provided for convenience in the case that users want to write a custom client-side middleware/layer and wrap its usage in an HTTP.jl-like client.

There also existed a few internal methods previously for manipulating the global stack of client-side layers (insert, insert_default!, etc.). These have been removed and replaced with a more formal (and documented) API via HTTP.pushlayer! and HTTP.poplayer!. These can be used to globally manipulate the client-side stack of layers for any HTTP.request that is made.

diff --git a/previews/PR1116/reference/index.html b/previews/PR1116/reference/index.html index 94698ea74..746341498 100644 --- a/previews/PR1116/reference/index.html +++ b/previews/PR1116/reference/index.html @@ -49,14 +49,14 @@ bytes = readavailable(io) play_audio(bytes) end -endsource
HTTP.get — Function
HTTP.get(url [, headers]; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("GET", ...). See HTTP.request.

source
HTTP.put — Function
HTTP.put(url, headers, body; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("PUT", ...). See HTTP.request.

source
HTTP.post — Function
HTTP.post(url, headers, body; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("POST", ...). See HTTP.request.

source
HTTP.head — Function
HTTP.head(url; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("HEAD", ...). See HTTP.request.

source
HTTP.patch — Function
HTTP.patch(url, headers, body; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("PATCH", ...). See HTTP.request.

source
HTTP.delete — Function
HTTP.delete(url [, headers]; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("DELETE", ...). See HTTP.request.

source
HTTP.open — Function
WebSockets.open(handler, url; verbose=false, kw...)

Initiate a websocket connection to url (which should have schema like ws:// or wss://), and call handler(ws) with the websocket connection. Passing verbose=true or verbose=2 will enable debug logging for the life of the websocket connection. handler should be a function of the form f(ws) -> nothing, where ws is a WebSocket. Supported keyword arguments are the same as supported by HTTP.request. Typical websocket usage is:

WebSockets.open(url) do ws
+end
source
HTTP.get — Function
HTTP.get(url [, headers]; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("GET", ...). See HTTP.request.

source
HTTP.put — Function
HTTP.put(url, headers, body; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("PUT", ...). See HTTP.request.

source
HTTP.post — Function
HTTP.post(url, headers, body; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("POST", ...). See HTTP.request.

source
HTTP.head — Function
HTTP.head(url; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("HEAD", ...). See HTTP.request.

source
HTTP.patch — Function
HTTP.patch(url, headers, body; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("PATCH", ...). See HTTP.request.

source
HTTP.delete — Function
HTTP.delete(url [, headers]; <keyword arguments>) -> HTTP.Response

Shorthand for HTTP.request("DELETE", ...). See HTTP.request.

source
HTTP.open — Function
WebSockets.open(handler, url; verbose=false, kw...)

Initiate a websocket connection to url (which should have schema like ws:// or wss://), and call handler(ws) with the websocket connection. Passing verbose=true or verbose=2 will enable debug logging for the life of the websocket connection. handler should be a function of the form f(ws) -> nothing, where ws is a WebSocket. Supported keyword arguments are the same as supported by HTTP.request. Typical websocket usage is:

WebSockets.open(url) do ws
     # iterate incoming websocket messages
     for msg in ws
         # send message back to server or do other logic here
         send(ws, msg)
     end
     # iteration ends when the websocket connection is closed by server or error
-end
source
HTTP.open(method, url, [,headers]) do io
+end
source
HTTP.open(method, url, [,headers]) do io
     write(io, body)
     [startread(io) -> HTTP.Response]
     while !eof(io)
@@ -66,16 +66,16 @@
     open(`vlc -q --play-and-exit --intf dummy -`, "w") do vlc
         write(vlc, http)
     end
-end
source
HTTP.download — Function
download(url, [local_path], [headers]; update_period=1, kw...)

Similar to Base.download this downloads a file, returning the filename. If the local_path:

  • is not provided, then it is saved in a temporary directory
  • if part to a directory is provided then it is saved into that directory
  • otherwise the local path is uses as the filename to save to.

When saving into a directory, the filename is determined (where possible), from the rules of the HTTP.

  • update_period controls how often (in seconds) to report the progress.
    • set to Inf to disable reporting
  • headers specifies headers to be used for the HTTP GET request
  • any additional keyword args (kw...) are passed on to the HTTP request.
source

Request/Response Objects

HTTP.Messages.Request — Type
HTTP.Request(
+end
source
HTTP.download — Function
download(url, [local_path], [headers]; update_period=1, kw...)

Similar to Base.download this downloads a file, returning the filename. If the local_path:

  • is not provided, then it is saved in a temporary directory
  • if part to a directory is provided then it is saved into that directory
  • otherwise the local path is uses as the filename to save to.

When saving into a directory, the filename is determined (where possible), from the rules of the HTTP.

  • update_period controls how often (in seconds) to report the progress.
    • set to Inf to disable reporting
  • headers specifies headers to be used for the HTTP GET request
  • any additional keyword args (kw...) are passed on to the HTTP request.
source

Request/Response Objects

HTTP.Messages.Request — Type
HTTP.Request(
     method, target, headers=[], body=nobody;
     version=v"1.1", url::URI=URI(), responsebody=nothing, parent=nothing, context=HTTP.Context()
-)

Represents a HTTP Request Message with fields:

  • method::String RFC7230 3.1.1

  • target::String RFC7230 5.3

  • version::HTTPVersion RFC7230 2.6

  • headers::HTTP.Headers RFC7230 3.2

  • body::Union{Vector{UInt8}, IO} RFC7230 3.3

  • response, the Response to this Request

  • url::URI, the full URI of the request

  • parent, the Response (if any) that led to this request (e.g. in the case of a redirect). RFC7230 6.4

  • context, a Dict{Symbol, Any} store used by middleware to share state

source
HTTP.Messages.Response — Type
HTTP.Response(status, headers::HTTP.Headers, body; request=nothing)
+)

Represents a HTTP Request Message with fields:

  • method::String RFC7230 3.1.1

  • target::String RFC7230 5.3

  • version::HTTPVersion RFC7230 2.6

  • headers::HTTP.Headers RFC7230 3.2

  • body::Union{Vector{UInt8}, IO} RFC7230 3.3

  • response, the Response to this Request

  • url::URI, the full URI of the request

  • parent, the Response (if any) that led to this request (e.g. in the case of a redirect). RFC7230 6.4

  • context, a Dict{Symbol, Any} store used by middleware to share state

source
HTTP.Messages.Response — Type
HTTP.Response(status, headers::HTTP.Headers, body; request=nothing)
 HTTP.Response(status, body)
-HTTP.Response(body)

Represents an HTTP response message with fields:

source
HTTP.Streams.Stream — Type
Stream(::Request, ::IO)

Creates a HTTP.Stream that wraps an existing IO stream.

  • startwrite(::Stream) sends the Request headers to the IO stream.

  • write(::Stream, body) sends the body (or a chunk of the body).

  • closewrite(::Stream) sends the final 0 chunk (if needed) and calls closewrite on the IO stream.

  • startread(::Stream) calls startread on the IO stream then reads and parses the Response headers.

  • eof(::Stream) and readavailable(::Stream) parse the body from the IO stream.

  • closeread(::Stream) reads the trailers and calls closeread on the IO stream. When the IO stream is a HTTP.Connections.Connection, calling closeread releases the connection back to the connection pool for reuse. If a complete response has not been received, closeread throws EOFError.

source
HTTP.WebSockets.WebSocket — Type
WebSocket(io::HTTP.Connection, req, resp; client=true)

Representation of a websocket connection. Use WebSockets.open to open a websocket connection, passing a handler function f(ws) to send and receive messages. Use WebSockets.listen to listen for incoming websocket connections, passing a handler function f(ws) to send and receive messages.

Call send(ws, msg) to send a message; if msg is an AbstractString, a TEXT websocket message will be sent; if msg is an AbstractVector{UInt8}, a BINARY websocket message will be sent. Otherwise, msg should be an iterable of either AbstractString or AbstractVector{UInt8}, and a fragmented message will be sent, one frame for each iterated element.

Control frames can be sent by calling ping(ws[, data]), pong(ws[, data]), or close(ws[, body::WebSockets.CloseFrameBody]). Calling close will initiate the close sequence and close the underlying connection.

To receive messages, call receive(ws), which will block until a non-control, full message is received. PING messages will automatically be responded to when received. CLOSE messages will also be acknowledged and then a WebSocketError will be thrown with the WebSockets.CloseFrameBody payload, which may include a non-error CLOSE frame status code. WebSockets.isok(err) can be called to check if the CLOSE was normal or unexpected. Fragmented messages will be received until the final frame is received and the full concatenated payload can be returned. receive(ws) returns a Vector{UInt8} for BINARY messages, and a String for TEXT messages.

For convenience, WebSockets support the iteration protocol, where each iteration will receive a non-control message, with iteration terminating when the connection is closed. E.g.:

WebSockets.open(url) do ws
+HTTP.Response(body)

Represents an HTTP response message with fields:

source
HTTP.Streams.Stream — Type
Stream(::Request, ::IO)

Creates a HTTP.Stream that wraps an existing IO stream.

  • startwrite(::Stream) sends the Request headers to the IO stream.

  • write(::Stream, body) sends the body (or a chunk of the body).

  • closewrite(::Stream) sends the final 0 chunk (if needed) and calls closewrite on the IO stream.

  • startread(::Stream) calls startread on the IO stream then reads and parses the Response headers.

  • eof(::Stream) and readavailable(::Stream) parse the body from the IO stream.

  • closeread(::Stream) reads the trailers and calls closeread on the IO stream. When the IO stream is a HTTP.Connections.Connection, calling closeread releases the connection back to the connection pool for reuse. If a complete response has not been received, closeread throws EOFError.

source
HTTP.WebSockets.WebSocket — Type
WebSocket(io::HTTP.Connection, req, resp; client=true)

Representation of a websocket connection. Use WebSockets.open to open a websocket connection, passing a handler function f(ws) to send and receive messages. Use WebSockets.listen to listen for incoming websocket connections, passing a handler function f(ws) to send and receive messages.

Call send(ws, msg) to send a message; if msg is an AbstractString, a TEXT websocket message will be sent; if msg is an AbstractVector{UInt8}, a BINARY websocket message will be sent. Otherwise, msg should be an iterable of either AbstractString or AbstractVector{UInt8}, and a fragmented message will be sent, one frame for each iterated element.

Control frames can be sent by calling ping(ws[, data]), pong(ws[, data]), or close(ws[, body::WebSockets.CloseFrameBody]). Calling close will initiate the close sequence and close the underlying connection.

To receive messages, call receive(ws), which will block until a non-control, full message is received. PING messages will automatically be responded to when received. CLOSE messages will also be acknowledged and then a WebSocketError will be thrown with the WebSockets.CloseFrameBody payload, which may include a non-error CLOSE frame status code. WebSockets.isok(err) can be called to check if the CLOSE was normal or unexpected. Fragmented messages will be received until the final frame is received and the full concatenated payload can be returned. receive(ws) returns a Vector{UInt8} for BINARY messages, and a String for TEXT messages.

For convenience, WebSockets support the iteration protocol, where each iteration will receive a non-control message, with iteration terminating when the connection is closed. E.g.:

WebSockets.open(url) do ws
     for msg in ws
         # do cool stuff with msg
     end
-end
source
HTTP.Messages.header — Function
HTTP.header(::Message, key [, default=""]) -> String

Get header value for key (case-insensitive).

source
HTTP.Messages.headers — Function
HTTP.headers(m::Message, key) -> Vector{String}

Get all headers with key k or empty if none

source
HTTP.Messages.hasheader — Function
HTTP.hasheader(::Message, key) -> Bool

Does header value for key exist (case-insensitive)?

source
HTTP.hasheader(::Message, key, value) -> Bool

Does header for key match value (both case-insensitive)?

source
HTTP.Messages.headercontains — Function
HTTP.headercontains(::Message, key, value) -> Bool

Does the header for key (interpreted as comma-separated list) contain value (both case-insensitive)?

source
HTTP.Messages.setheader — Function
HTTP.setheader(::Message, key => value)

Set header value for key (case-insensitive).

source
HTTP.Messages.appendheader — Function
HTTP.appendheader(::Message, key => value)

Append a header value to message.headers.

If key is the same as the previous header, the value is appended to the value of the previous header with a comma delimiter

Set-Cookie headers are not comma-combined because cookies often contain internal commas.

source
HTTP.Messages.removeheader — Function
HTTP.removeheader(::Message, key)

Remove header for key (case-insensitive).

source
HTTP.Messages.decode — Function
HTTP.decode(r::Union{Request, Response}) -> Vector{UInt8}

For a gzip encoded request/response body, decompress it and return the decompressed body.

source

Request body types

HTTP.Forms.Form — Type
HTTP.Form(data; boundary=string(rand(UInt128), base=16))

Construct a request body for multipart/form-data encoding from data.

data must iterate key-value pairs (e.g. AbstractDict or Vector{Pair}) where the key/value of the iterator is the key/value of each mutipart boundary chunk. Files and other large data arguments can be provided as values as IO arguments: either an IOStream such as returned via open(file), or an IOBuffer for in-memory data.

For complete control over a multipart chunk's details, an HTTP.Multipart type is provided to support setting the filename, Content-Type, and Content-Transfer-Encoding.

Examples

data = Dict(
+end
source
HTTP.Messages.header — Function
HTTP.header(::Message, key [, default=""]) -> String

Get header value for key (case-insensitive).

source
HTTP.Messages.headers — Function
HTTP.headers(m::Message, key) -> Vector{String}

Get all headers with key k or empty if none

source
HTTP.Messages.hasheader — Function
HTTP.hasheader(::Message, key) -> Bool

Does header value for key exist (case-insensitive)?

source
HTTP.hasheader(::Message, key, value) -> Bool

Does header for key match value (both case-insensitive)?

source
HTTP.Messages.headercontains — Function
HTTP.headercontains(::Message, key, value) -> Bool

Does the header for key (interpreted as comma-separated list) contain value (both case-insensitive)?

source
HTTP.Messages.setheader — Function
HTTP.setheader(::Message, key => value)

Set header value for key (case-insensitive).

source
HTTP.Messages.appendheader — Function
HTTP.appendheader(::Message, key => value)

Append a header value to message.headers.

If key is the same as the previous header, the value is appended to the value of the previous header with a comma delimiter

Set-Cookie headers are not comma-combined because cookies often contain internal commas.

source
HTTP.Messages.removeheader — Function
HTTP.removeheader(::Message, key)

Remove header for key (case-insensitive).

source
HTTP.Messages.decode — Function
HTTP.decode(r::Union{Request, Response}) -> Vector{UInt8}

For a gzip encoded request/response body, decompress it and return the decompressed body.

source

Request body types

HTTP.Forms.Form — Type
HTTP.Form(data; boundary=string(rand(UInt128), base=16))

Construct a request body for multipart/form-data encoding from data.

data must iterate key-value pairs (e.g. AbstractDict or Vector{Pair}) where the key/value of the iterator is the key/value of each mutipart boundary chunk. Files and other large data arguments can be provided as values as IO arguments: either an IOStream such as returned via open(file), or an IOBuffer for in-memory data.

For complete control over a multipart chunk's details, an HTTP.Multipart type is provided to support setting the filename, Content-Type, and Content-Transfer-Encoding.

Examples

data = Dict(
     "text" => "text data",
     # filename (cat.png) and content-type (image/png) inferred from the IOStream
     "file1" => open("cat.png"),
@@ -84,11 +84,11 @@
 )
 body = HTTP.Form(data)
 headers = []
-HTTP.post(url, headers, body)
source
HTTP.Forms.Multipart — Type
HTTP.Multipart(filename::String, data::IO, content_type=HTTP.sniff(data), content_transfer_encoding="")

A type to represent a single multipart upload chunk for a file. This type would be used as the value in a key-value pair when constructing a HTTP.Form for a request body (see example below). The data argument must be an IO type such as IOStream, or IOBuffer. The content_type and content_transfer_encoding arguments allow manual setting of these multipart headers. Content-Type will default to the result of the HTTP.sniff(data) mimetype detection algorithm, whereas Content-Transfer-Encoding will be left out if not specified.

Examples

body = HTTP.Form(Dict(
+HTTP.post(url, headers, body)
source
HTTP.Forms.Multipart — Type
HTTP.Multipart(filename::String, data::IO, content_type=HTTP.sniff(data), content_transfer_encoding="")

A type to represent a single multipart upload chunk for a file. This type would be used as the value in a key-value pair when constructing a HTTP.Form for a request body (see example below). The data argument must be an IO type such as IOStream, or IOBuffer. The content_type and content_transfer_encoding arguments allow manual setting of these multipart headers. Content-Type will default to the result of the HTTP.sniff(data) mimetype detection algorithm, whereas Content-Transfer-Encoding will be left out if not specified.

Examples

body = HTTP.Form(Dict(
     "key" => HTTP.Multipart("File.txt", open("MyFile.txt"), "text/plain"),
 ))
 headers = []
-HTTP.post(url, headers, body)

Extended help

Filename SHOULD be included when the Multipart represents the contents of a file RFC7578 4.2

Content-Disposition set to "form-data" MUST be included with each Multipart. An additional "name" parameter MUST be included An optional "filename" parameter SHOULD be included if the contents of a file are sent This will be formatted such as: Content-Disposition: form-data; name="user"; filename="myfile.txt" RFC7578 4.2

Content-Type for each Multipart is optional, but SHOULD be included if the contents of a file are sent. RFC7578 4.4

Content-Transfer-Encoding for each Multipart is deprecated RFC7578 4.7

Other Content- header fields MUST be ignored RFC7578 4.8

source

Request exceptions

Request functions may throw the following exceptions:

HTTP.Exceptions.ConnectError — Type
HTTP.ConnectError

Raised when an error occurs while trying to establish a request connection to the remote server. To see the underlying error, see the error field.

source
HTTP.Exceptions.TimeoutError — Type
HTTP.TimeoutError

Raised when a request times out according to readtimeout keyword argument provided.

source
HTTP.Exceptions.StatusError — Type
HTTP.StatusError

Raised when an HTTP.Response has a 4xx, 5xx or unrecognised status code.

Fields:

  • status::Int16, the response status code.
  • method::String, the request method.
  • target::String, the request target.
  • response, the HTTP.Response
source
HTTP.Exceptions.RequestError — Type
HTTP.RequestError

Raised when an error occurs while physically sending a request to the remote server or reading the response back. To see the underlying error, see the error field.

source

URIs

HTTP.jl uses the URIs.jl package for handling URIs. Some functionality from URIs.jl, relevant to HTTP.jl, are listed below:

URIs.URI — Type
URI(; scheme="", host="", port="", etc...)
+HTTP.post(url, headers, body)

Extended help

Filename SHOULD be included when the Multipart represents the contents of a file RFC7578 4.2

Content-Disposition set to "form-data" MUST be included with each Multipart. An additional "name" parameter MUST be included An optional "filename" parameter SHOULD be included if the contents of a file are sent This will be formatted such as: Content-Disposition: form-data; name="user"; filename="myfile.txt" RFC7578 4.2

Content-Type for each Multipart is optional, but SHOULD be included if the contents of a file are sent. RFC7578 4.4

Content-Transfer-Encoding for each Multipart is deprecated RFC7578 4.7

Other Content- header fields MUST be ignored RFC7578 4.8

source

Request exceptions

Request functions may throw the following exceptions:

HTTP.Exceptions.ConnectError — Type
HTTP.ConnectError

Raised when an error occurs while trying to establish a request connection to the remote server. To see the underlying error, see the error field.

source
HTTP.Exceptions.TimeoutError — Type
HTTP.TimeoutError

Raised when a request times out according to readtimeout keyword argument provided.

source
HTTP.Exceptions.StatusError — Type
HTTP.StatusError

Raised when an HTTP.Response has a 4xx, 5xx or unrecognised status code.

Fields:

  • status::Int16, the response status code.
  • method::String, the request method.
  • target::String, the request target.
  • response, the HTTP.Response
source
HTTP.Exceptions.RequestError — Type
HTTP.RequestError

Raised when an error occurs while physically sending a request to the remote server or reading the response back. To see the underlying error, see the error field.

source

URIs

HTTP.jl uses the URIs.jl package for handling URIs. Some functionality from URIs.jl, relevant to HTTP.jl, are listed below:

URIs.URI — Type
URI(; scheme="", host="", port="", etc...)
 URI(str) = parse(URI, str::String)

A type representing a URI (e.g. a URL). Can be constructed from distinct parts using the various supported keyword arguments, or from a string. The URI constructors will automatically escape any provided query arguments, typically provided as "key"=>"value"::Pair or Dict("key"=>"value"). For all other components, you need to manually percent encode them before passing them to the URI constructor. Note that multiple values for a single query key can provided like Dict("key"=>["value1", "value2"]), in which case the constructor will percent encode only the values you pass in as the query part.

When constructing a URI from a String, you need to ensure that the string is correctly percent encoded already.

The URI struct stores the complete URI in the uri::String field and the component parts in the following SubString fields:

  • scheme, e.g. "http" or "https"
  • userinfo, e.g. "username:password"
  • host e.g. "julialang.org"
  • port e.g. "80" or ""
  • path e.g "/"
  • query e.g. "Foo=1&Bar=2"
  • fragment

The queryparams(::URI) function returns a Dict containing the query.

Note that you manually need to percent decode the content of the individual component fields before you further use their content, as they are returned in percent-encoded form.

URIs.escapeuri — Function
escapeuri(x)

Apply URI percent-encoding to escape special characters in x.

URIs.unescapeuri — Function
unescapeuri(str)

Percent-decode a string according to the URI escaping rules.

URIs.splitpath — Function
URIs.splitpath(path|uri; rstrip_empty_segment=true)

Splits the path into component segments based on /, according to http://tools.ietf.org/html/rfc3986#section-3.3. Any fragment and query parts of the string are ignored if present.

A final empty path segment (trailing '/') is removed, if present. This is technically incompatible with the segment grammar of RFC3986, but it seems to be a common recommendation to make paths with and without a trailing slash equivalent. To preserve any final empty path segment, set rstrip_empty_segment=false.

Examples

julia> URIs.splitpath(URI("http://example.com/foo/bar?a=b&c=d"))
 2-element Array{String,1}:
  "foo"
@@ -99,20 +99,20 @@
  "foo"
  "bar"
Base.isvalid — Method

checks if a URI is valid

Cookies

HTTP.Cookies.Cookie — Type
Cookie()
 Cookie(; kwargs...)
-Cookie(name, value; kwargs...)

A Cookie represents an HTTP cookie as sent in the "Set-Cookie" header of an HTTP response or the Cookie header of an HTTP request. Supported fields (which can be set using keyword arguments) include:

  • name::String: name of the cookie
  • value::String: value of the cookie
  • path::String: applicable path for the cookie
  • domain::String: applicable domain for the cookie
  • expires::Dates.DateTime: when the cookie should expire
  • maxage::Int: maxage == 0 means no max age, maxage < 0 means delete cookie now, maxage > 0 means the # of seconds until expiration
  • secure::Bool: secure cookie attribute
  • httponly::Bool: httponly cookie attribute
  • hostonly::Bool: hostonly cookie attribute
  • samesite::Bool: SameSite cookie attribute

See IETF RFC 6265 for details.

The string representation of a cookie is generated by calling stringify(cookie; isrequest=true), where isrequest=true will only include the name=value pair for requests, and if false, will generate the "Set-Cookie" representation for a response header.

A Vector{Cookie} can be retrieved from a Request/Response object by calling Cookies.cookies(r).

A Cookie can be added to a Request/Response object by calling Cookies.addcookie!(r, cookie).

source
HTTP.Cookies.stringify — Function
stringify(cookie::Cookie, isrequest=true)

Generate the string representation of a cookie. By default, isrequest=true, and only the name=value pair will be included in the cookie string. For isrequest=false, the other cookie attributes will be included, ;-separated, for use in a "Set-Cookie" header.

source
HTTP.Cookies.addcookie! — Function
Cookies.addcookie!(r::Union{HTTP.Request, HTTP.Response}, c::Cookie)

Convenience function for adding a single cookie to a request or response object. For requests, the cookie will be stringified and concatenated to any existing "Cookie" header. For responses, an additional "Set-Cookie" header will be appended.

source
HTTP.Cookies.cookies — Function
HTTP.cookies(r::Union{Request, Response}) -> Vector{Cookie}

Return a list of cookies, if any, parsed from the request "Cookie" or response "Set-Cookie" headers.

source

WebSockets

Sockets.send — Method
send(ws::WebSocket, msg)

Send a message on a websocket connection. If msg is an AbstractString, a TEXT websocket message will be sent; if msg is an AbstractVector{UInt8}, a BINARY websocket message will be sent. Otherwise, msg should be an iterable of either AbstractString or AbstractVector{UInt8}, and a fragmented message will be sent, one frame for each iterated element.

Control frames can be sent by calling ping(ws[, data]), pong(ws[, data]), or close(ws[, body::WebSockets.CloseFrameBody]). Calling close will initiate the close sequence and close the underlying connection.

source
HTTP.WebSockets.receive — Function
receive(ws::WebSocket) -> Union{String, Vector{UInt8}}

Receive a message from a websocket connection. Returns a String if the message was TEXT, or a Vector{UInt8} if the message was BINARY. If control frames (ping or pong) are received, they are handled automatically and a non-control message is waited for. If a CLOSE message is received, it is responded to and a WebSocketError is thrown with the WebSockets.CloseFrameBody as the error value. This error can be checked with WebSockets.isok(err) to see if the closing was "normal" or if an actual error occurred. For fragmented messages, the incoming frames will continue to be read until the final fragment is received. The bodies of each fragment are concatenated into the final message returned by receive. Note that WebSocket objects can be iterated, where each iteration yields a message until the connection is closed.

source
Missing docstring.

Missing docstring for HTTP.WebSockets.close(::HTTP.WebSockets.WebSocket, body). Check Documenter's build log for details.

HTTP.WebSockets.ping — Function
ping(ws, data=[])

Send a PING control frame on a websocket connection. data is an optional body to send with the message. PONG messages are automatically responded to when a PING message is received by a websocket connection.

source
HTTP.WebSockets.pong — Function
pong(ws, data=[])

Send a PONG control frame on a websocket connection. data is an optional body to send with the message. Note that PING messages are automatically responded to internally by the websocket connection with a corresponding PONG message, but in certain cases, a unidirectional PONG message can be used as a one-way heartbeat.

source
Base.iterate — Method
iterate(ws)

Continuously call receive(ws) on a WebSocket connection, with each iteration yielding a message until the connection is closed. E.g.

for msg in ws
+Cookie(name, value; kwargs...)

A Cookie represents an HTTP cookie as sent in the "Set-Cookie" header of an HTTP response or the Cookie header of an HTTP request. Supported fields (which can be set using keyword arguments) include:

  • name::String: name of the cookie
  • value::String: value of the cookie
  • path::String: applicable path for the cookie
  • domain::String: applicable domain for the cookie
  • expires::Dates.DateTime: when the cookie should expire
  • maxage::Int: maxage == 0 means no max age, maxage < 0 means delete cookie now, maxage > 0 means the # of seconds until expiration
  • secure::Bool: secure cookie attribute
  • httponly::Bool: httponly cookie attribute
  • hostonly::Bool: hostonly cookie attribute
  • samesite::Bool: SameSite cookie attribute

See IETF RFC 6265 for details.

The string representation of a cookie is generated by calling stringify(cookie; isrequest=true), where isrequest=true will only include the name=value pair for requests, and if false, will generate the "Set-Cookie" representation for a response header.

A Vector{Cookie} can be retrieved from a Request/Response object by calling Cookies.cookies(r).

A Cookie can be added to a Request/Response object by calling Cookies.addcookie!(r, cookie).

source
HTTP.Cookies.stringify — Function
stringify(cookie::Cookie, isrequest=true)

Generate the string representation of a cookie. By default, isrequest=true, and only the name=value pair will be included in the cookie string. For isrequest=false, the other cookie attributes will be included, ;-separated, for use in a "Set-Cookie" header.

source
HTTP.Cookies.addcookie! — Function
Cookies.addcookie!(r::Union{HTTP.Request, HTTP.Response}, c::Cookie)

Convenience function for adding a single cookie to a request or response object. For requests, the cookie will be stringified and concatenated to any existing "Cookie" header. For responses, an additional "Set-Cookie" header will be appended.

source
HTTP.Cookies.cookies — Function
HTTP.cookies(r::Union{Request, Response}) -> Vector{Cookie}

Return a list of cookies, if any, parsed from the request "Cookie" or response "Set-Cookie" headers.

source

WebSockets

Sockets.send — Method
send(ws::WebSocket, msg)

Send a message on a websocket connection. If msg is an AbstractString, a TEXT websocket message will be sent; if msg is an AbstractVector{UInt8}, a BINARY websocket message will be sent. Otherwise, msg should be an iterable of either AbstractString or AbstractVector{UInt8}, and a fragmented message will be sent, one frame for each iterated element.

Control frames can be sent by calling ping(ws[, data]), pong(ws[, data]), or close(ws[, body::WebSockets.CloseFrameBody]). Calling close will initiate the close sequence and close the underlying connection.

source
HTTP.WebSockets.receive — Function
receive(ws::WebSocket) -> Union{String, Vector{UInt8}}

Receive a message from a websocket connection. Returns a String if the message was TEXT, or a Vector{UInt8} if the message was BINARY. If control frames (ping or pong) are received, they are handled automatically and a non-control message is waited for. If a CLOSE message is received, it is responded to and a WebSocketError is thrown with the WebSockets.CloseFrameBody as the error value. This error can be checked with WebSockets.isok(err) to see if the closing was "normal" or if an actual error occurred. For fragmented messages, the incoming frames will continue to be read until the final fragment is received. The bodies of each fragment are concatenated into the final message returned by receive. Note that WebSocket objects can be iterated, where each iteration yields a message until the connection is closed.

source
Missing docstring.

Missing docstring for HTTP.WebSockets.close(::HTTP.WebSockets.WebSocket, body). Check Documenter's build log for details.

HTTP.WebSockets.ping — Function
ping(ws, data=[])

Send a PING control frame on a websocket connection. data is an optional body to send with the message. PONG messages are automatically responded to when a PING message is received by a websocket connection.

source
HTTP.WebSockets.pong — Function
pong(ws, data=[])

Send a PONG control frame on a websocket connection. data is an optional body to send with the message. Note that PING messages are automatically responded to internally by the websocket connection with a corresponding PONG message, but in certain cases, a unidirectional PONG message can be used as a one-way heartbeat.

source
Base.iterate — Method
iterate(ws)

Continuously call receive(ws) on a WebSocket connection, with each iteration yielding a message until the connection is closed. E.g.

for msg in ws
     # do something with msg
-end
source
HTTP.WebSockets.isclosed — Function
WebSockets.isclosed(ws) -> Bool

Check whether a WebSocket has sent and received CLOSE frames.

source
HTTP.WebSockets.isok — Function
WebSockets.isok(x::WebSocketError) -> Bool

Returns true if the WebSocketError has a non-error status code. When calling receive(websocket), if a CLOSE frame is received, the CLOSE frame body is parsed and thrown inside the WebSocketError, but if the CLOSE frame has a non-error status code, it's safe to ignore the error and return from the WebSockets.open or WebSockets.listen calls without throwing.

source

Utilities

HTTP.Sniff.sniff — Function
HTTP.sniff(content::Union{Vector{UInt8}, String, IO}) => String (mimetype)

HTTP.sniff will look at the first 512 bytes of content to try and determine a valid mimetype. If a mimetype can't be determined appropriately, "application/octet-stream" is returned.

Supports JSON detection through the HTTP.isjson(content) function.

Examples

julia> HTTP.sniff("Hello world!!")
+end
source
HTTP.WebSockets.isclosed — Function
WebSockets.isclosed(ws) -> Bool

Check whether a WebSocket has sent and received CLOSE frames.

source
HTTP.WebSockets.isok — Function
WebSockets.isok(x::WebSocketError) -> Bool

Returns true if the WebSocketError has a non-error status code. When calling receive(websocket), if a CLOSE frame is received, the CLOSE frame body is parsed and thrown inside the WebSocketError, but if the CLOSE frame has a non-error status code, it's safe to ignore the error and return from the WebSockets.open or WebSockets.listen calls without throwing.

source

Utilities

HTTP.Sniff.sniff — Function
HTTP.sniff(content::Union{Vector{UInt8}, String, IO}) => String (mimetype)

HTTP.sniff will look at the first 512 bytes of content to try and determine a valid mimetype. If a mimetype can't be determined appropriately, "application/octet-stream" is returned.

Supports JSON detection through the HTTP.isjson(content) function.

Examples

julia> HTTP.sniff("Hello world!!")
 "text/plain; charset=utf-8"
 
 julia> HTTP.sniff("<html><body>Hello world!!</body></html>")
 "text/html; charset=utf-8"
 
 julia> HTTP.sniff("{"a": -1.0}")
-"application/json; charset=utf-8"
source
HTTP.Strings.escapehtml — Function
escapehtml(i::String)

Returns a string with special HTML characters escaped: &, <, >, ", '

source
HTTP.Strings.tocameldash — Function
tocameldash(s::String)

Ensure the first character and characters that follow a '-' are uppercase.

source
HTTP.Strings.iso8859_1_to_utf8 — Function
iso8859_1_to_utf8(bytes::AbstractVector{UInt8})

Convert from ISO8859_1 to UTF8.

source
HTTP.Strings.ascii_lc_isequal — Function

Case insensitive ASCII character comparison.

source
HTTP.ascii_lc_isequal(a::String, b::String)

Case insensitive ASCII string comparison.

source
HTTP.StatusCodes.statustext — Function
statustext(::Int) -> String

String representation of a HTTP status code.

Examples

julia> statustext(200)
+"application/json; charset=utf-8"
source
HTTP.Strings.escapehtml — Function
escapehtml(i::String)

Returns a string with special HTML characters escaped: &, <, >, ", '

source
HTTP.Strings.tocameldash — Function
tocameldash(s::String)

Ensure the first character and characters that follow a '-' are uppercase.

source
HTTP.Strings.iso8859_1_to_utf8 — Function
iso8859_1_to_utf8(bytes::AbstractVector{UInt8})

Convert from ISO8859_1 to UTF8.

source
HTTP.Strings.ascii_lc_isequal — Function

Case insensitive ASCII character comparison.

source
HTTP.ascii_lc_isequal(a::String, b::String)

Case insensitive ASCII string comparison.

source
HTTP.StatusCodes.statustext — Function
statustext(::Int) -> String

String representation of a HTTP status code.

Examples

julia> statustext(200)
 "OK"
 
 julia> statustext(404)
-"Not Found"
source

Server / Handlers

Core Server

HTTP.Servers.listen — Function
HTTP.listen(handler, host=Sockets.localhost, port=8081; kw...)
+"Not Found"
source

Server / Handlers

Core Server

HTTP.Servers.listen — Function
HTTP.listen(handler, host=Sockets.localhost, port=8081; kw...)
 HTTP.listen(handler, port::Integer=8081; kw...)
 HTTP.listen(handler, server::Base.IOServer; kw...)
 HTTP.listen!(args...; kw...) -> HTTP.Server

Listen for HTTP connections and execute the handler function for each request. Listening details can be passed as host/port pair, a single port (host will default to localhost), or an already listening server object, as returned from Sockets.listen. To open up a server to external requests, the host argument is typically "0.0.0.0".

The HTTP.listen! form is non-blocking and returns an HTTP.Server object which can be wait(server)ed on manually, or close(server)ed to gracefully shut down the server. Calling HTTP.forceclose(server) will immediately force close the server and all active connections. HTTP.listen will block on the server listening loop until interrupted or and an irrecoverable error occurs.

The handler function should be of the form f(::HTTP.Stream)::Nothing, and should at the minimum set a status via setstatus() and call startwrite() either explicitly or implicitly by writing out a response via write(). Failure to do this will result in an HTTP 500 error being transmitted to the client.

Optional keyword arguments:

  • sslconfig=nothing, Provide an MbedTLS.SSLConfig object to handle ssl connections. Pass sslconfig=MbedTLS.SSLConfig(false) to disable ssl verification (useful for testing). Construct a custom SSLConfig object with MbedTLS.SSLConfig(certfile, keyfile).
  • tcpisvalid = tcp->true, function f(::TCPSocket)::Bool to check if accepted connections are valid before processing requests. e.g. to do source IP filtering.
  • readtimeout::Int=0, close the connection if no data is received for this many seconds. Use readtimeout = 0 to disable.
  • reuseaddr::Bool=false, allow multiple servers to listen on the same port. Not supported on some OS platforms. Can check HTTP.Servers.supportsreuseaddr().
  • server::Base.IOServer=nothing, provide an IOServer object to listen on; allows manually closing or configuring the server socket.
  • verbose::Bool=false, log connection information to stdout.
  • access_log::Function, function for formatting access log messages. The function should accept two arguments, io::IO to which the messages should be written, and http::HTTP.Stream which can be used to query information from. See also @logfmt_str.
  • on_shutdown::Union{Function, Vector{<:Function}, Nothing}=nothing, one or more functions to be run if the server is closed (for example by an InterruptException). Note, shutdown function(s) will not run if an IOServer object is supplied to the server keyword argument and closed by close(server).

e.g.

# start a blocking server
@@ -157,7 +157,7 @@
 
 chat_client() = HTTP.open("POST", "http://127.0.0.1:8087") do io
     chat(io)
-end
source
HTTP.Handlers.serve — Function
HTTP.serve(handler, host=Sockets.localhost, port=8081; kw...)
+end
source
HTTP.Handlers.serve — Function
HTTP.serve(handler, host=Sockets.localhost, port=8081; kw...)
 HTTP.serve(handler, port::Integer=8081; kw...)
 HTTP.serve(handler, server::Base.IOServer; kw...)
 HTTP.serve!(args...; kw...) -> HTTP.Server

Listen for HTTP connections and execute the handler function for each request. Listening details can be passed as host/port pair, a single port (host will default to localhost), or an already listening server object, as returned from Sockets.listen. To open up a server to external requests, the host argument is typically "0.0.0.0".

The HTTP.serve! form is non-blocking and returns an HTTP.Server object which can be wait(server)ed on manually, or close(server)ed to gracefully shut down the server. Calling HTTP.forceclose(server) will immediately force close the server and all active connections. HTTP.serve will block on the server listening loop until interrupted or and an irrecoverable error occurs.

The handler function should be of the form f(req::HTTP.Request)::HTTP.Response. Alternatively, passing stream=true requires the handler to be of the form f(stream::HTTP.Stream) -> Nothing. See HTTP.Router for details on using it as a request handler.

Optional keyword arguments:

  • sslconfig=nothing, Provide an MbedTLS.SSLConfig object to handle ssl connections. Pass sslconfig=MbedTLS.SSLConfig(false) to disable ssl verification (useful for testing). Construct a custom SSLConfig object with MbedTLS.SSLConfig(certfile, keyfile).
  • tcpisvalid = tcp->true, function f(::TCPSocket)::Bool to check if accepted connections are valid before processing requests. e.g. to do source IP filtering.
  • readtimeout::Int=0, close the connection if no data is received for this many seconds. Use readtimeout = 0 to disable.
  • reuseaddr::Bool=false, allow multiple servers to listen on the same port. Not supported on some OS platforms. Can check HTTP.Servers.supportsreuseaddr().
  • server::Base.IOServer=nothing, provide an IOServer object to listen on; allows manually closing or configuring the server socket.
  • verbose::Bool=false, log connection information to stdout.
  • access_log::Function, function for formatting access log messages. The function should accept two arguments, io::IO to which the messages should be written, and http::HTTP.Stream which can be used to query information from. See also @logfmt_str.
  • on_shutdown::Union{Function, Vector{<:Function}, Nothing}=nothing, one or more functions to be run if the server is closed (for example by an InterruptException). Note, shutdown function(s) will not run if an IOServer object is supplied to the server keyword argument and closed by close(server).
# start a blocking echo server
@@ -170,7 +170,7 @@
     return HTTP.Response(200, "response body")
 end
 # can gracefully close server manually
-close(server)
source
HTTP.WebSockets.listen — Function
WebSockets.listen(handler, host, port; verbose=false, kw...)
+close(server)
source
HTTP.WebSockets.listen — Function
WebSockets.listen(handler, host, port; verbose=false, kw...)
 WebSockets.listen!(handler, host, port; verbose=false, kw...) -> HTTP.Server

Listen for websocket connections on host and port, and call handler(ws), which should be a function taking a single WebSocket argument. Keyword arguments kw... are the same as supported by HTTP.listen. Typical usage is like:

WebSockets.listen(host, port) do ws
     # iterate incoming websocket messages
     for msg in ws
@@ -178,11 +178,11 @@
         send(ws, msg)
     end
     # iteration ends when the websocket connection is closed by client or error
-end
source

Middleware / Handlers

HTTP.Handlers.Handler — Type
Handler

Abstract type for the handler interface that exists for documentation purposes. A Handler is any function of the form f(req::HTTP.Request) -> HTTP.Response. There is no requirement to subtype Handler and users should not rely on or dispatch on Handler. A Handler function f can be passed to HTTP.serve wherein a server will pass each incoming request to f to be handled and a response to be returned. Handler functions are also the inputs to Middleware functions which are functions of the form f(::Handler) -> Handler, i.e. they take a Handler function as input, and return a "modified" or enhanced Handler function.

For advanced cases, a Handler function can also be of the form f(stream::HTTP.Stream) -> Nothing. In this case, the server would be run like HTTP.serve(f, ...; stream=true). For this use-case, the handler function reads the request and writes the response to the stream directly. Note that any middleware used with a stream handler also needs to be of the form f(stream_handler) -> stream_handler, i.e. it needs to accept a stream Handler function and return a stream Handler function.

source
HTTP.Handlers.Middleware — Type
Middleware

Abstract type for the middleware interface that exists for documentation purposes. A Middleware is any function of the form f(::Handler) -> Handler (ref: Handler). There is no requirement to subtype Middleware and users should not rely on or dispatch on the Middleware type. While HTTP.serve(f, ...) requires a handler function f to be passed, middleware can be "stacked" to create a chain of functions that are called in sequence, like HTTP.serve(base_handler |> cookie_middleware |> auth_middlware, ...), where the base_handler Handler function is passed to cookie_middleware, which takes the handler and returns a "modified" handler (that parses and stores cookies). This "modified" handler is then an input to the auth_middlware, which further enhances/modifies the handler.

source
HTTP.Handlers.streamhandler — Function
streamhandler(request_handler) -> stream handler

Middleware that takes a request handler and returns a stream handler. Used by default in HTTP.serve to take the user-provided request handler and process the Stream from HTTP.listen and pass the parsed Request to the handler.

Is included by default in HTTP.serve as the base "middleware" when stream=false is passed.

source
HTTP.Handlers.Router — Type
HTTP.Router(_404, _405, middleware=nothing)

Define a router object that maps incoming requests by path to registered routes and associated handlers. Paths can be registered using HTTP.register!. The router object itself is a "request handler" that can be called like:

r = HTTP.Router()
-resp = r(request)

Which will inspect the request, find the matching, registered handler from the url, and pass the request on to be handled further.

See HTTP.register! for additional information on registering handlers based on routes.

If a request doesn't have a matching, registered handler, the _404 handler is called which, by default, returns a HTTP.Response(404). If a route matches the path, but not the method/verb (e.g. there's a registerd route for "GET /api", but the request is "POST /api"), then the _405 handler is called, which by default returns HTTP.Response(405) (method not allowed).

A middleware (Middleware) can optionally be provided as well, which will be called after the router has matched the request to a route, but before the route's handler is called. This provides a "hook" for matched routes that can be helpful for metric tracking, logging, etc. Note that the middleware is only called if the route is matched; for the 404 and 405 cases, users should wrap those handlers in the middleware manually.

source
HTTP.Handlers.register! — Function
HTTP.register!(r::Router, method, path, handler)
-HTTP.register!(r::Router, path, handler)

Register a handler function that should be called when an incoming request matches path and the optionally provided method (if not provided, any method is allowed). Can be used to dynamically register routes. When a registered route is matched, the original route string is stored in the request.context[:route] variable. The following path types are allowed for matching:

  • /api/widgets: exact match of static strings
  • /api/*/owner: single * to wildcard match anything for a single segment
  • /api/widget/{id}: Define a path variable id that matches any valued provided for this segment; path variables are available in the request context like HTTP.getparams(req)["id"]
  • /api/widget/{id:[0-9]+}: Define a path variable id that does a regex match for integers for this segment
  • /api/**: double wildcard matches any number of trailing segments in the request path; the double wildcard must be the last segment in the path
source
HTTP.Handlers.getparam — Function
HTTP.getparam(req, name, default=nothing) -> String

Retrieve a matched path parameter with name name from request context. If a path was registered with a router via HTTP.register! like "/api/widget/{id}", then the path parameter can be retrieved like `id = HTTP.getparam(req, "id").

source
HTTP.Handlers.getparams — Function
HTTP.getparams(req) -> Dict{String, String}

Retrieve any matched path parameters from the request context. If a path was registered with a router via HTTP.register! like "/api/widget/{id}", then the path parameters are available in the request context and can be retrieved like id = HTTP.getparams(req)["id"].

source
HTTP.Handlers.cookie_middleware — Function
HTTP.Handlers.cookie_middleware(handler) -> handler

Middleware that parses and stores any cookies in the incoming request in the request context. Cookies can then be retrieved by calling HTTP.getcookies(req) in subsequent middlewares/handlers.

source
HTTP.Handlers.getcookies — Function
HTTP.getcookies(req) -> Vector{Cookie}

Retrieve any parsed cookies from a request context. Cookies are expected to be stored in the req.context[:cookies] of the request context as implemented in the HTTP.Handlers.cookie_middleware middleware.

source
HTTP.@logfmt_str — Macro
logfmt"..."

Parse an NGINX-style log format string and return a function mapping (io::IO, http::HTTP.Stream) -> body suitable for passing to HTTP.listen using the access_log keyword argument.

The following variables are currently supported:

  • $http_name: arbitrary request header (with - replaced with _, e.g. http_user_agent)
  • $sent_http_name: arbitrary response header (with - replaced with _)
  • $request: the request line, e.g. GET /index.html HTTP/1.1
  • $request_method: the request method
  • $request_uri: the request URI
  • $remote_addr: client address
  • $remote_port: client port
  • $remote_user: user name supplied with the Basic authentication
  • $server_protocol: server protocol
  • $time_iso8601: local time in ISO8601 format
  • $time_local: local time in Common Log Format
  • $status: response status code
  • $body_bytes_sent: number of bytes in response body

Examples

logfmt"[$time_iso8601] \"$request\" $status" # [2021-05-01T12:34:40+0100] "GET /index.html HTTP/1.1" 200
+end
source

Middleware / Handlers

HTTP.Handlers.Handler — Type
Handler

Abstract type for the handler interface that exists for documentation purposes. A Handler is any function of the form f(req::HTTP.Request) -> HTTP.Response. There is no requirement to subtype Handler and users should not rely on or dispatch on Handler. A Handler function f can be passed to HTTP.serve wherein a server will pass each incoming request to f to be handled and a response to be returned. Handler functions are also the inputs to Middleware functions which are functions of the form f(::Handler) -> Handler, i.e. they take a Handler function as input, and return a "modified" or enhanced Handler function.

For advanced cases, a Handler function can also be of the form f(stream::HTTP.Stream) -> Nothing. In this case, the server would be run like HTTP.serve(f, ...; stream=true). For this use-case, the handler function reads the request and writes the response to the stream directly. Note that any middleware used with a stream handler also needs to be of the form f(stream_handler) -> stream_handler, i.e. it needs to accept a stream Handler function and return a stream Handler function.

source
HTTP.Handlers.Middleware — Type
Middleware

Abstract type for the middleware interface that exists for documentation purposes. A Middleware is any function of the form f(::Handler) -> Handler (ref: Handler). There is no requirement to subtype Middleware and users should not rely on or dispatch on the Middleware type. While HTTP.serve(f, ...) requires a handler function f to be passed, middleware can be "stacked" to create a chain of functions that are called in sequence, like HTTP.serve(base_handler |> cookie_middleware |> auth_middlware, ...), where the base_handler Handler function is passed to cookie_middleware, which takes the handler and returns a "modified" handler (that parses and stores cookies). This "modified" handler is then an input to the auth_middlware, which further enhances/modifies the handler.

source
HTTP.Handlers.streamhandler — Function
streamhandler(request_handler) -> stream handler

Middleware that takes a request handler and returns a stream handler. Used by default in HTTP.serve to take the user-provided request handler and process the Stream from HTTP.listen and pass the parsed Request to the handler.

Is included by default in HTTP.serve as the base "middleware" when stream=false is passed.

source
HTTP.Handlers.Router — Type
HTTP.Router(_404, _405, middleware=nothing)

Define a router object that maps incoming requests by path to registered routes and associated handlers. Paths can be registered using HTTP.register!. The router object itself is a "request handler" that can be called like:

r = HTTP.Router()
+resp = r(request)

Which will inspect the request, find the matching, registered handler from the url, and pass the request on to be handled further.

See HTTP.register! for additional information on registering handlers based on routes.

If a request doesn't have a matching, registered handler, the _404 handler is called which, by default, returns a HTTP.Response(404). If a route matches the path, but not the method/verb (e.g. there's a registerd route for "GET /api", but the request is "POST /api"), then the _405 handler is called, which by default returns HTTP.Response(405) (method not allowed).

A middleware (Middleware) can optionally be provided as well, which will be called after the router has matched the request to a route, but before the route's handler is called. This provides a "hook" for matched routes that can be helpful for metric tracking, logging, etc. Note that the middleware is only called if the route is matched; for the 404 and 405 cases, users should wrap those handlers in the middleware manually.

source
HTTP.Handlers.register! — Function
HTTP.register!(r::Router, method, path, handler)
+HTTP.register!(r::Router, path, handler)

Register a handler function that should be called when an incoming request matches path and the optionally provided method (if not provided, any method is allowed). Can be used to dynamically register routes. When a registered route is matched, the original route string is stored in the request.context[:route] variable. The following path types are allowed for matching:

  • /api/widgets: exact match of static strings
  • /api/*/owner: single * to wildcard match anything for a single segment
  • /api/widget/{id}: Define a path variable id that matches any valued provided for this segment; path variables are available in the request context like HTTP.getparams(req)["id"]
  • /api/widget/{id:[0-9]+}: Define a path variable id that does a regex match for integers for this segment
  • /api/**: double wildcard matches any number of trailing segments in the request path; the double wildcard must be the last segment in the path
source
HTTP.Handlers.getparam — Function
HTTP.getparam(req, name, default=nothing) -> String

Retrieve a matched path parameter with name name from request context. If a path was registered with a router via HTTP.register! like "/api/widget/{id}", then the path parameter can be retrieved like `id = HTTP.getparam(req, "id").

source
HTTP.Handlers.getparams — Function
HTTP.getparams(req) -> Dict{String, String}

Retrieve any matched path parameters from the request context. If a path was registered with a router via HTTP.register! like "/api/widget/{id}", then the path parameters are available in the request context and can be retrieved like id = HTTP.getparams(req)["id"].

source
HTTP.Handlers.cookie_middleware — Function
HTTP.Handlers.cookie_middleware(handler) -> handler

Middleware that parses and stores any cookies in the incoming request in the request context. Cookies can then be retrieved by calling HTTP.getcookies(req) in subsequent middlewares/handlers.

source
HTTP.Handlers.getcookies — Function
HTTP.getcookies(req) -> Vector{Cookie}

Retrieve any parsed cookies from a request context. Cookies are expected to be stored in the req.context[:cookies] of the request context as implemented in the HTTP.Handlers.cookie_middleware middleware.

source
HTTP.@logfmt_str — Macro
logfmt"..."

Parse an NGINX-style log format string and return a function mapping (io::IO, http::HTTP.Stream) -> body suitable for passing to HTTP.listen using the access_log keyword argument.

The following variables are currently supported:

  • $http_name: arbitrary request header (with - replaced with _, e.g. http_user_agent)
  • $sent_http_name: arbitrary response header (with - replaced with _)
  • $request: the request line, e.g. GET /index.html HTTP/1.1
  • $request_method: the request method
  • $request_uri: the request URI
  • $remote_addr: client address
  • $remote_port: client port
  • $remote_user: user name supplied with the Basic authentication
  • $server_protocol: server protocol
  • $time_iso8601: local time in ISO8601 format
  • $time_local: local time in Common Log Format
  • $status: response status code
  • $body_bytes_sent: number of bytes in response body

Examples

logfmt"[$time_iso8601] \"$request\" $status" # [2021-05-01T12:34:40+0100] "GET /index.html HTTP/1.1" 200
 
-logfmt"$remote_addr \"$http_user_agent\"" # 127.0.0.1 "curl/7.47.0"
source

Advanced Topics

Messages Interface

HTTP.Messages.iserror — Function
iserror(::Response)

Does this Response have an error status?

source
HTTP.Messages.isredirect — Function
isredirect(::Response)

Does this Response have a redirect status?

source
HTTP.Messages.ischunked — Function
ischunked(::Message)

Does the Message have a "Transfer-Encoding: chunked" header?

source
HTTP.Messages.issafe — Function
issafe(::Request)

https://tools.ietf.org/html/rfc7231#section-4.2.1

source
HTTP.Messages.isidempotent — Function
isidempotent(::Request)

https://tools.ietf.org/html/rfc7231#section-4.2.2

source
HTTP.Messages.retryable — Function
retryable(::Request)

Whether a Request is eligible to be retried.

source
HTTP.Messages.defaultheader! — Function
defaultheader!(::Message, key => value)

Set header value in message for key if it is not already set.

source
HTTP.Messages.readheaders — Function
readheaders(::IO, ::Message)

Read headers (and startline) from an IO stream into a Message struct. Throw EOFError if input is incomplete.

source
HTTP.HeadersRequest.setuseragent! — Function
setuseragent!(x::Union{String, Nothing})

Set the default User-Agent string to be used in each HTTP request. Can be manually overridden by passing an explicit User-Agent header. Setting nothing will prevent the default User-Agent header from being passed.

source
HTTP.Messages.readchunksize — Function

Read chunk-size from an IO stream. After the final zero size chunk, read trailers into a Message struct.

source
HTTP.Messages.headerscomplete — Method
headerscomplete(::Message)

Have the headers been read into this Message?

source
HTTP.Messages.writestartline — Function
writestartline(::IO, ::Message)

e.g. "GET /path HTTP/1.1\r\n" or "HTTP/1.1 200 OK\r\n"

source
HTTP.Messages.writeheaders — Function
writeheaders(::IO, ::Message)

Write Message start line and a line for each "name: value" pair and a trailing blank line.

source
Base.write — Method
write(::IO, ::Message)

Write start line, headers and body of HTTP Message.

source
HTTP.Streams.closebody — Function
closebody(::Stream)

Write the final 0 chunk if needed.

source
HTTP.Streams.isaborted — Function
isaborted(::Stream{<:Response})

Has the server signaled that it does not wish to receive the message body?

"If [the response] indicates the server does not wish to receive the message body and is closing the connection, the client SHOULD immediately cease transmitting the body and close the connection." RFC7230, 6.5

source
HTTP.Cookies.CookieJar — Type
CookieJar()

A thread-safe object for storing cookies returned in "Set-Cookie" response headers. Keyed by appropriate host from the original request made. Can be created manually and passed like HTTP.get(url; cookiejar=mycookiejar) to avoid using the default global CookieJar. The 2 main functions for interacting with a CookieJar are Cookies.getcookies!, which returns a Vector{Cookie} for a given url (and will remove expired cookies from the jar), and Cookies.setcookies!, which will store "Set-Cookie" response headers in the cookie jar.

source
HTTP.Cookies.getcookies! — Function
Cookies.getcookies!(jar::CookieJar, url::URI)

Retrieve valid Cookies from the CookieJar according to the provided url. Cookies will be returned as a Vector{Cookie}. Only cookies for http or https scheme in the url will be returned. Cookies will be checked according to the canonical host of the url and any cookie max age or expiration will be accounted for. Expired cookies will not be returned and will be removed from the cookie jar.

source
HTTP.Cookies.setcookies! — Function
Cookies.setcookies!(jar::CookieJar, url::URI, headers::Headers)

Identify, "Set-Cookie" response headers from headers, parse the Cookies, and store valid entries in the cookie jar according to the canonical host in url. Cookies can be retrieved from the jar via Cookies.getcookies!.

source

Client-side Middleware (Layers)

HTTP.Layer — Type
Layer

Abstract type to represent a client-side middleware that exists for documentation purposes. A layer is any function of the form f(::Handler) -> Handler, where Handler is a function of the form f(::Request) -> Response. Note that the Handler definition is from the server-side documentation, and is "hard-coded" on the client side. It may also be apparent that a Layer is the same as the Middleware interface from server-side, which is true, but we define Layer to clarify the client-side distinction and its unique usage. Custom layers can be deployed in one of two ways:

  • HTTP.@client: Create a custom "client" with shorthand verb definitions, but which include custom layers; only these new verb methods will use the custom layers.
  • HTTP.pushlayer!/HTTP.poplayer!: Allows globally adding and removing layers from the default HTTP.jl layer stack; all http requests will then use the custom layers

Quick Examples

module Auth
+logfmt"$remote_addr \"$http_user_agent\"" # 127.0.0.1 "curl/7.47.0"
source

Advanced Topics

Messages Interface

HTTP.Messages.iserror — Function
iserror(::Response)

Does this Response have an error status?

source
HTTP.Messages.isredirect — Function
isredirect(::Response)

Does this Response have a redirect status?

source
HTTP.Messages.ischunked — Function
ischunked(::Message)

Does the Message have a "Transfer-Encoding: chunked" header?

source
HTTP.Messages.issafe — Function
issafe(::Request)

https://tools.ietf.org/html/rfc7231#section-4.2.1

source
HTTP.Messages.isidempotent — Function
isidempotent(::Request)

https://tools.ietf.org/html/rfc7231#section-4.2.2

source
HTTP.Messages.retryable — Function
retryable(::Request)

Whether a Request is eligible to be retried.

source
HTTP.Messages.defaultheader! — Function
defaultheader!(::Message, key => value)

Set header value in message for key if it is not already set.

source
HTTP.Messages.readheaders — Function
readheaders(::IO, ::Message)

Read headers (and startline) from an IO stream into a Message struct. Throw EOFError if input is incomplete.

source
HTTP.HeadersRequest.setuseragent! — Function
setuseragent!(x::Union{String, Nothing})

Set the default User-Agent string to be used in each HTTP request. Can be manually overridden by passing an explicit User-Agent header. Setting nothing will prevent the default User-Agent header from being passed.

source
HTTP.Messages.readchunksize — Function

Read chunk-size from an IO stream. After the final zero size chunk, read trailers into a Message struct.

source
HTTP.Messages.headerscomplete — Method
headerscomplete(::Message)

Have the headers been read into this Message?

source
HTTP.Messages.writestartline — Function
writestartline(::IO, ::Message)

e.g. "GET /path HTTP/1.1\r\n" or "HTTP/1.1 200 OK\r\n"

source
HTTP.Messages.writeheaders — Function
writeheaders(::IO, ::Message)

Write Message start line and a line for each "name: value" pair and a trailing blank line.

source
Base.write — Method
write(::IO, ::Message)

Write start line, headers and body of HTTP Message.

source
HTTP.Streams.closebody — Function
closebody(::Stream)

Write the final 0 chunk if needed.

source
HTTP.Streams.isaborted — Function
isaborted(::Stream{<:Response})

Has the server signaled that it does not wish to receive the message body?

"If [the response] indicates the server does not wish to receive the message body and is closing the connection, the client SHOULD immediately cease transmitting the body and close the connection." RFC7230, 6.5

source
HTTP.Cookies.CookieJar — Type
CookieJar()

A thread-safe object for storing cookies returned in "Set-Cookie" response headers. Keyed by appropriate host from the original request made. Can be created manually and passed like HTTP.get(url; cookiejar=mycookiejar) to avoid using the default global CookieJar. The 2 main functions for interacting with a CookieJar are Cookies.getcookies!, which returns a Vector{Cookie} for a given url (and will remove expired cookies from the jar), and Cookies.setcookies!, which will store "Set-Cookie" response headers in the cookie jar.

source
HTTP.Cookies.getcookies! — Function
Cookies.getcookies!(jar::CookieJar, url::URI)

Retrieve valid Cookies from the CookieJar according to the provided url. Cookies will be returned as a Vector{Cookie}. Only cookies for http or https scheme in the url will be returned. Cookies will be checked according to the canonical host of the url and any cookie max age or expiration will be accounted for. Expired cookies will not be returned and will be removed from the cookie jar.

source
HTTP.Cookies.setcookies! — Function
Cookies.setcookies!(jar::CookieJar, url::URI, headers::Headers)

Identify, "Set-Cookie" response headers from headers, parse the Cookies, and store valid entries in the cookie jar according to the canonical host in url. Cookies can be retrieved from the jar via Cookies.getcookies!.

source

Client-side Middleware (Layers)

HTTP.Layer — Type
Layer

Abstract type to represent a client-side middleware that exists for documentation purposes. A layer is any function of the form f(::Handler) -> Handler, where Handler is a function of the form f(::Request) -> Response. Note that the Handler definition is from the server-side documentation, and is "hard-coded" on the client side. It may also be apparent that a Layer is the same as the Middleware interface from server-side, which is true, but we define Layer to clarify the client-side distinction and its unique usage. Custom layers can be deployed in one of two ways:

  • HTTP.@client: Create a custom "client" with shorthand verb definitions, but which include custom layers; only these new verb methods will use the custom layers.
  • HTTP.pushlayer!/HTTP.poplayer!: Allows globally adding and removing layers from the default HTTP.jl layer stack; all http requests will then use the custom layers

Quick Examples

module Auth
 
 using HTTP
 
@@ -214,9 +214,9 @@
 HTTP.pushlayer!(Auth.auth_layer)
 
 # Now can use normal HTTP.jl methods and auth_layer will be included
-HTTP.get(url; authcreds=creds)
source
HTTP.@client — Macro
HTTP.@client requestlayers
+HTTP.get(url; authcreds=creds)
source
HTTP.@client — Macro
HTTP.@client requestlayers
 HTTP.@client requestlayers streamlayers
-HTTP.@client (first=requestlayers, last=requestlayers) (first=streamlayers, last=streamlayers)

Convenience macro for creating a custom HTTP.jl client that will include custom layers when performing requests. It's common to want to define a custom Layer to enhance a specific category of requests, such as custom authentcation for a web API. Instead of affecting the global HTTP.jl request stack via HTTP.pushlayer!, a custom wrapper client can be defined with convenient shorthand methods. See Layer for an example of defining a custom layer and creating a new client that includes the layer.

Custom layer arguments can be provided as a collection of request or stream-based layers; alternatively, a NamedTuple with keys first and last can be provided with values being a collection of layers. The NamedTuple form provides finer control over the order in which the layers will be included in the default http layer stack: first request layers are executed before all other layers, last request layers are executed right before all stream layers, and similarly for stream layers.

An empty collection can always be passed for request or stream layers when not needed.

One use case for custom clients is to control the value of standard HTTP.request keyword arguments. This can be achieved by passing a (first=[defaultkeywordlayer],) where defaultkeywordlayer is defined like:

defaultkeywordlayer(handler) = (req; kw...) -> handler(req; retry=false, redirect=false, kw...)

This client-side layer is basically a no-op as it doesn't modify the request at all, except that it hard-codes the value of the retry and redirect keyword arguments. When we pass this layer as (first=[defaultkeywordlayer],) this ensures this layer will be executed before all other layers, effectively over-writing the default and any user-provided keyword arguments for retry or redirect.

source
HTTP.pushlayer! — Function
HTTP.pushlayer!(layer; request=true)

Push a layer onto the stack of layers that will be applied to all requests. The "layer" is expected to be a function that takes and returns a Handler function. See Layer for more details. If request=false, the layer is expected to take and return a "stream" handler function. The custom layer will be put on the top of the stack, so it will be the first layer executed. To add a layer at the bottom of the stack, see HTTP.pushfirstlayer!.

source
HTTP.pushfirstlayer! — Function
HTTP.pushfirstlayer!(layer; request=true)

Push a layer to the start of the stack of layers that will be applied to all requests. The "layer" is expected to be a function that takes and returns a Handler function. See Layer for more details. If request=false, the layer is expected to take and return a "stream" handler function. The custom layer will be put on the bottom of the stack, so it will be the last layer executed. To add a layer at the top of the stack, see HTTP.pushlayer!.

source
HTTP.poplayer! — Function
HTTP.poplayer!(; request=true)

Inverse of HTTP.pushlayer!, removes the top layer of the global HTTP.jl layer stack. Can be used to "cleanup" after a custom layer has been added. If request=false, will remove the top "stream" layer as opposed to top "request" layer.

source
HTTP.popfirstlayer! — Function
HTTP.popfirstlayer!(; request=true)

Inverse of HTTP.pushfirstlayer!, removes the bottom layer of the global HTTP.jl layer stack. Can be used to "cleanup" after a custom layer has been added. If request=false, will remove the bottom "stream" layer as opposed to bottom "request" layer.

source
HTTP.MessageRequest.messagelayer — Function
messagelayer(handler) -> handler

Construct a Request object from method, url, headers, and body. Hard-coded as the first layer in the request pipeline.

source
HTTP.RedirectRequest.redirectlayer — Function
redirectlayer(handler) -> handler

Redirects the request in the case of 3xx response status.

source
HTTP.HeadersRequest.headerslayer — Function
headerslayer(handler) -> handler

Sets default expected headers.

source
HTTP.CookieRequest.cookielayer — Function
cookielayer(handler) -> handler

Check for host-appropriate cookies to include in the outgoing request from the cookiejar keyword argument (by default, a global cookiejar is used). Store "Set-Cookie" cookies from the response headers.

source
HTTP.TimeoutRequest.timeoutlayer — Function
timeoutlayer(handler) -> handler

Close the HTTP.Stream if no data has been received for readtimeout seconds.

source
HTTP.ExceptionRequest.exceptionlayer — Function
exceptionlayer(handler) -> handler

Throw a StatusError if the request returns an error response status.

source
HTTP.RetryRequest.retrylayer — Function
retrylayer(handler) -> handler

Retry the request if it throws a recoverable exception.

Base.retry and Base.ExponentialBackOff implement a randomised exponentially increasing delay is introduced between attempts to avoid exacerbating network congestion.

By default, requests that have a retryable body, where the request wasn't written or is idempotent will be retried. If the request is made and a response is received with a status code of 403, 408, 409, 429, or 5xx, the request will be retried.

retries controls the # of total retries that will be attempted.

retry_check allows passing a custom retry check in the case where the default retry check wouldn't retry, if retry_check returns true, then the request will be retried anyway.

source
HTTP.ConnectionRequest.connectionlayer — Function
connectionlayer(handler) -> handler

Retrieve an IO connection from the Connections.

Close the connection if the request throws an exception. Otherwise leave it open so that it can be reused.

source
HTTP.StreamRequest.streamlayer — Function
streamlayer(stream) -> HTTP.Response

Create a Stream to send a Request and body to an IO stream and read the response.

Send the Request body in a background task and begins reading the response immediately so that the transmission can be aborted if the Response status indicates that the server does not wish to receive the message body. RFC7230 6.5.

source

Raw Request Connection

HTTP.openraw — Function
HTTP.openraw(method, url, [, headers])::Tuple{Connection, Response}

Open a raw socket that is unmanaged by HTTP.jl. Useful for doing HTTP upgrades to other protocols. Any bytes of the body read from the socket when reading headers, is returned as excess bytes in the last tuple argument.

Example of a WebSocket upgrade:

headers = Dict(
+HTTP.@client (first=requestlayers, last=requestlayers) (first=streamlayers, last=streamlayers)

Convenience macro for creating a custom HTTP.jl client that will include custom layers when performing requests. It's common to want to define a custom Layer to enhance a specific category of requests, such as custom authentcation for a web API. Instead of affecting the global HTTP.jl request stack via HTTP.pushlayer!, a custom wrapper client can be defined with convenient shorthand methods. See Layer for an example of defining a custom layer and creating a new client that includes the layer.

Custom layer arguments can be provided as a collection of request or stream-based layers; alternatively, a NamedTuple with keys first and last can be provided with values being a collection of layers. The NamedTuple form provides finer control over the order in which the layers will be included in the default http layer stack: first request layers are executed before all other layers, last request layers are executed right before all stream layers, and similarly for stream layers.

An empty collection can always be passed for request or stream layers when not needed.

One use case for custom clients is to control the value of standard HTTP.request keyword arguments. This can be achieved by passing a (first=[defaultkeywordlayer],) where defaultkeywordlayer is defined like:

defaultkeywordlayer(handler) = (req; kw...) -> handler(req; retry=false, redirect=false, kw...)

This client-side layer is basically a no-op as it doesn't modify the request at all, except that it hard-codes the value of the retry and redirect keyword arguments. When we pass this layer as (first=[defaultkeywordlayer],) this ensures this layer will be executed before all other layers, effectively over-writing the default and any user-provided keyword arguments for retry or redirect.

source
HTTP.pushlayer! — Function
HTTP.pushlayer!(layer; request=true)

Push a layer onto the stack of layers that will be applied to all requests. The "layer" is expected to be a function that takes and returns a Handler function. See Layer for more details. If request=false, the layer is expected to take and return a "stream" handler function. The custom layer will be put on the top of the stack, so it will be the first layer executed. To add a layer at the bottom of the stack, see HTTP.pushfirstlayer!.

source
HTTP.pushfirstlayer! — Function
HTTP.pushfirstlayer!(layer; request=true)

Push a layer to the start of the stack of layers that will be applied to all requests. The "layer" is expected to be a function that takes and returns a Handler function. See Layer for more details. If request=false, the layer is expected to take and return a "stream" handler function. The custom layer will be put on the bottom of the stack, so it will be the last layer executed. To add a layer at the top of the stack, see HTTP.pushlayer!.

source
HTTP.poplayer! — Function
HTTP.poplayer!(; request=true)

Inverse of HTTP.pushlayer!, removes the top layer of the global HTTP.jl layer stack. Can be used to "cleanup" after a custom layer has been added. If request=false, will remove the top "stream" layer as opposed to top "request" layer.

source
HTTP.popfirstlayer! — Function
HTTP.popfirstlayer!(; request=true)

Inverse of HTTP.pushfirstlayer!, removes the bottom layer of the global HTTP.jl layer stack. Can be used to "cleanup" after a custom layer has been added. If request=false, will remove the bottom "stream" layer as opposed to bottom "request" layer.

source
HTTP.MessageRequest.messagelayer — Function
messagelayer(handler) -> handler

Construct a Request object from method, url, headers, and body. Hard-coded as the first layer in the request pipeline.

source
HTTP.RedirectRequest.redirectlayer — Function
redirectlayer(handler) -> handler

Redirects the request in the case of 3xx response status.

source
HTTP.HeadersRequest.headerslayer — Function
headerslayer(handler) -> handler

Sets default expected headers.

source
HTTP.CookieRequest.cookielayer — Function
cookielayer(handler) -> handler

Check for host-appropriate cookies to include in the outgoing request from the cookiejar keyword argument (by default, a global cookiejar is used). Store "Set-Cookie" cookies from the response headers.

source
HTTP.TimeoutRequest.timeoutlayer — Function
timeoutlayer(handler) -> handler

Close the HTTP.Stream if no data has been received for readtimeout seconds.

source
HTTP.ExceptionRequest.exceptionlayer — Function
exceptionlayer(handler) -> handler

Throw a StatusError if the request returns an error response status.

source
HTTP.RetryRequest.retrylayer — Function
retrylayer(handler) -> handler

Retry the request if it throws a recoverable exception.

Base.retry and Base.ExponentialBackOff implement a randomised exponentially increasing delay is introduced between attempts to avoid exacerbating network congestion.

By default, requests that have a retryable body, where the request wasn't written or is idempotent will be retried. If the request is made and a response is received with a status code of 403, 408, 409, 429, or 5xx, the request will be retried.

retries controls the # of total retries that will be attempted.

retry_check allows passing a custom retry check in the case where the default retry check wouldn't retry, if retry_check returns true, then the request will be retried anyway.

source
HTTP.ConnectionRequest.connectionlayer — Function
connectionlayer(handler) -> handler

Retrieve an IO connection from the Connections.

Close the connection if the request throws an exception. Otherwise leave it open so that it can be reused.

source
HTTP.StreamRequest.streamlayer — Function
streamlayer(stream) -> HTTP.Response

Create a Stream to send a Request and body to an IO stream and read the response.

Send the Request body in a background task and begins reading the response immediately so that the transmission can be aborted if the Response status indicates that the server does not wish to receive the message body. RFC7230 6.5.

source

Raw Request Connection

HTTP.openraw — Function
HTTP.openraw(method, url, [, headers])::Tuple{Connection, Response}

Open a raw socket that is unmanaged by HTTP.jl. Useful for doing HTTP upgrades to other protocols. Any bytes of the body read from the socket when reading headers, is returned as excess bytes in the last tuple argument.

Example of a WebSocket upgrade:

headers = Dict(
     "Upgrade" => "websocket",
     "Connection" => "Upgrade",
     "Sec-WebSocket-Key" => "dGhlIHNhbXBsZSBub25jZQ==",
@@ -226,4 +226,4 @@
 
 # Write a WebSocket frame
 frame = UInt8[0x81, 0x85, 0x37, 0xfa, 0x21, 0x3d, 0x7f, 0x9f, 0x4d, 0x51, 0x58]
-write(socket, frame)
source
HTTP.Connections.Connection — Type
Connection

A Sockets.TCPSocket, MbedTLS.SSLContext or OpenSSL.SSLStream connection to a HTTP host and port.

Fields:

  • host::String
  • port::String, exactly as specified in the URI (i.e. may be empty).
  • idle_timeout, No. of seconds to maintain connection after last request/response.
  • require_ssl_verification, whether ssl verification is required for an ssl connection
  • keepalive, whether the tcp socket should have keepalive enabled
  • peerip, remote IP adress (used for debug/log messages).
  • peerport, remote TCP port number (used for debug/log messages).
  • localport, local TCP port number (used for debug messages).
  • io::T, the Sockets.TCPSocket, MbedTLS.SSLContext or OpenSSL.SSLStream.
  • clientconnection::Bool, whether the Connection was created from client code (as opposed to server code)
  • buffer::IOBuffer, left over bytes read from the connection after the end of a response header (or chunksize). These bytes are usually part of the response body.
  • timestamp, time data was last received.
  • readable, whether the Connection object is readable
  • writable, whether the Connection object is writable
source

Parser Interface

HTTP.Parsers.find_end_of_header — Function
find_end_of_header(bytes) -> length or 0

Find length of header delimited by \r\n\r\n or \n\n.

source
HTTP.Parsers.find_end_of_chunk_size — Function

Find \n after chunk size in bytes.

source
HTTP.Parsers.find_end_of_trailer — Function
find_end_of_trailer(bytes) -> length or 0

Find length of trailer delimited by \r\n\r\n (or starting with \r\n). RFC7230 4.1

source
HTTP.Parsers.parse_status_line! — Function

Parse HTTP response-line bytes and set the status and version fields of response. Return a SubString containing the header-field lines.

source
HTTP.Parsers.parse_request_line! — Function

Parse HTTP request-line bytes and set the method, target and version fields of request. Return a SubString containing the header-field lines.

source
HTTP.Parsers.parse_header_field — Function

Parse HTTP header-field. Return Pair(field-name => field-value) and a SubString containing the remaining header-field lines.

source
HTTP.Parsers.parse_chunk_size — Function

Parse HTTP chunk-size. Return number of bytes of chunk-data.

chunk-size = 1*HEXDIG

RFC7230 4.1

source
+write(socket, frame)source
HTTP.Connections.Connection — Type
Connection

A Sockets.TCPSocket, MbedTLS.SSLContext or OpenSSL.SSLStream connection to a HTTP host and port.

Fields:

  • host::String
  • port::String, exactly as specified in the URI (i.e. may be empty).
  • idle_timeout, No. of seconds to maintain connection after last request/response.
  • require_ssl_verification, whether ssl verification is required for an ssl connection
  • keepalive, whether the tcp socket should have keepalive enabled
  • peerip, remote IP adress (used for debug/log messages).
  • peerport, remote TCP port number (used for debug/log messages).
  • localport, local TCP port number (used for debug messages).
  • io::T, the Sockets.TCPSocket, MbedTLS.SSLContext or OpenSSL.SSLStream.
  • clientconnection::Bool, whether the Connection was created from client code (as opposed to server code)
  • buffer::IOBuffer, left over bytes read from the connection after the end of a response header (or chunksize). These bytes are usually part of the response body.
  • timestamp, time data was last received.
  • readable, whether the Connection object is readable
  • writable, whether the Connection object is writable
source

Parser Interface

HTTP.Parsers.find_end_of_header — Function
find_end_of_header(bytes) -> length or 0

Find length of header delimited by \r\n\r\n or \n\n.

source
HTTP.Parsers.find_end_of_chunk_size — Function

Find \n after chunk size in bytes.

source
HTTP.Parsers.find_end_of_trailer — Function
find_end_of_trailer(bytes) -> length or 0

Find length of trailer delimited by \r\n\r\n (or starting with \r\n). RFC7230 4.1

source
HTTP.Parsers.parse_status_line! — Function

Parse HTTP response-line bytes and set the status and version fields of response. Return a SubString containing the header-field lines.

source
HTTP.Parsers.parse_request_line! — Function

Parse HTTP request-line bytes and set the method, target and version fields of request. Return a SubString containing the header-field lines.

source
HTTP.Parsers.parse_header_field — Function

Parse HTTP header-field. Return Pair(field-name => field-value) and a SubString containing the remaining header-field lines.

source
HTTP.Parsers.parse_chunk_size — Function

Parse HTTP chunk-size. Return number of bytes of chunk-data.

chunk-size = 1*HEXDIG

RFC7230 4.1

source
diff --git a/previews/PR1116/search/index.html b/previews/PR1116/search/index.html index 68ca9f6fd..23361735a 100644 --- a/previews/PR1116/search/index.html +++ b/previews/PR1116/search/index.html @@ -1,2 +1,2 @@ -Search · HTTP.jl

Loading search...

    +Search · HTTP.jl

    Loading search...

      diff --git a/previews/PR1116/server/index.html b/previews/PR1116/server/index.html index 7b693c2e3..49a105cca 100644 --- a/previews/PR1116/server/index.html +++ b/previews/PR1116/server/index.html @@ -1,4 +1,4 @@ Server · HTTP.jl
      Edit on GitHub

      Server

      For server-side functionality, HTTP.jl provides a robust framework for core HTTP and websocket serving, flexible handler and middleware interfaces, and low-level access for unique workflows. The core server listening code is in the /src/Servers.jl file, while the handler, middleware, router, and higher level HTTP.serve function are defined in the /src/Handlers.jl file.

      HTTP.serve

      HTTP.serve/HTTP.serve! are the primary entrypoints for HTTP server functionality, while HTTP.listen/HTTP.listen! are considered the lower-level core server loop methods that only operate directly with HTTP.Streams. HTTP.serve is also built directly integrated with the Handler and Middleware interfaces and provides easy flexibility by doing so. The signature is:

      HTTP.serve(f, host, port; kw...)
 HTTP.serve!(f, host, port; kw...) -> HTTP.Server

      Where f is a Handler function, typically of the form f(::Request) -> Response, but can also operate directly on an HTTP.Stream of the form f(::Stream) -> Nothing while also passing stream=true to the keyword arguments. The host argument should be a String, or IPAddr, created like ip"0.0.0.0". port should be a valid port number as an Integer. HTTP.serve is the blocking server method, whereas HTTP.serve! is non-blocking and returns the listening HTTP.Server object that can be close(server)ed manually.

      Supported keyword arguments include:

      • sslconfig=nothing, Provide an MbedTLS.SSLConfig object to handle ssl connections. Pass sslconfig=MbedTLS.SSLConfig(false) to disable ssl verification (useful for testing). Construct a custom SSLConfig object with MbedTLS.SSLConfig(certfile, keyfile).
      • tcpisvalid = tcp->true, function f(::TCPSocket)::Bool to check if accepted connections are valid before processing requests. e.g. to do source IP filtering.
      • readtimeout::Int=0, close the connection if no data is received for this many seconds. Use readtimeout = 0 to disable.
      • reuseaddr::Bool=false, allow multiple servers to listen on the same port. Not supported on some OS platforms. Can check HTTP.Servers.supportsreuseaddr().
      • server::Base.IOServer=nothing, provide an IOServer object to listen on; allows manually closing or configuring the server socket.
      • verbose::Union{Int,Bool}=false, log connection information to stdout. Use -1 to also silence the server start and stop logs.
      • access_log::Function, function for formatting access log messages. The function should accept two arguments, io::IO to which the messages should be written, and http::HTTP.Stream which can be used to query information from. See also @logfmt_str.
      • on_shutdown::Union{Function, Vector{<:Function}, Nothing}=nothing, one or more functions to be run if the server is closed (for example by an InterruptException). Note, shutdown function(s) will not run if an IOServer object is supplied to the server keyword argument and closed by close(server).

      HTTP.Handler

      Abstract type for the handler interface that exists for documentation purposes. A Handler is any function of the form f(req::HTTP.Request) -> HTTP.Response. There is no requirement to subtype Handler and users should not rely on or dispatch on Handler. A Handler function f can be passed to HTTP.serve wherein a server will pass each incoming request to f to be handled and a response to be returned. Handler functions are also the inputs to Middleware functions which are functions of the form f(::Handler) -> Handler, i.e. they take a Handler function as input, and return a "modified" or enhanced Handler function.

      For advanced cases, a Handler function can also be of the form f(stream::HTTP.Stream) -> Nothing. In this case, the server would be run like HTTP.serve(f, ...; stream=true). For this use-case, the handler function reads the request and writes the response to the stream directly. Note that any middleware used with a stream handler also needs to be of the form f(stream_handler) -> stream_handler, i.e. it needs to accept a stream Handler function and return a stream Handler function.

      HTTP.Middleware

      Abstract type for the middleware interface that exists for documentation purposes. A Middleware is any function of the form f(::Handler) -> Handler (ref: Handler). There is no requirement to subtype Middleware and users should not rely on or dispatch on the Middleware type. While HTTP.serve(f, ...) requires a handler function f to be passed, middleware can be "stacked" to create a chain of functions that are called in sequence, like HTTP.serve(base_handler |> cookie_middleware |> auth_middlware, ...), where the base_handler Handler function is passed to cookie_middleware, which takes the handler and returns a "modified" handler (that parses and stores cookies). This "modified" handler is then an input to the auth_middlware, which further enhances/modifies the handler.

      HTTP.Router

      Object part of the Handler framework for routing requests based on path matching registered routes.

      r = HTTP.Router(_404, _405)

      Define a router object that maps incoming requests by path to registered routes and associated handlers. Paths can be registered using HTTP.register!. The router object itself is a "request handler" that can be called like:

      r = HTTP.Router()
-resp = r(request)

      Which will inspect the request, find the matching, registered handler from the url, and pass the request on to be handled further.

      See HTTP.register! for additional information on registering handlers based on routes.

      If a request doesn't have a matching, registered handler, the _404 handler is called which, by default, returns a HTTP.Response(404). If a route matches the path, but not the method/verb (e.g. there's a registered route for "GET /api", but the request is "POST /api"), then the _405 handler is called, which by default returns HTTP.Response(405) (method not allowed).

      HTTP.listen

      Lower-level core server functionality that only operates on HTTP.Stream. Provides a level of separation from HTTP.serve and the Handler framework. Supports all the same arguments and keyword arguments as HTTP.serve, but the handler function f must take a single HTTP.Stream as argument. HTTP.listen! is the non-blocking counterpart to HTTP.listen (like HTTP.serve! is to HTTP.serve).

      Log formatting

      Nginx-style log formatting is supported via the HTTP.@logfmt_str macro and can be passed via the access_log keyword argument for HTTP.listen or HTTP.serve.

      Serving on the interactive thead pool

      Beginning in Julia 1.9, the main server loop is spawned on the interactive threadpool by default. If users do a Threads.@spawn from a handler, those threaded tasks should run elsewhere and not in the interactive threadpool, keeping the web server responsive.

      Note that just having a reserved interactive thread doesn’t guarantee CPU cycles, so users need to properly configure their running Julia session appropriately (i.e. ensuring non-interactive threads available to run tasks, etc).

      +resp = r(request)

      Which will inspect the request, find the matching, registered handler from the url, and pass the request on to be handled further.

      See HTTP.register! for additional information on registering handlers based on routes.

      If a request doesn't have a matching, registered handler, the _404 handler is called which, by default, returns a HTTP.Response(404). If a route matches the path, but not the method/verb (e.g. there's a registered route for "GET /api", but the request is "POST /api"), then the _405 handler is called, which by default returns HTTP.Response(405) (method not allowed).

      HTTP.listen

      Lower-level core server functionality that only operates on HTTP.Stream. Provides a level of separation from HTTP.serve and the Handler framework. Supports all the same arguments and keyword arguments as HTTP.serve, but the handler function f must take a single HTTP.Stream as argument. HTTP.listen! is the non-blocking counterpart to HTTP.listen (like HTTP.serve! is to HTTP.serve).

      Log formatting

      Nginx-style log formatting is supported via the HTTP.@logfmt_str macro and can be passed via the access_log keyword argument for HTTP.listen or HTTP.serve.

      Serving on the interactive thead pool

      Beginning in Julia 1.9, the main server loop is spawned on the interactive threadpool by default. If users do a Threads.@spawn from a handler, those threaded tasks should run elsewhere and not in the interactive threadpool, keeping the web server responsive.

      Note that just having a reserved interactive thread doesn’t guarantee CPU cycles, so users need to properly configure their running Julia session appropriately (i.e. ensuring non-interactive threads available to run tasks, etc).

      diff --git a/previews/PR1116/websockets/index.html b/previews/PR1116/websockets/index.html index 59315d7c4..989d505ad 100644 --- a/previews/PR1116/websockets/index.html +++ b/previews/PR1116/websockets/index.html @@ -30,4 +30,4 @@ # simple echo server send(ws, msg) end -end +end