Can't run example code in v1.6.0

[chelyabinsk]

Hi

This might be a "read the f-ing manual" problem, so apologies in advance. I'm using Julia 1.8.2 and I tried running the below code

using WebSockets
import JSON


const url = "wss://ws.kraken.com/"

WebSockets.open(url) do ws_client  
    # msg = "{\"event\":\"subscribe\", \"subscription\":{\"name\":\"trade\"}, \"pair\":[\"XBT/USD\",\"XRP/USD\"]}"

    msg = Dict(
        "event" => "subscribe", 
        "subscription" => Dict("name" => "trade"),
        "pair" => ["XBT/USDT", "XBT/USD"]
    )
    msg = JSON.json(msg)

    writeguarded(ws_client, msg)

    while (true)
        data, success = readguarded(ws_client)
        if success
            println(stderr, ws_client, " received:", String(data))
        end
            
    end
end;

When I use v1.5.9 it works fine, but with v1.6.0 I get no output.

I also tried this code

using WebSockets, JSON

uri = "wss://www.bitmex.com/realtime"

payload = Dict(
               :op => "subscribe",
               :args => "trade:XBT"
           )

function open_websocket() 
   WebSockets.open(uri) do ws
     if isopen(ws)
      try
       write(ws, JSON.json(payload))
      catch e
        println(e)
      end
     end

     while isopen(ws)
       data, success = readguarded(ws)
       if success
         data = JSON.parse(String(data))
         print(data, "\n")
       end
     end

     if !isopen(ws)
       @async open_websocket()
     end

   end
 end
       
open_websocket()

Which works in v1.5.9 but in v.1.6.0 I get the following error

ErrorException("OpenSSL.SSLStream does not support byte I/O")

I noticed that when I downgrade down to v1.5.9 HTTP library is also downgraded quite significantly. Here is the log after I run add [email protected]

(@v1.8) pkg> add [email protected]
   Resolving package versions...
    Updating `~/.julia/environments/v1.8/Project.toml`
⌅ [cd3eb016] ↓ HTTP v1.9.6 ⇒ v0.9.17
⌃ [104b5d7c] ↓ WebSockets v1.6.0 ⇒ v1.5.9
    Updating `~/.julia/environments/v1.8/Manifest.toml`
  [d1d4a3ce] - BitFlags v0.1.7
  [944b1d66] - CodecZlib v0.7.1
  [f0e56b4a] - ConcurrentUtilities v2.2.0
⌅ [cd3eb016] ↓ HTTP v1.9.6 ⇒ v0.9.17
  [83e8ac13] + IniFile v0.5.1
  [e6f89c97] - LoggingExtras v1.0.0
  [4d8831e6] - OpenSSL v1.4.1
  [777ac1f9] - SimpleBufferStream v1.1.0
  [3bb67fe8] - TranscodingStreams v0.9.13
⌃ [104b5d7c] ↓ WebSockets v1.6.0 ⇒ v1.5.9

I'm happy to continue using v1.5.9, but I wonder if anyone else had the same issue. Feel free to ask any clarifying questions :)

[hustf]

Hello, chelyabinsk. A surprisingly large part of Julia's ecosystem depend on HTTP.jl, so relying on older versions of it is likely to have consequences. If this is a problem we can fix, we ought to.

I doubt this is a 'read the manual' issue; I get an error, too. I modified your first example to harvest error messages:

using WebSockets
import JSON


const url = "wss://ws.kraken.com/"

WebSockets.open(url) do ws_client  
    # msg = "{\"event\":\"subscribe\", \"subscription\":{\"name\":\"trade\"}, \"pair\":[\"XBT/USD\",\"XRP/USD\"]}"
    @wslog ws_client
    msg = Dict(
        "event" => "subscribe", 
        "subscription" => Dict("name" => "trade"),
        "pair" => ["XBT/USDT", "XBT/USD"]
    )
    msg = JSON.json(msg)
    @wslog msg
    writeguarded(ws_client, msg)
    @wslog "msg sent"
end

Note that we're just trying to subscribe and then close the websocket from our side. But a similar "not support byte I/O" pops up after roughly ten seconds, which may indicate this error during closing handshake's "reasonable" waiting period.

Here's the output:

(@ws_iss186) pkg> status
Status `C:\Users\f\.julia\environments\ws_iss186\Project.toml`
  [682c06a0] JSON v0.21.4
  [104b5d7c] WebSockets v1.6.0

julia> include("issue186.jl")
[ LogLevel(50): WebSocket{SSLStream}(client, CONNECTED)
[ LogLevel(50): {"event":"subscribe","subscription":{"name":"trade"},"pair":["XBT/USDT","XBT/USD"]}
[ LogLevel(50): msg sent

ERROR: LoadError: HTTP.Exceptions.RequestError(HTTP.Messages.Request:
"""
GET / HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: 7IiGZ0Kc1OPdH+GYMsvB4g==
Sec-WebSocket-Version: 13
Host: ws.kraken.com
Accept: */*
User-Agent: HTTP.jl/1.9.0-beta4
Accept-Encoding: gzip

[Message Body was streamed]""", ErrorException("OpenSSL.SSLStream does not support byte I/O"))
Stacktrace:
...
....
 [16] open(f::Function, url::String; verbose::Bool, subprotocol::String, kw::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ WebSockets C:\Users\f\.julia\packages\WebSockets\rEB2c\src\HTTP.jl:89
 [17] open(f::Function, url::String)
    @ WebSockets C:\Users\f\.julia\packages\WebSockets\rEB2c\src\HTTP.jl:68
 [18] top-level scope
    @ C:\Users\f\.julia\environments\ws_iss186\iss186.jl:7

We can now dive into the test suite to find where it closes a wss connection, and see if that works. After all, "wss://ws.kraken.com/" might do something awry too. I need to run now, but hope you are willing to investigate more.

[chelyabinsk]

Thank you for your response, it was really helpful! I managed to "fix" it by using HTTP.jl package explicitly. The following code works in Julia 1.9.0 now.

using JSON, HTTP

uri = "wss://www.bitmex.com/realtime"

payload = Dict(
               :op => "subscribe",
               :args => "trade:XBT"
           )

verbose = false
HTTP.WebSockets.open(uri; verbose, suppress_close_error=true) do ws
    HTTP.send(ws, JSON.json(payload))
    for msg in ws
        println(msg)
    end
end

(@v1.9) pkg> status
Status `~/.julia/environments/v1.9/Project.toml`
  [cd3eb016] HTTP v1.9.6
  [682c06a0] JSON v0.21.4
  [104b5d7c] WebSockets v1.6.0

I am not sure why this works, but luckily it's not a big change.

[alfa33333]

I am having the same situation with v1.6.0. OpenSSL.SSLStream does not support byte I/O.

In other cases, I just get a false when trying to read the websocket

WebSockets.open("wss://www.bitmex.com/realtime?subscribe=trade:XBT") do ws
           data, success  = readguarded(ws)
           message = String(data)
           println(message)
           println(success)
           # error("This is an error")
       end

false
HTTP.Messages.Response:
"""
HTTP/1.1 101 Switching Protocols
Date: Sun, 20 Aug 2023 14:52:52 GMT
Connection: upgrade
upgrade: websocket
sec-websocket-accept: 4ukLODpquaylbpP/e113XowIfdI=
CF-Cache-Status: DYNAMIC
Set-Cookie: __cf_bm=5eQXe7.8LHE3eYCkcUTYYLz03gDkWvCiiv64t9WzxmU-1692543172-0-AVbGSWXry/CAzqoN5TyqEFm0KhCbRYkJ0Dyv7QXdMpEKDzZW8A4qhEd/OfJYokbK6KlK/Tcod/DHdevZ2BmKrpY=; path=/; expires=Sun, 20-Aug-23 15:22:52 GMT; domain=.bitmex.com; HttpOnly; Secure; SameSite=None
Server: cloudflare
CF-RAY: 7f9b76675b4439d8-YYZ
alt-svc: h3=":443"; ma=86400

"""

This code works returns true and the welcome message when I tried it 1.5.9 in the same machine. I am using Windows 11 and Julia 1.8.2 if that helps.

[hustf]

Your citation is not unambiguous, but I take it that you read, without success. The message is (as it should be) empty. readguatded suppresses the actual error message.

This behaviour is nice. It could perhaps be "dangerous" to try and interpret an incomplete row of bytes as a string. So check for success first, and if you have it, translate the bytes you got into a string.

Readguarded is meant for stability in case you communicate over radio or similar. Details of how a connection broke are uninteresting most times.

But in this case, we would like to know exactly why things fail so we can fix it. So use read instead, and throw errors all over your screen. I can recommend abbreviatedstacktraces.jl, though I have not tried it with HTTP.jl. That could alleviate the pain of reading the full and crazy long stack traces.

[alfa33333]

The read is without success, when it shouldn't. I found that you need to specify HTTP.jl 1.5 or 1.6, and then everything works without issues. Adding WebSockets 1.6.0 adds HTTP 1.9 automatically, but I think this version is not not supported?