Question about reading WebSocket streams

[BoZenKhaa]

I am learning about streams and WebSockets and I can't figure out why the !eof in the following example runs the body of the loop twice:

#close(server)

server = Sockets.listen(Sockets.localhost, UInt16(8098))
@async HTTP.WebSockets.listen("1.0.0.1", UInt16(1001); verbose=true, server = server) do ws
    i=0
    while !eof(ws) 
        i+=1
        println(i)
        data = readavailable(ws)  
        read_data=String(data)
        if length(read_data)>0
            println("Msg (parsed): ", JSON.parse(read_data))
        else
            println("Zero length data: ", data)
        end
        response = json(Dict("response"=>"ok"))
        write(ws, response)
    end
end

I see this by running:

HTTP.WebSockets.open("ws://127.0.0.1:8098") do ws
    write(ws, json(Dict("message"=>"Hello")))
    x = readavailable(ws)
    sleep(3)
    println(String(x))
end;

Which prints:

1
Msg (parsed): Dict{String, Any}("message" => "Hello")
{"response":"ok"}
2
Zero length data: UInt8[]

I am having trouble replicating this behavior with other kinds of streams:

julia> i=0
0

julia> while !(eof(bf))
               i+=1
               println(i)
               data = readavailable(bf)
               println(String(data))
       end
1
Hello

I ran into this when getting cryptic messages while trying to parse the empty values as json and getting cryptic error messages.

Versions:

• Julia 1.7.2
• HTTP.jl 0.9.17
• MbedTLS.jl 1.0.3

[quinnj]

The issue here is that when the client finishes writing, the server receives that message correctly, but then the client connection "closes" (i.e. the WebSockets.open call finishes) and a CLOSE frame is sent to the server. These "control" frames are not meant to be user-facing, but are more internal to the state of the websocket connection. Unfortunately, the old websockets code would return a CLOSE frame as an empty message to the user.

I just merged an overhaul of the websockets code and this is now more correct in that CLOSE frames are not returned directly to the user as messages, so we get this behavior now:

julia> @async HTTP.WebSockets.listen(Sockets.localhost, 8098; verbose=true, server = server) do ws
           i=0
           for msg in ws
               i += 1
               println(i)
               println(JSON.parse(msg))
               send(ws, JSON.json(Dict("response" => "ok")))
           end
       end
[ Info: Listening on: 127.0.0.1:8098
Task (runnable) @0x000000010f59a120

julia> HTTP.WebSockets.open("ws://127.0.0.1:8098") do ws
           send(ws, json(Dict("message"=>"Hello")))
           msg = receive(ws)
           sleep(3)
           println(msg)
       end;
1
Dict{String, Any}("message" => "Hello")
{"response":"ok"}
┌ LogLevel(-1001): ✋  "Connection: close": 💀    0s 127.0.0.1:8098:8098 RawFD(4294967295)
└ @ HTTP.Streams ~/.julia/dev/HTTP/src/Streams.jl:137

[BoZenKhaa]

Thank you very much for an answer and a fix! I spent some time looking at the sources, but being new to streams, async, and websockets, I was just scratching my head...