redirect stdin to frontends

[stevengj]

We should use @loladiro's new redirect_stdin request to turn STDIN into a zmq socket: when something is trying to read from stdin, we send an input_request to the front-end, and write the resulting input_reply (which will apparently be sent one line at a time) to stdin.

Update: readline(STDIN) works in IJulia, but it's just a hack; other stdin reads won't work.

[stevengj]

@loladiro, once I've done rd, wr = redirect_stdin(), I want to write an async task that blocks until something tries to read from STDIN. How do I do this? wait(rd.readnotify)? Somehow wait for the status to become StatusActive?

[Keno]

I'm actually not sure that's possible (assuming you also want this to work for things trying to read the file descriptor. Do you know of a C API that does this?

[stevengj]

I don't know, but that doesn't mean much as I almost never work with asynchronous I/O.

At the very least, we could make it work for ordinary Julia-based I/O: start_reading could notify a writenotify condition that IJulia could wait on.

[stevengj]

Also, we may need a way to throw an exception to the task that is attempting to read from STDIN if the current front-end does not allow stdin. (Although this may not be an issue anymore as @minrk apparently added stdin support to the notebook a few months ago, and we only support IPython 1.0 anyway.)

[stevengj]

Hmm, doing a notify(stream.writenotify) in Base.start_reading does not actually seem to be the right thing. We only want to send an input_request message to IPython if a task is blocked on reading STDIN, not in cases where it starts reading and the buffer already contains the requisite bytes.

@loladiro or @JeffBezanson, any suggestions on how to make this work for stdin from Julia tasks (never mind about external libraries for now)?

[stevengj]

(I'd like to at least get readline working, as BinDeps uses this.)

[StefanKarpinski]

Prompting the user is a legacy behavior in BinDeps – up-to-date packages shouldn't do that.

[stevengj]

I see; still, I'd like to have some idea of how we can handle stdin redirection before the 0.2 feature-freeze happens.

[toivoh]

Until readline works, is there any way to get textual user input while running in IJulia? Interactive debugging with the Debug package will not work without it, see toivoh/Debug.jl#52.

[stevengj]

Well, you can always type textual input into the terminal where you launched ipython notebook --profile=julia, since that is where it is waiting for STDIN....

It looks like the best thing for now would be to give up on a general solution for redirecting all stdin, due to the obstacles described above, and just hack it to make readline(STDIN) work in Julia only....

[toivoh]

Sounds good to me.

[stevengj]

@minrk, @Carreau, I've hacked IJulia so that readline(STDIN) sends an input_request to the stdin socket, but nothing is happening. Where am I supposed to see the input prompt? I tried it in both the notebook and qtconsole front-ends, and I'm not seeing anything.

What am I missing?

This is the log of what I am sending (prior to translation into JSON format):

SENDING IPython Msg [ idents input_request ] {
  header = ["msg_type"=>"input_request","msg_id"=>"6a54c793-6052-4172-8a56-d11a9fa2e106","username"=>"username","session"=>"61AC7FA67E90428DB5192493DF5FF3EA"],
  metadata = Dict{String,Any}(),
  content = ["prompt"=>"STDIN> "]
}

[quinnj]

Not sure why the notebook itself wouldn't support it, but in the messaging protocol, there is this little blurb I remember reading:

# Some frontends (e.g. the Notebook) do not support stdin requests. If
# raw_input is called from code executed from such a frontend, a
# StdinNotImplementedError will be raised.
'allow_stdin' : True,

In the execute_request message.

[stevengj]

Except allow_stdin is true in the execute request:

RECEIVED IPython Msg [ idents 61AC7FA67E90428DB5192493DF5FF3EA ] {
  header = ["session"=>"61AC7FA67E90428DB5192493DF5FF3EA","msg_id"=>"20C04BD8D5E94D4D86948D14FA846940","username"=>"username","msg_type"=>"execute_request"],
  metadata = Dict{String,Any}(),
  content = ["user_variables"=>{},"store_history"=>true,"silent"=>false,"code"=>"readline(STDIN)","user_expressions"=>Dict{String,Any}(),"allow_stdin"=>true]
}

[minrk]

Not sure why the notebook itself wouldn't support it, but in the messaging protocol, there is this little blurb I remember reading:

That key was added because the notebook didn't support stdin requests, but it does in 1.0, so that wouldn't be the issue.

@stevengj can I see what you are working on? The most likely way to lose the message is that you must specify the same zmq IDENTITY prefix on the stdin_request as you would the execute_reply.

[stevengj]

@minrk, thanks that fixed it! For future reference, could you clarify this in the messaging-protocol docs for input_request?

[stevengj]

With the latest commit, readline() should work.

[minrk]

note added in messaging docs