When using Pluto, output from pyprint is buffered

[klaff]

From within Pluto, pyprint("hello world!") does not immediately print to the terminal running the Pluto server. Also, if one uses PlutoUI and with_terminal() do ..., the output also does not immediately print.

If the notebook is stopped, or if one pyprints a large number of bytes (~ 10 kB?) then the buffered output of the pyprint statements will be displayed.

To replicate, start a Pluto notebook by using Pluto; Pluto.run(), then create a new notebook with one cell containing using PythonCall and one containing pyprint("hello world!").

[klaff]

Similar behavior occurs with PyCall - this may be a Pluto issue.

[klaff]

Closing on the assumption it is a Pluto bug: https://github.com/fonsp/Pluto.jl/issues/1656

[schlichtanders]

@klaff Please reopen

The problem is still existing and the linked Pluto issues explicitly states that this is not a Pluto bug, but a bad interaction of PythonCall and Distributed printing.

[schlichtanders]

I adapted the example to show what is missing:

julia> using Distributed

julia> p = Distributed.addprocs(1) |> first
2

julia> Distributed.remotecall_eval(Main, p, :(using PythonCall))

julia> Distributed.remotecall_eval(Main, p, :(print("hello Julia!")))

julia> Distributed.remotecall_eval(Main, p, :(@info("flush")))
From worker 2:    hello Julia![ Info: flush

julia> Distributed.remotecall_eval(Main, p, :(pyprint("hello PythonCall!")))

julia> Distributed.remotecall_eval(Main, p, :(@info("flush")))
From worker 2:    [ Info: flush

julia> Distributed.rmprocs(p)
From worker 2:    hello PythonCall!
Task (done) @0x00007f96a3589aa0

The "hello PythonCall!" output should be printed together with "flush".

[schlichtanders]

Found a workaround: Explicitly call flush from python side

pyprint("hello world", flush=true)

[cjdoris]

Yes indeed, the issue is that in Python stdout has a large buffer by default (unless it's an interactive session and it line buffers too) which isn't being flushed as often as you'd like. You just need to flush it yourself.

I suppose that if PythonCall could detect that it's running inside a Distributed worker and that the main process is interactive, then it could force Python's stdout to line buffer. Sounds tricky though.