Bug #15367
IO.select is not resumed when io-object gets closed
Description
Here is sample code:
rp, wp = IO.pipe t2 = Thread.new { IO.select([rp]) } # This also does not work: # t2 = Thread.new { IO.select([rp], nil, [rp]) } sleep 0.01 rp.close t2 # => #<Thread:0x00000000089b6ce0@(pry):51 sleep>
It happens only on linux, tested with 2.5.1, 2.6.0-preview2. On macOS it gives error, as expected:
#<Thread:0x00007fab3aebce58@(pry):5 run> terminated with exception (report_on_exception is true): Traceback (most recent call last): 1: from (pry):5:in `block in <main>' (pry):5:in `select': Bad file descriptor (Errno::EBADF) > t2 => #<Thread:0x00007fab3aebce58@(pry):5 dead>
Updated by normalperson (Eric Wong) over 1 year ago
melentievm@gmail.com wrote:
It happens only on linux, tested with 2.5.1, 2.6.0-preview2. On macOS it
gives error, as expected:
Right, it's platform-specific. I brought it up a few months ago
and can implement it; but it'll add some overhead
(current IO#close during IO#write/IO#write already does).
akr (Akira Tanaka) doesn't want it, matz (Yukihiro Matsumoto) hasn't responded, yet.
Updated by shevegen (Robert A. Heiler) over 1 year ago
If I may inquire, how significant would the overhead be?
While I think Akira's comment is perfectly fine on its own, I
feel that if other people notice a different behaviour between
different operating systems with regards to IO.select then this
may surprise them. (I myself use almost exclusively Linux.)
(On a not-so-relevant comment, my first project in ruby was an
IRC bot and that was also when I used IO.select for the first
time.)
Updated by MSP-Greg (Greg L) over 1 year ago
On Windows (MinGW), the thread is also sleeping...
Updated by normalperson (Eric Wong) over 1 year ago
shevegen@gmail.com wrote:
If I may inquire, how significant would the overhead be?
I'd have to implement it to know for sure...
Thread::Light [Bug #13618] has a process-wide FD map anyways
(similar to the kernel fdtable) to deal with multiple
threads waiting on different operations on the same FD,
so we could take advantage of that.
IO.select is a heavy operation, already
Right now, the IO#close notification isn't so bad if few
threads are operating on FDs, but it's O(n) where `n' is
the number of threads calling rb_io_blocking_region in parallel,
regardless of FD.
Back to the Thread::Light FD map, the `n' would be reduced
to the number of threads for a certain FD, because there's
a per-FD linked-list (and getting to that linked-list is
just an array lookup, so O(1).
Updated by printercu (Max Melentiev) over 1 year ago
I'm using
IO.select with ssl socket as it's suggested in docs :
def read_from_socket socket.read_nonblock(read_buffer_size) rescue IO::WaitReadable IO.select([socket.to_io]) retry rescue IO::WaitWritable IO.select(nil, [socket.to_io]) retry end
Other code is like this (simplified):
def run loop do data = read_from_socket process(data) end rescue Errno::EBADF, IOError raise unless @stopped end def stop @stopped = true socket.close end Signal.trap('TERM') { Thread.new { stop } } # thread is just workaround for trap contex run # exit
I can't use just
Thread#kill to not stop thread while it runs
#process.
It works fine on macOS because
socket.close makes read_from_socket fail with Errno::EBADF which is rescued later.
However this does not work on linux and
#run hangs forever.
Is there a right way for this task to not face multi-thread limitations of
IO.select?
For now I use workaround:
SELECT_TIMEOUT = 10 def read_from_socket socket.read_nonblock(read_buffer_size) rescue IO::WaitReadable nil until IO.select([socket.to_io], nil, nil, SELECT_TIMEOUT) retry rescue IO::WaitWritable nil until IO.select(nil, [socket.to_io], nil, SELECT_TIMEOUT) retry end
Updated by normalperson (Eric Wong) over 1 year ago
melentievm@gmail.com wrote:
I can't use just
Thread#killto not stop thread while it runs
#process.
It works fine on macOS because
socket.closemakes read_from_socket fail with Errno::EBADF which is rescued later.
However this does not work on linux and
#runhangs forever.
Is there a right way for this task to not face multi-thread
limitations of
IO.select?
Several ways (there may be more, but I'm tired)
a) You can use Thread#kill with Thread.handle_interrupt around
#process, I think...
b) IO.select allows waiting on multiple FDs, so you could
wait on one process-wide pipe which every IO.select
call checks on:
int_pipe = IO.pipe
trap(:TERM) { int_pipe[1].write('.') }
IO.select([@socket, int_pipe[0]])
IO.select([int_pipe[0]], [@socket])
(Btw, you don't need #to_io when calling IO.select,
it already calls #to_io).
c) if you're dealing with Internet traffic, it's unreliable
and there's malicious clients trying to DoS you.
So you're going to need a SELECT_TIMEOUT anyways, I think...
Also available in: Atom PDF