EWONTFIX

Updates on close, EINTR, & cancellation

03 Oct 2012 00:05 GMT

Last week I took the time to file a report with the Austin Group (responsible for POSIX) about the close issue. It is Issue #614, and it turns out the problem was already solved by fixing the specification of close when interrupted by a signal. Whew. I thought that latter would be a lot more controversial and harder to get fixed

Some basic history on the issue: Apparently, there was a historical disagreement over the behavior of close when interrupted by a signal. Some implementations (e.g. HPUX) had it leave the file descriptor open when returning with EINTR; others (Linux, AIX) closed it unconditionally, but returned with EINTR if a signal arrived while close() was interrupted before returning. This ambiguity was acceptable for single-threaded applications, which could just unconditionally call close() again on EINTR, possibly getting EBADF if the file descriptor no longer existed due to the Linux/AIX behavior. It's not acceptable for multi-threaded applications, where calling close again could close a file descriptor just obtained by another thread - which can have far-reaching and extremely dangerous consequences.

Last December it was refered to the Austin Group for interpretation as Issue #529. The issue was resolved by requiring the HPUX behavior if close returns with EINTR, but allowing implementations which want to deallocate the file descriptor even if interrupted by a signal to return with the EINPROGRESS error instead of EINTR.

This is really the best possible solution, and the latter choice is the choice high-quality implementations should make. The Linux developers objected vehemently to the HPUX behavior on many grounds (see this thread for details), but the most important is that, under the HPUX behavior, it becomes impossible for a program that needs to deallocate a file descriptor to make forward progress when blocked on close; retrying the close will just block again. The situation is even worse with cancellation: the cleanup handler must retry the close, now with cancellation disabled, and it will now block indefinitely. But the Linux behavior is wrong too: in all other cases, EINTR means the application should retry the operation if it needs the effects to have completed, and given how POSIX defines the side effects on cancellation in terms of the side effects on EINTR, it yields unacceptable behavior with respect to cancellation. Avoiding this would have required POSIX to special-case close in the rules for side effects at cancellation.

Unfortunately, Linux (the kernel) is still doing the same thing it always did, returning -EINTR from the close syscall despite having the EINPROGRESS semantics. We're now working around this in musl, and as far as I know, have the first Linux-based libc where close conforms to the amended POSIX requirements. See commit 82dc1e2e783815e00a90cd3f681436a80d54a314 for details. Avoiding the cancellation issue (acting on cancellation after close deallocated the file descriptor) requires additional work at the point where cancellation is processed, but I already took care of this a long time ago; it's in src/thread/cancel_impl.c.

By the way, why does close ever block at all? Some genius way back thought it would be clever to overload the close function with responsibility for rewinding tape devices, and block until rewinding is complete. Never mind that there are perfectly good ways to rewind the device before closing it. If not for this historical blunder, close would never have been a blocking function, would never have been subject to interruption-by-signal, would never fail with EINTR, and would not have been specified by POSIX as a cancellation point. And this would have in turn made programming with file descriptors, signals, and thread cancellation a lot easier and less error-prone...