[python] chore: fix python 3.11 bug with async wait_for#488
Conversation
|
@leekeiabstraction PTAL 🙏 |
|
Hi @fresh-borzoni, great catch! I took a look and it appears that the entire functionality is preserved, while avoiding the downfall of the pre-3.12 deadlocking problem. Should this be noted somewhere in the documentation that users should avoid this Python function if they are running a Python version <3.12? We can add it in
|
leekeiabstraction
left a comment
There was a problem hiding this comment.
Approved. Thank your for fixing!
|
@qzyu999 It's a good idea, thank you, but I prefer to wait a bit to verify that the fix is reliable and we don't have any further issues. |
The tests wrapped async iterator consumption in asyncio.wait_for() as a safety net against hangs. But on Python prior to 3.12 fix(python/cpython#96764), wait_for itself has a bug (cpython#96764). Combined with future_into_py/Python::attach(), this creates a GIL deadlock. The safety net was causing the very hang it was meant to prevent.
pytest-timeout (30s per test) is the safety net now. It works at the process level, not the asyncio level, so it can't deadlock. If the async iterator hangs, it's a real bug that deserves a full stack trace from pytest-timeout.
related to https://github.com/apache/fluss-rust/actions/runs/24237394524/job/70768262474?pr=485