You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently between portpicker picks a port, and the port number is sent to another thread / process to use, there can be a large time window where neither the portpicker nor the port consumer holds the ownership of the port. As a consequence, some other user of the socket API can grab that port, causing the portuser to fail to bind that port number.
This pattern is likely the cause of several flaking tests in TensorFlow OSS workflows, and negatively affected the productivity of github contributors to the surrounding community (OpenXLA, JAX etc).
I wonder if we can add some facility in portpicker to facilitate the transfer of allocated ports between the picker thread/process to the user thread/process. It seems to be rather doable using current Python multiprocessing primitives.
Currently between portpicker picks a port, and the port number is sent to another thread / process to use, there can be a large time window where neither the portpicker nor the port consumer holds the ownership of the port. As a consequence, some other user of the socket API can grab that port, causing the portuser to fail to bind that port number.
This pattern is likely the cause of several flaking tests in TensorFlow OSS workflows, and negatively affected the productivity of github contributors to the surrounding community (OpenXLA, JAX etc).
I wonder if we can add some facility in portpicker to facilitate the transfer of allocated ports between the picker thread/process to the user thread/process. It seems to be rather doable using current Python multiprocessing primitives.
Thoughts? @gpshead