ConnectionRefusedError Errno 111, when running pymp application

[laytonjbgmail]

Good morning,

I get an error when running a code using pymp. The error is:

Traceback (most recent call last):
File "test8_chunk.py", line 3478, in
currentfigure2, currenttable2, currenttable_ascii2 = iops_output_stats(f_iops_stats, CmdCounter, BeginTime, EndTime, WRITE, READ, TOTAL,
File "/raid/jlayton/TEMP5/iops_summary_output.py", line 125, in iops_output_stats
perf_summary, thread_io_func2 = iops_getstats(CmdCounter, BeginTime, EndTime, WRITE, READ, TOTAL, VFLAGS, MinIOPS,
File "/raid/jlayton/TEMP5/iops_summary_output.py", line 1955, in iops_getstats
with pymp.Parallel(nchunk_leftover) as p:
File "/usr/local/lib/python3.8/dist-packages/pymp/init.py", line 56, in init
self._dynamic_queue = _shared.queue()
File "/usr/local/lib/python3.8/dist-packages/pymp/shared.py", line 58, in queue
return _MANAGER.Queue(*args, **kwargs)
File "/usr/lib/python3.8/multiprocessing/managers.py", line 740, in temp
token, exp = self._create(typeid, *args, **kwds)
File "/usr/lib/python3.8/multiprocessing/managers.py", line 623, in _create
conn = self._Client(self._address, authkey=self._authkey)
File "/usr/lib/python3.8/multiprocessing/connection.py", line 502, in Client
c = SocketClient(address)
File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused

Prior to this, the code has used pump to spawn processes (up to 50 on a server with 64 cores). There is also plenty of memory. Since pymp has worked earlier in the code, I know it can open connections (I'm not sure what port), but it has worked earlier in the code.

I have tried different input data sets, six of them of various sizes, and all have finished correctly (they match the "serial" version of the code).

Any suggestions where to continue looking? What port is used by default?

Thanks!

Jeff

[classner]

Hi @laytonjbgmail ,

That's interesting! I assume you are importing and using PyMP already in a multi-process environment in this failure case? In PyMP, I do not configure a port and I am not sure what port is used by default by the multiprocessing package.

Best,
Christoph

[laytonjbgmail]

Christoph,

Yes, I'm using pymp in an earlier part of the code. It works correctly and I can see the various processes being created, running, and then stopping. I'm trying to create a re-producer since the full code is very large as is the data set.

I wonder if there is a way to specify the port for the multiprocessor package? I'm definitely not the best coder, but I'll look around a bit.

Thanks!

Jeff

[classner]

Hi Jeff!

Getting back on this stale issue: do you have a reproducer, or can I close this issue in the next week assuming it is resolved in the meantime?

Best,
Christoph