Skip to content

Stop using fork method to spawn processes #1600

@progval

Description

@progval

Python 3.14 switches multiprocessing's default start method from fork to forkserver, which requires target functions to be picklable. This is because fork() is unsafe in threaded programs, like Limnoria.

However, Limnoria currently strongly depends on the fork() method (and explicitly opts into it in order to support Python >= 3.14), so that plugins can use commands.process() to call functions defined in plugins, despite plugins not being in Python's import path through:

https://github.com/progval/Limnoria/blob/54c09809786db7a6468c48dedc788287fbcded72/src/plugin.py

With the forkserver method, this wouldn't work because when a plugin calls commands.process(target=f) where f is a function defined in the plugin, the forkee would try to import the plugin (because it is needed to unpickle f), and fail.

Even if we could somehow make the forkee import all plugins and unpickle of Limnoria's state, this would be very wasteful, especially considering some plugins run commands.process() very often.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions