Skip to content

Typing for map, submit and Future #9099

@moi90

Description

@moi90

In concurrent.future typeshed, Executor.map and Executor.submit return a typed Future[T]. This is very nice for hinting in the IDE. Would it be possible to add this to dask.distributed?

Something along the following lines:

from typing import Callable, Generic, ParamSpec, TypeVar

_T = TypeVar("_T")
_P = ParamSpec("_P") # ParamSpec was introduced in 3.10.


class Future(TaskRef, Generic[_T]):
    def result(self, timeout=None) -> _T:
        ...

class Client(SyncMethodMixin):
    def submit(
        self,
        func: Callable[_P, _T],
        *args: _P.args,
        key=None,
        workers=None,
        resources=None,
        retries=None,
        priority=0,
        fifo_timeout="100 ms",
        allow_other_workers=False,
        actor=False,
        actors=False,
        pure=True,
        **kwargs: _P.kwargs,
    ) -> Future[_T]:
        ...

    def map(
        self,
        func: Callable[..., _T],
        *iterables: Collection,
        key: str | list | None = None,
        workers: str | Iterable[str] | None = None,
        retries: int | None = None,
        resources: dict[str, Any] | None = None,
        priority: int = 0,
        allow_other_workers: bool = False,
        fifo_timeout: str = "100 ms",
        actor: bool = False,
        actors: bool = False,
        pure: bool = True,
        batch_size=None,
        **kwargs,
    ) -> list[Future[_T]]:
        ...

c = Client()
f = c.submit(lambda x: 0, 1)
# Revealed type is "mock.Future[builtins.int]"
futures = c.map(lambda x: 0, [1, 2, 3])
# Revealed type is "builtins.list[mock.Future[builtins.int]]"

map would need some more work because the types of *iterables* will be hard to distribute to the different parameters of func. It might make sense to use an @overload for the simple case of only one iterable (and maybe additional ones for two and three to cover the most common cases).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions