Improvements to arguments, types with stubtest #1294

loicdiridollou · 2025-07-27T02:55:52Z

More improvements with the stubtest flagging some drift with pandas.

One point that was raised is the handling of deprecated items, maybe the other possibility than purely removing it from the stubs is to force the stubs to adopt the default value so that whatever the user is doing it won't allow any other behavior.
We need to see how much we can use stubtest, I don't see it being used in CI at the moment or anytime soon considering the mountain of work that it raises (mostly correctly but I have seen a few places where it is flagging things that are fine), there is also the problem of the no_default that pandas uses abundantly and which is hard to replicate in the stubs.

Closes #xxxx (Replace xxxx with the Github issue number)
Tests added: Please use assert_type() to assert the type of any return value

loicdiridollou · 2025-07-27T02:56:41Z

pandas-stubs/core/frame.pyi

@@ -1628,24 +1643,7 @@ class DataFrame(NDFrame, OpsMixin, _GetItemHack):
    def isin(self, values: Iterable | Series | DataFrame | dict) -> Self: ...
    @property
    def plot(self) -> PlotAccessor: ...
-    def hist(


Code has changed and now points directly to https://github.com/pandas-dev/pandas/blob/c888af6d0bb674932007623c0867e1fbd4bdc2c6/pandas/plotting/_core.py#L145-L268

But hist_frame is in a private module, and we should remove that module, so revert this change.

loicdiridollou · 2025-07-27T15:34:06Z

pandas-stubs/core/frame.pyi

@@ -308,6 +309,21 @@ else:
        @overload
        def __getitem__(self, key: Hashable) -> Series: ...

+AstypeArgExt: TypeAlias = (


moved them out of the DataFrame class

We should make them private, i.e., renameAstypeArgExt to _AstypeArgExt

pandas-stubs/core/frame.pyi

pandas-stubs/core/indexes/accessors.pyi

pandas-stubs/core/indexes/base.pyi

pandas-stubs/core/indexes/category.pyi

pandas-stubs/core/indexes/datetimes.pyi

pandas-stubs/core/indexes/interval.pyi

loicdiridollou · 2025-07-27T15:36:54Z

pandas-stubs/core/indexes/multi.pyi

@@ -76,10 +76,9 @@ class MultiIndex(Index):
    @property
    def codes(self): ...
    def set_codes(self, codes, *, level=..., verify_integrity: bool = ...): ...
-    def copy(  # pyright: ignore[reportIncompatibleMethodOverride] # pyrefly: ignore
-        self, names=..., deep: bool = ...
+    def copy(  # type: ignore[override]  # pyright: ignore[reportIncompatibleMethodOverride] # pyrefly: ignore


Need overwriting of the parent class (as no names exist).

loicdiridollou · 2025-07-27T15:39:32Z

pandas-stubs/core/indexes/multi.pyi

@@ -136,7 +135,7 @@ class MultiIndex(Index):
        self, indices, axis: int = ..., allow_fill: bool = ..., fill_value=..., **kwargs
    ): ...
    def append(self, other): ...
-    def argsort(self, *args, **kwargs): ...
+    def argsort(self, *args, na_position: NaPosition = ..., **kwargs): ...


Not documented but at runtime it exists, not sure why it is not documented though.

Can you create an issue in pandas about this?

loicdiridollou · 2025-07-27T15:39:41Z

pandas-stubs/core/indexes/period.pyi

@@ -53,7 +52,6 @@ class PeriodIndex(DatetimeIndexOpsMixin[pd.Period], PeriodIndexFieldOps):
    def __rsub__(  # pyright: ignore[reportIncompatibleMethodOverride]
        self, other: NaTType
    ) -> NaTType: ...
-    def __array__(self, dtype=...) -> np.ndarray: ...


Using parent definition.

pandas-stubs/core/series.pyi

loicdiridollou · 2025-07-27T15:57:09Z

pandas-stubs/plotting/_core.pyi

@@ -435,3 +435,23 @@ class PlotAccessor:
    ) -> npt.NDArray[np.object_]: ...

    density = kde
+
+def hist_frame(


Defining it here for pd.DataFrame.hist

Since plotting._core is not documented, we should remove it, and then revert this change.

Dr-Irv

thanks. a number of things that stubgen picks up we still need to follow the docs on

Dr-Irv · 2025-07-28T13:41:35Z

pandas-stubs/core/frame.pyi

@@ -308,6 +309,21 @@ else:
        @overload
        def __getitem__(self, key: Hashable) -> Series: ...

+AstypeArgExt: TypeAlias = (


We should make them private, i.e., renameAstypeArgExt to _AstypeArgExt

Dr-Irv · 2025-07-28T13:42:01Z

pandas-stubs/core/frame.pyi

+        "datetime64[ns]",
+    ]
+)
+AstypeArgExtList: TypeAlias = AstypeArgExt | list[AstypeArgExt]


Suggested change

AstypeArgExtList: TypeAlias = AstypeArgExt | list[AstypeArgExt]

_AstypeArgExtList: TypeAlias = _AstypeArgExt | list[_AstypeArgExt]

make private

Dr-Irv · 2025-07-28T13:46:45Z

pandas-stubs/core/frame.pyi

    def stack(
-        self, level: IndexLabel = ..., dropna: _bool = ..., sort: _bool = ...
+        self,
+        level: IndexLabel = ...,
+        dropna: _bool = ...,
+        sort: _bool = ...,
+        future_stack: Literal[False] = ...,
    ) -> Self | Series: ...
    @overload
    def stack(
-        self, level: IndexLabel = ..., future_stack: _bool = ...
+        self,
+        level: IndexLabel = ...,
+        dropna: _NoDefaultDoNotUse = ...,
+        sort: _NoDefaultDoNotUse = ...,
+        future_stack: Literal[True] = ...,
    ) -> Self | Series: ...


The overload with future_stack: Literal[True] should come first, and have future_stack: Literal[True], and not have the dropna and sort arguments in the list of args.

Dr-Irv · 2025-07-28T13:53:01Z

pandas-stubs/core/frame.pyi

        self,
-        func: AggFuncTypeBase | AggFuncTypeDictSeries,
+        func: AggFuncTypeBase | AggFuncTypeDictSeries = ...,


This is not valid. If you don't specify the value of func, an exception will be raised. Please revert.

Dr-Irv · 2025-07-28T13:55:32Z

pandas-stubs/core/frame.pyi

-    def count(
-        self, axis: Axis = ..., level: None = ..., numeric_only: _bool = ...
-    ) -> Series: ...
+    def count(self, axis: Axis = ..., numeric_only: _bool = ...) -> Self: ...


Should be Series[int]

Dr-Irv · 2025-07-28T14:38:46Z

pandas-stubs/core/indexes/multi.pyi

@@ -136,7 +135,7 @@ class MultiIndex(Index):
        self, indices, axis: int = ..., allow_fill: bool = ..., fill_value=..., **kwargs
    ): ...
    def append(self, other): ...
-    def argsort(self, *args, **kwargs): ...
+    def argsort(self, *args, na_position: NaPosition = ..., **kwargs): ...


Can you create an issue in pandas about this?

Dr-Irv · 2025-07-28T14:42:36Z

pandas-stubs/core/series.pyi

@@ -938,7 +937,7 @@ class Series(IndexOpsMixin[S1], NDFrame):
        self, i: Level = ..., j: Level = ..., copy: _bool = ...
    ) -> Series[S1]: ...
    def reorder_levels(self, order: list) -> Series[S1]: ...
-    def explode(self) -> Series[S1]: ...
+    def explode(self, ignore_index: bool = ...) -> Series[S1]: ...


I think this should be _bool. I think all the args in Series methods that have bool should be changed to _bool

Dr-Irv · 2025-07-28T14:43:42Z

pandas-stubs/core/series.pyi

@@ -1683,7 +1680,6 @@ class Series(IndexOpsMixin[S1], NDFrame):
    ) -> TimedeltaSeries: ...
    @overload
    def __rmul__(self, other: num | _ListLike | Series) -> Series: ...
-    def __rnatmul__(self, other: num | _ListLike | Series[S1]) -> Series[S1]: ...


should be __rmatmul__ - that was a typo, so put it back with the correct name

Dr-Irv · 2025-07-28T14:45:07Z

pandas-stubs/core/series.pyi

    ) -> ExponentialMovingWindow[Series]: ...
    @final
    def expanding(
        self,
        min_periods: int = ...,
+        axis: Axes = ...,


Suggested change

axis: Axes = ...,

axis: Literal[0] = ...,

Dr-Irv · 2025-07-28T14:47:36Z

tests/test_series.py

+                s1.ewm(com=0.3, min_periods=0, adjust=False, ignore_na=True),
                "ExponentialMovingWindow[pd.Series]",


if this is valid usage, then move it outside of the if TYPE_CHECKING_INVALID_USAGE test

Improvements to arguments, types with stubtest

7c201ba