Add defaults for parameters #1293

yangdanny97 · 2025-07-27T02:42:42Z

This PR uses a semi-automated/LLM-assisted method to add a few hundred parameter defaults to pandas-stubs, replacing what was previously ....

There were 3 annotations that had to change because their type signatures are not compatible with the default.

The codemod that made these transformations is here: https://gist.github.com/yangdanny97/0d81e16f6374582413dd5bcf90d902fd

How it works is:

parse pandas-stubs and find the module, method/function, & parameter name for any params w/ default ...
parse the docstring of the method/function in pandas, and if the parameter's docstring contains "default" extract the part after that keyword
look up the type annotation in a mapping of the extracted snippet to type annotation
use libCST to apply the type annotation to the parameter

The main thing that needs careful reviewing is the default_type_map which maps the docstring snippet to the type annotation we replace the ... with. That part was generated with chatGPT's help (I gave it every snippet I got after step 2), and I commented out the mappings that seemed wrong or unclear. So, even though this was "LLM-assisted" the script is deterministic and will always give you the same thing each time, and the LLM is only involved in a one-time manual step.

yangdanny97 · 2025-07-27T02:47:19Z

pandas-stubs/core/indexes/timedeltas.pyi

-    start: TimedeltaConvertibleTypes = ...,
-    end: TimedeltaConvertibleTypes = ...,
-    periods: int | None = ...,
+    start: TimedeltaConvertibleTypes | None = None,


Docstring says the default is None so I had to change the annotation

https://github.com/pandas-dev/pandas/blob/e4a03b6e47a8ef9cd045902916289cbc976d3d33/pandas/core/indexes/timedeltas.py#L256-L259

Based on the docstring, this should probably be rewritten to have overloads:

Of the four parameters start, end, periods, and freq,
exactly three must be specified.

Yes, this is true for date_range() as well

pandas-stubs/core/arrays/datetimelike.pyi

loicdiridollou · 2025-07-27T02:53:02Z

Just for my own knowledge, what is adding the defaults instead of the ellipsis trying to achieve? Is it to help when looking at the docs in the IDE, for pyrefly, or something else?

pandas-stubs/core/frame.pyi

pandas-stubs/core/arrays/datetimelike.pyi

yangdanny97 · 2025-07-27T03:00:18Z

pandas-stubs/core/series.pyi

@@ -1133,7 +1141,7 @@ class Series(IndexOpsMixin[S1], NDFrame):
        self,
        periods: int | Sequence[int] = ...,
        freq: DateOffset | timedelta | _str | None = ...,
-        axis: Axis = ...,
+        axis: Axis | None = None,


this annotation changed

docs aren't right here for Series.align

yangdanny97 · 2025-07-27T03:04:59Z

Just for my own knowledge, what is adding the defaults instead of the ellipsis trying to achieve? Is it to help when looking at the docs in the IDE, for pyrefly, or something else?

It's to help with looking at docs/signature in the IDE. This work is unrelated to Pyrefly.

Here's a thread on typeshed w/ some discussion on the pros and cons of defaults: python/typeshed#8988

The consensus seems to be that if the default is simple (like None, or some literal) then it's fine, but if it's complex we use .... Mypy's stub generator has been updated to emit stubs containing simple defaults.

The second reason I made the PR is just to test out this way of doing LLM-assisted codemods. I have another PR in the works that uses the same approach to add a few hundred types for un-annotated parameters and returns, but that one requires more manual filtering to make sure the generated types are good.

Dr-Irv

I checked every method you changed by looking in the docs. It picks up things that we need to delete or modify anyway. Some general comments:

I use the documented signatures to determine default values, not just what the docs say, because the docs don't always indicate the correct default value, but the signature does because it is picked up in the docs from the actual python declaration.
The copy argument often has a default value of None, so I had to change the declaration a lot of times in your PR, and probably missed a few.

It might be worth modifying your script to look at the default values that are in the signatures rather than the docs. If you do that, you'll pick up a lot more defaults (and probably catch a bunch that I caught via this review).

This PR points out that the pandas docs could be improved, but creating an issue for each improvement would be a lot of work.

When you update, please don't force push, and don't resolve conversations, because then I can see if everything I found was addressed, and can just concentrate on what is changed in your next commit(s).

Thanks for doing this.

Dr-Irv · 2025-07-28T14:57:38Z

pandas-stubs/_libs/tslibs/period.pyi

@@ -197,12 +197,12 @@ class Period(PeriodMixin):
    def day_of_year(self) -> int: ...
    @property
    def day_of_week(self) -> int: ...
-    def asfreq(self, freq: str | BaseOffset, how: _PeriodFreqHow = ...) -> Period: ...
+    def asfreq(self, freq: str | BaseOffset, how: _PeriodFreqHow = "end") -> Period: ...


Not sure whether this matters. Code has default of how as "E", but docs say "end"

I'm fine with leaving it as you have it, but open to which way to go.

Dr-Irv · 2025-07-28T14:58:37Z

pandas-stubs/_libs/tslibs/period.pyi

    @classmethod
    def now(cls, freq: str | BaseOffset = ...) -> Period: ...
    def strftime(self, fmt: str) -> str: ...
    def to_timestamp(
        self,
        freq: str | BaseOffset | None = ...,
-        how: _PeriodToTimestampHow = ...,
+        how: _PeriodToTimestampHow = "S",


same issue as in asfreq(). docs say "S", but code says "start"

I'm fine with leaving it as you have it, but open to which way to go.

Dr-Irv · 2025-07-28T16:10:09Z

pandas-stubs/_libs/tslibs/timestamps.pyi

@@ -157,7 +157,7 @@ class Timestamp(datetime, SupportsIndex):
    ) -> Timestamp: ...
    def astimezone(self, tz: _tzinfo | None = ...) -> Self: ...
    def ctime(self) -> str: ...
-    def isoformat(self, sep: str = ..., timespec: str = ...) -> str: ...
+    def isoformat(self, sep: str = "T", timespec: str = "auto") -> str: ...


Suggested change

def isoformat(self, sep: str = "T", timespec: str = "auto") -> str: ...

def isoformat(self, sep: str = "T", timespec: Literal[‘auto’, ‘hours’, ‘minutes’, ‘seconds’, ‘milliseconds’, ‘microseconds’, ‘nanoseconds’]= "auto") -> str: ...

pandas-stubs/core/arrays/datetimelike.pyi

Dr-Irv · 2025-07-28T19:41:34Z

pandas-stubs/core/series.pyi

-        periods: int = ...,
-        fill_method: None = ...,
+        periods: int = 1,
+        fill_method: None = None,
        freq: DateOffset | timedelta | _str | None = ...,


Dr-Irv · 2025-07-28T19:43:59Z

pandas-stubs/core/series.pyi

-        periods: int = ...,
-        fill_method: None = ...,
+        periods: int = 1,
+        fill_method: None = None,
        freq: DateOffset | timedelta | _str | None = ...,
        *,
        fill_value: Scalar | NAType | None = ...,


remove fill_value param - not documented

Dr-Irv · 2025-07-28T19:44:57Z

pandas-stubs/core/series.pyi

        span: float | None = ...,
        halflife: float | None = ...,
        alpha: float | None = ...,


Dr-Irv · 2025-07-28T19:47:43Z

pandas-stubs/core/series.pyi

@@ -1950,67 +1958,67 @@ class Series(IndexOpsMixin[S1], NDFrame):
        self,
        other: num | _ListLike | Series[S1],
        level: Level | None = ...,
-        fill_value: float | None = ...,
+        fill_value: float | None = None,


should be changed in mul too

Dr-Irv · 2025-07-28T19:49:26Z

pandas-stubs/core/series.pyi

    def pow(
        self,
        other: num | _ListLike | Series[S1],
        level: Level | None = ...,
-        fill_value: float | None = ...,
+        fill_value: float | None = None,
        axis: AxisIndex | None = ...,
    ) -> Series[S1]: ...
    def prod(
        self,
        axis: AxisIndex | None = ...,


For all of these operations, the axis value defaults to 0. There are lots of them in here - I'm commenting on this one, but you should fix the others. I think it is the case that if axis is an argument to a Series method, the default value is 0, because that's the only value that makes sense.

Dr-Irv · 2025-07-28T23:35:19Z

2. The copy argument often has a default value of None, so I had to change the declaration a lot of times in your PR, and probably missed a few.

I looked further into the copy argument issue. Eventually, that argument will be removed. So I think it would be best to not change copy: _bool = False or copy: _bool = True to be copy: _bool | None = None, because None won't be accepted in the future. So ignore those requests in my review.

yangdanny97 · 2025-07-29T13:22:16Z

Thanks for the detailed review!

That's a good point about taking the default values directly from the function declaration. I've pushed a commit doing that using https://gist.github.com/yangdanny97/170f82ee5389584f8b6292bc4ea9c24d

I'll go thru the comments one by one and address them over the next few days, and keep track of which followup issues we need to file.

yangdanny97 added 3 commits July 26, 2025 22:22

add defaults for parameters

047e5d2

add more defaults

2e371b6

fix type errors

9617ee6

yangdanny97 commented Jul 27, 2025

View reviewed changes

loicdiridollou reviewed Jul 27, 2025

View reviewed changes

pandas-stubs/core/arrays/datetimelike.pyi Show resolved Hide resolved

yangdanny97 commented Jul 27, 2025

View reviewed changes

pandas-stubs/core/frame.pyi Show resolved Hide resolved

yangdanny97 commented Jul 27, 2025

View reviewed changes

pandas-stubs/core/arrays/datetimelike.pyi Show resolved Hide resolved

yangdanny97 commented Jul 27, 2025

View reviewed changes

yangdanny97 marked this pull request as ready for review July 27, 2025 03:05

add more defaults

e66384f

yangdanny97 marked this pull request as draft July 27, 2025 18:21

fix deprecated function

b544b4e

yangdanny97 marked this pull request as ready for review July 27, 2025 19:51

Dr-Irv requested changes Jul 28, 2025

View reviewed changes

yangdanny97 added 3 commits July 29, 2025 09:08

apply defaults from source code

29d2fd7

address some comments and fix mypy

ba9d227

remove fill_value from tests

e086068

	def isoformat(self, sep: str = "T", timespec: str = "auto") -> str: ...
	def isoformat(self, sep: str = "T", timespec: Literal[‘auto’, ‘hours’, ‘minutes’, ‘seconds’, ‘milliseconds’, ‘microseconds’, ‘nanoseconds’]= "auto") -> str: ...

	freq: DateOffset \| timedelta \| _str \| None = ...,
	freq: DateOffset \| timedelta \| _str \| None = None,

-        span: float | None = ...,
-        halflife: float | None = ...,
-        alpha: float | None = ...,
+        com: float | None =  None,
+        span: float | None = None,
+        halflife: float | None =  None,
+        alpha: float | None = None,

Uh oh!

Add defaults for parameters #1293

Are you sure you want to change the base?

Add defaults for parameters #1293

Conversation

yangdanny97 commented Jul 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yangdanny97 Jul 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

loicdiridollou commented Jul 27, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yangdanny97 commented Jul 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dr-Irv left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dr-Irv commented Jul 28, 2025

Uh oh!

yangdanny97 commented Jul 29, 2025

Uh oh!

Uh oh!

yangdanny97 commented Jul 27, 2025 •

edited

Loading

yangdanny97 Jul 27, 2025 •

edited

Loading

yangdanny97 commented Jul 27, 2025 •

edited

Loading