From 3b690758dd45e7accd948e8b6f69730e2f05222a Mon Sep 17 00:00:00 2001 From: David Foster Date: Sat, 17 Aug 2024 18:11:59 -0400 Subject: [PATCH 01/13] Refactor rename TypeExpr symbol -> TypeForm --- peps/pep-0747.rst | 306 +++++++++++++++++++++++----------------------- 1 file changed, 153 insertions(+), 153 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 1f362c0606a..04b51eed307 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -1,5 +1,5 @@ PEP: 747 -Title: TypeExpr: Type Hint for a Type Expression +Title: TypeForm: Type Hint for a Type Expression Author: David Foster Sponsor: Jelle Zijlstra Discussions-To: https://discuss.python.org/t/pep-747-typeexpr-type-hint-for-a-type-expression/55984 @@ -20,12 +20,12 @@ not allow ``type[C]`` to refer to arbitrary :ref:`type expression ` objects such as the runtime object ``str | None``, even if ``C`` is an unbounded ``TypeVar``. [#type_c]_ In cases where that restriction is unwanted, this -PEP proposes a new notation ``TypeExpr[T]`` where ``T`` is a type, to +PEP proposes a new notation ``TypeForm[T]`` where ``T`` is a type, to refer to a either a class object or some other type expression object that is a subtype of ``T``, allowing any kind of type to be referenced. This PEP makes no Python grammar changes. Correct usage of -``TypeExpr[]`` is intended to be enforced only by static and runtime +``TypeForm[]`` is intended to be enforced only by static and runtime type checkers and need not be enforced by Python itself at runtime. @@ -34,7 +34,7 @@ type checkers and need not be enforced by Python itself at runtime. Motivation ========== -The introduction of ``TypeExpr`` allows new kinds of metaprogramming +The introduction of ``TypeForm`` allows new kinds of metaprogramming functions that operate on type expressions to be type-annotated and understood by type checkers. @@ -44,9 +44,9 @@ original value: :: - def trycast[T](typx: TypeExpr[T], value: object) -> T | None: ... + def trycast[T](typx: TypeForm[T], value: object) -> T | None: ... -The use of ``TypeExpr[]`` and the type variable ``T`` enables the return +The use of ``TypeForm[]`` and the type variable ``T`` enables the return type of this function to be influenced by a ``typx`` value passed at runtime, which is quite powerful. @@ -56,9 +56,9 @@ variable of a particular type, and if so returns ``True`` (as a special :: - def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]: ... + def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]: ... -The use of ``TypeExpr[]`` and ``TypeIs[]`` together enables type +The use of ``TypeForm[]`` and ``TypeIs[]`` together enables type checkers to narrow the return type appropriately depending on what type expression is passed in: @@ -74,7 +74,7 @@ conforms to a particular structure`_ of nested ``TypedDict``\ s, lists, unions, ``Literal``\ s, and other types. This kind of check was alluded to in :pep:`PEP 589 <589#using-typeddict-types>` but could not be implemented at the time without a notation similar to -``TypeExpr[]``. +``TypeForm[]``. .. _checking whether a value decoded from JSON conforms to a particular structure: https://mail.python.org/archives/list/typing-sig@python.org/thread/I5ZOQICTJCENTCDPHLZR7NT42QJ43GP4/ @@ -84,13 +84,13 @@ Why can’t ``type[]`` be used? One might think you could define the example functions above to take a ``type[C]`` - which is syntax that already exists - rather than a -``TypeExpr[T]``. However if you were to do that then certain type +``TypeForm[T]``. However if you were to do that then certain type expressions like ``str | None`` - which are not class objects and therefore not ``type``\ s at runtime - would be rejected: :: - # NOTE: Uses a type[C] parameter rather than a TypeExpr[T] + # NOTE: Uses a type[C] parameter rather than a TypeForm[T] def trycast_type[C](typ: type[C], value: object) -> T | None: ... trycast_type(str, 'hi') # ok; str is a type @@ -99,10 +99,10 @@ therefore not ``type``\ s at runtime - would be rejected: trycast_type(MyTypedDict, dict(value='hi')) # questionable; accepted by mypy 1.9.0 To solve that problem, ``type[]`` could be widened to include the -additional values allowed by ``TypeExpr``. However doing so would lose +additional values allowed by ``TypeForm``. However doing so would lose ``type[]``\ ’s current ability to spell a class object which always supports instantiation and ``isinstance`` checks, unlike arbitrary type -expression objects. Therefore ``TypeExpr`` is proposed as new notation +expression objects. Therefore ``TypeForm`` is proposed as new notation instead. For a longer explanation of why we don’t just widen ``type[T]`` to @@ -112,11 +112,11 @@ accept all type expressions, see .. _runtime_type_checkers_using_typeexpr: -Common kinds of functions that would benefit from TypeExpr +Common kinds of functions that would benefit from TypeForm ---------------------------------------------------------- `A survey of various Python libraries`_ revealed a few kinds of commonly -defined functions which would benefit from ``TypeExpr[]``: +defined functions which would benefit from ``TypeForm[]``: .. _A survey of various Python libraries: https://github.com/python/mypy/issues/9773#issuecomment-2017998886 @@ -126,9 +126,9 @@ defined functions which would benefit from ``TypeExpr[]``: then also narrows the type of the value to match the type expression. - Pattern 1: - ``def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]`` + ``def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]`` - Pattern 2: - ``def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]`` + ``def ismatch[T](value: object, typx: TypeForm[T]) -> TypeGuard[T]`` - Examples: beartype.\ `is_bearable`_, trycast.\ `isassignable`_, typeguard.\ `check_type`_, xdsl.\ `isa`_ @@ -143,7 +143,7 @@ defined functions which would benefit from ``TypeExpr[]``: a *converter* returns the value narrowed to (or coerced to) that type expression. Otherwise, it raises an exception. - Pattern 1: - ``def convert[T](value: object, typx: TypeExpr[T]) -> T`` + ``def convert[T](value: object, typx: TypeForm[T]) -> T`` - Examples: cattrs.BaseConverter.\ `structure`_, trycast.\ `checkcast`_, typedload.\ `load`_ @@ -153,7 +153,7 @@ defined functions which would benefit from ``TypeExpr[]``: :: class Converter[T]: - def __init__(self, typx: TypeExpr[T]) -> None: ... + def __init__(self, typx: TypeForm[T]) -> None: ... def convert(self, value: object) -> T: ... - Examples: pydantic.\ `TypeAdapter(T).validate_python`_, @@ -172,7 +172,7 @@ defined functions which would benefit from ``TypeExpr[]``: :: class Field[T]: - value_type: TypeExpr[T] + value_type: TypeForm[T] - Examples: attrs.\ `make_class`_, dataclasses.\ `make_dataclass`_ [#DataclassInitVar]_, `openapify`_ @@ -183,7 +183,7 @@ defined functions which would benefit from ``TypeExpr[]``: The survey also identified some introspection functions that take annotation expressions as input using plain ``object``\ s which would -*not* gain functionality by marking those inputs as ``TypeExpr[]``: +*not* gain functionality by marking those inputs as ``TypeForm[]``: - General introspection operations: @@ -210,7 +210,7 @@ different kinds of type annotations: | | | +---------------------+ | | | | | | | Class object | | | | = type[C] | | | +---------------------+ | | | - | | | Type expression object | | | = TypeExpr[T] <-- new! + | | | Type expression object | | | = TypeForm[T] <-- new! | | +-------------------------+ | | | | Annotation expression object | | | +------------------------------+ | @@ -234,52 +234,52 @@ different kinds of type annotations: - Examples: ``Final[int]``, ``Required[str]``, ``ClassVar[str]``, any type expression -``TypeExpr`` aligns with an existing definition from the above list - +``TypeForm`` aligns with an existing definition from the above list - *type expression* - to avoid introducing yet another subset of type annotations that users of Python typing need to think about. -``TypeExpr`` aligns with *type expression* specifically +``TypeForm`` aligns with *type expression* specifically because a type expression is already used to parameterize type variables, which are used in combination with ``TypeIs`` and ``TypeGuard`` to enable the compelling examples mentioned in :ref:`Motivation `. -``TypeExpr`` does not align with *annotation expression* for reasons given in +``TypeForm`` does not align with *annotation expression* for reasons given in :ref:`Rejected Ideas » Accept arbitrary annotation expressions `. Specification ============= -A ``TypeExpr`` value represents a :ref:`type expression ` +A ``TypeForm`` value represents a :ref:`type expression ` such as ``str | None``, ``dict[str, int]``, or ``MyTypedDict``. -A ``TypeExpr`` type is written as -``TypeExpr[T]`` where ``T`` is a type or a type variable. It can also be -written without brackets as just ``TypeExpr``, which is treated the same as -to ``TypeExpr[Any]``. +A ``TypeForm`` type is written as +``TypeForm[T]`` where ``T`` is a type or a type variable. It can also be +written without brackets as just ``TypeForm``, which is treated the same as +to ``TypeForm[Any]``. -Using TypeExprs +Using TypeForms --------------- -A ``TypeExpr`` is a new kind of type expression, usable in any context where a +A ``TypeForm`` is a new kind of type expression, usable in any context where a type expression is valid, as a function parameter type, a return type, or a variable type: :: - def is_union_type(typx: TypeExpr) -> bool: ... # parameter type + def is_union_type(typx: TypeForm) -> bool: ... # parameter type :: - def union_of[S, T](s: TypeExpr[S], t: TypeExpr[T]) \ - -> TypeExpr[S | T]: ... # return type + def union_of[S, T](s: TypeForm[S], t: TypeForm[T]) \ + -> TypeForm[S | T]: ... # return type :: - STR_TYPE: TypeExpr = str # variable type + STR_TYPE: TypeForm = str # variable type Note however that an *unannotated* variable assigned a type expression literal -will not be inferred to be of ``TypeExpr`` type by type checkers because PEP +will not be inferred to be of ``TypeForm`` type by type checkers because PEP 484 :pep:`reserves that syntax for defining type aliases <484#type-aliases>`: - No: @@ -290,71 +290,71 @@ will not be inferred to be of ``TypeExpr`` type by type checkers because PEP If you want a type checker to recognize a type expression literal in a bare assignment you’ll need to explicitly declare the assignment-target as -having ``TypeExpr`` type: +having ``TypeForm`` type: - Yes: :: - STR_TYPE: TypeExpr = str + STR_TYPE: TypeForm = str - Yes: :: - STR_TYPE: TypeExpr + STR_TYPE: TypeForm STR_TYPE = str - Okay, but discouraged: :: - STR_TYPE = str # type: TypeExpr # the type comment is significant + STR_TYPE = str # type: TypeForm # the type comment is significant -``TypeExpr`` values can be passed around and assigned just like normal +``TypeForm`` values can be passed around and assigned just like normal values: :: - def swap1[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]: - t1_new: TypeExpr[T] = t2 # assigns a TypeExpr value to a new annotated variable - t2_new: TypeExpr[S] = t1 + def swap1[S, T](t1: TypeForm[S], t2: TypeForm[T]) -> tuple[TypeForm[T], TypeForm[S]]: + t1_new: TypeForm[T] = t2 # assigns a TypeForm value to a new annotated variable + t2_new: TypeForm[S] = t1 return (t1_new, t2_new) - def swap2[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]: - t1_new = t2 # assigns a TypeExpr value to a new unannotated variable + def swap2[S, T](t1: TypeForm[S], t2: TypeForm[T]) -> tuple[TypeForm[T], TypeForm[S]]: + t1_new = t2 # assigns a TypeForm value to a new unannotated variable t2_new = t1 - assert_type(t1_new, TypeExpr[T]) - assert_type(t2_new, TypeExpr[S]) + assert_type(t1_new, TypeForm[T]) + assert_type(t2_new, TypeForm[S]) return (t1_new, t2_new) # NOTE: A more straightforward implementation would use isinstance() def ensure_int(value: object) -> None: - value_type: TypeExpr = type(value) # assigns a type (a subtype of TypeExpr) + value_type: TypeForm = type(value) # assigns a type (a subtype of TypeForm) assert value_type == int -TypeExpr Values +TypeForm Values --------------- -A variable of type ``TypeExpr[T]`` where ``T`` is a type, can hold any +A variable of type ``TypeForm[T]`` where ``T`` is a type, can hold any **type expression object** - the result of evaluating a :ref:`type expression ` at runtime - which is a subtype of ``T``. Incomplete expressions like a bare ``Optional`` or ``Union`` which do -not spell a type are not ``TypeExpr`` values. +not spell a type are not ``TypeForm`` values. -``TypeExpr[...]`` is itself a ``TypeExpr`` value: +``TypeForm[...]`` is itself a ``TypeForm`` value: :: - OPTIONAL_INT_TYPE: TypeExpr = TypeExpr[int | None] # OK + OPTIONAL_INT_TYPE: TypeForm = TypeForm[int | None] # OK assert isassignable(int | None, OPTIONAL_INT_TYPE) .. _non_universal_typeexpr: -``TypeExpr[]`` values include *all* type expressions including some +``TypeForm[]`` values include *all* type expressions including some **non-universal type expressions** which are not valid in all annotation contexts. In particular: @@ -363,24 +363,24 @@ In particular: - ``TypeIs[...]`` (valid only in some contexts) -Explicit TypeExpr Values +Explicit TypeForm Values '''''''''''''''''''''''' -The syntax ``TypeExpr(T)`` (with parentheses) can be used to -spell a ``TypeExpr[T]`` value explicitly: +The syntax ``TypeForm(T)`` (with parentheses) can be used to +spell a ``TypeForm[T]`` value explicitly: :: - NONE = TypeExpr(None) - INT1 = TypeExpr('int') # stringified type expression - INT2 = TypeExpr(int) + NONE = TypeForm(None) + INT1 = TypeForm('int') # stringified type expression + INT2 = TypeForm(int) -At runtime the ``TypeExpr(...)`` callable returns its single argument unchanged. +At runtime the ``TypeForm(...)`` callable returns its single argument unchanged. .. _implicit_typeexpr_values: -Implicit TypeExpr Values +Implicit TypeForm Values '''''''''''''''''''''''' Historically static type checkers have only needed to recognize @@ -399,7 +399,7 @@ Static type checkers already recognize **class objects** (``type[C]``): The following **unparameterized type expressions** can be recognized unambiguously: -- As a value expression, ``X`` has type ``TypeExpr[X]``, +- As a value expression, ``X`` has type ``TypeForm[X]``, for each of the following values of X: - ```` @@ -409,14 +409,14 @@ The following **unparameterized type expressions** can be recognized unambiguous - ```` **None**: The type expression ``None`` (``NoneType``) is ambiguous with the value ``None``, -so must use the explicit ``TypeExpr(...)`` syntax: +so must use the explicit ``TypeForm(...)`` syntax: -- As a value expression, ``TypeExpr(None)`` has type ``TypeExpr[None]``. +- As a value expression, ``TypeForm(None)`` has type ``TypeForm[None]``. - As a value expression, ``None`` continues to have type ``None``. The following **parameterized type expressions** can be recognized unambiguously: -- As a value expression, ``X`` has type ``TypeExpr[X]``, +- As a value expression, ``X`` has type ``TypeForm[X]``, for each of the following values of X: - `` '[' ... ']'`` @@ -435,20 +435,20 @@ so must be disambiguated based on its argument type: - As a value expression, ``Annotated[x, ...]`` has type ``type[C]`` if ``x`` has type ``type[C]``. -- As a value expression, ``Annotated[x, ...]`` has type ``TypeExpr[T]`` - if ``x`` has type ``TypeExpr[T]``. +- As a value expression, ``Annotated[x, ...]`` has type ``TypeForm[T]`` + if ``x`` has type ``TypeForm[T]``. - As a value expression, ``Annotated[x, ...]`` has type ``object`` - if ``x`` has a type that is not ``type[C]`` or ``TypeExpr[T]``. + if ``x`` has a type that is not ``type[C]`` or ``TypeForm[T]``. **Union**: The type expression ``T1 | T2`` is ambiguous with the value ``int1 | int2``, ``set1 | set2``, ``dict1 | dict2``, and more, -so must use the explicit ``TypeExpr(...)`` syntax: +so must use the explicit ``TypeForm(...)`` syntax: - Yes: :: - if isassignable(value, TypeExpr(int | str)): ... + if isassignable(value, TypeForm(int | str)): ... - No: @@ -457,15 +457,15 @@ so must use the explicit ``TypeExpr(...)`` syntax: if isassignable(value, int | str): ... Future PEPs may make it possible to recognize the value expression ``T1 | T2`` directly as an -implicit TypeExpr value and avoid the need to use the explicit ``TypeExpr(...)`` syntax, +implicit TypeForm value and avoid the need to use the explicit ``TypeForm(...)`` syntax, but that work is :ref:`deferred for now `. The **stringified type expression** ``"T"`` is ambiguous with both the stringified annotation expression ``"T"`` and the string literal ``"T"``, -so must use the explicit ``TypeExpr(...)`` syntax: +so must use the explicit ``TypeForm(...)`` syntax: -- As a value expression, ``TypeExpr("T")`` has type ``TypeExpr[T]``, +- As a value expression, ``TypeForm("T")`` has type ``TypeForm[T]``, where ``T`` is a valid type expression - As a value expression, ``"T"`` continues to have type ``Literal["T"]``. @@ -475,29 +475,29 @@ New kinds of type expressions that are introduced should define how they will be recognized in a value expression context. -Literal[] TypeExprs +Literal[] TypeForms ''''''''''''''''''' A value of ``Literal[...]`` type is *not* considered assignable to -a ``TypeExpr`` variable even if all of its members spell valid types because +a ``TypeForm`` variable even if all of its members spell valid types because dynamic values are not allowed in type expressions: :: STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str' - STRS_TYPE: TypeExpr = STRS_TYPE_NAME # ERROR: Literal[] value is not a TypeExpr + STRS_TYPE: TypeForm = STRS_TYPE_NAME # ERROR: Literal[] value is not a TypeForm -However ``Literal[...]`` itself is still a ``TypeExpr``: +However ``Literal[...]`` itself is still a ``TypeForm``: :: - DIRECTION_TYPE: TypeExpr[Literal['left', 'right']] = Literal['left', 'right'] # OK + DIRECTION_TYPE: TypeForm[Literal['left', 'right']] = Literal['left', 'right'] # OK -Static vs. Runtime Representations of TypeExprs +Static vs. Runtime Representations of TypeForms ''''''''''''''''''''''''''''''''''''''''''''''' -A ``TypeExpr`` value appearing statically in a source file may be normalized +A ``TypeForm`` value appearing statically in a source file may be normalized to a different representation at runtime. For example string-based forward references are normalized at runtime to be ``ForwardRef`` instances in some contexts: [#forward_ref_normalization]_ @@ -508,80 +508,80 @@ in some contexts: [#forward_ref_normalization]_ >>> IntTree list[typing.Union[int, ForwardRef('IntTree')]] -The runtime representations of ``TypeExpr``\ s are considered implementation +The runtime representations of ``TypeForm``\ s are considered implementation details that may change over time and therefore static type checkers are not required to recognize them: :: - INT_TREE: TypeExpr = ForwardRef('IntTree') # ERROR: Runtime-only form + INT_TREE: TypeForm = ForwardRef('IntTree') # ERROR: Runtime-only form Runtime type checkers that wish to assign a runtime-only representation -of a type expression to a ``TypeExpr[]`` variable must use ``cast()`` to +of a type expression to a ``TypeForm[]`` variable must use ``cast()`` to avoid errors from static type checkers: :: - INT_TREE = cast(TypeExpr, ForwardRef('IntTree')) # OK + INT_TREE = cast(TypeForm, ForwardRef('IntTree')) # OK Subtyping --------- -Whether a ``TypeExpr`` value can be assigned from one variable to another is +Whether a ``TypeForm`` value can be assigned from one variable to another is determined by the following rules: Relationship with type '''''''''''''''''''''' -``TypeExpr[]`` is covariant in its argument type, just like ``type[]``: +``TypeForm[]`` is covariant in its argument type, just like ``type[]``: -- ``TypeExpr[T1]`` is a subtype of ``TypeExpr[T2]`` iff ``T1`` is a +- ``TypeForm[T1]`` is a subtype of ``TypeForm[T2]`` iff ``T1`` is a subtype of ``T2``. -- ``type[C1]`` is a subtype of ``TypeExpr[C2]`` iff ``C1`` is a subtype +- ``type[C1]`` is a subtype of ``TypeForm[C2]`` iff ``C1`` is a subtype of ``C2``. -An unparameterized ``type`` can be assigned to an unparameterized ``TypeExpr`` +An unparameterized ``type`` can be assigned to an unparameterized ``TypeForm`` but not the other way around: -- ``type[Any]`` is assignable to ``TypeExpr[Any]``. (But not the +- ``type[Any]`` is assignable to ``TypeForm[Any]``. (But not the other way around.) Relationship with object '''''''''''''''''''''''' -``TypeExpr[]`` is a kind of ``object``, just like ``type[]``: +``TypeForm[]`` is a kind of ``object``, just like ``type[]``: -- ``TypeExpr[T]`` for any ``T`` is a subtype of ``object``. +- ``TypeForm[T]`` for any ``T`` is a subtype of ``object``. -``TypeExpr[T]``, where ``T`` is a type variable, is assumed to have all +``TypeForm[T]``, where ``T`` is a type variable, is assumed to have all the attributes and methods of ``object`` and is not callable. Interactions with isinstance() and issubclass() ----------------------------------------------- -The ``TypeExpr`` special form cannot be used as the ``type`` argument to +The ``TypeForm`` special form cannot be used as the ``type`` argument to ``isinstance``: :: - >>> isinstance(str, TypeExpr) - TypeError: typing.TypeExpr cannot be used with isinstance() + >>> isinstance(str, TypeForm) + TypeError: typing.TypeForm cannot be used with isinstance() - >>> isinstance(str, TypeExpr[str]) + >>> isinstance(str, TypeForm[str]) TypeError: isinstance() argument 2 cannot be a parameterized generic -The ``TypeExpr`` special form cannot be used as any argument to +The ``TypeForm`` special form cannot be used as any argument to ``issubclass``: :: - >>> issubclass(TypeExpr, object) + >>> issubclass(TypeForm, object) TypeError: issubclass() arg 1 must be a class - >>> issubclass(object, TypeExpr) - TypeError: typing.TypeExpr cannot be used with issubclass() + >>> issubclass(object, TypeForm) + TypeError: typing.TypeForm cannot be used with issubclass() Affected signatures in the standard library @@ -591,7 +591,7 @@ Changed signatures '''''''''''''''''' The following signatures related to type expressions introduce -``TypeExpr`` where previously ``object`` or ``Any`` existed: +``TypeForm`` where previously ``object`` or ``Any`` existed: - ``typing.cast`` - ``typing.assert_type`` @@ -670,30 +670,30 @@ assigned to variables and manipulated like any other data in a program: a variable a type | | v v - MAYBE_INT_TYPE: TypeExpr = int | None + MAYBE_INT_TYPE: TypeForm = int | None ^ | the type of a type -``TypeExpr[]`` is how you spell the type of a variable containing a +``TypeForm[]`` is how you spell the type of a variable containing a type annotation object describing a type. -``TypeExpr[]`` is similar to ``type[]``, but ``type[]`` can only +``TypeForm[]`` is similar to ``type[]``, but ``type[]`` can only spell simple **class objects** like ``int``, ``str``, ``list``, or ``MyClass``. -``TypeExpr[]`` by contrast can additionally spell more complex types, +``TypeForm[]`` by contrast can additionally spell more complex types, including those with brackets (like ``list[int]``) or pipes (like ``int | None``), and including special types like ``Any``, ``LiteralString``, or ``Never``. -A ``TypeExpr`` variable (``maybe_float: TypeExpr``) looks similar to -a ``TypeAlias`` definition (``MaybeFloat: TypeAlias``), but ``TypeExpr`` +A ``TypeForm`` variable (``maybe_float: TypeForm``) looks similar to +a ``TypeAlias`` definition (``MaybeFloat: TypeAlias``), but ``TypeForm`` can only be used where a dynamic value is expected: - No: :: - maybe_float: TypeExpr = float | None - def sqrt(n: float) -> maybe_float: ... # ERROR: Can't use TypeExpr value in a type annotation + maybe_float: TypeForm = float | None + def sqrt(n: float) -> maybe_float: ... # ERROR: Can't use TypeForm value in a type annotation - Okay, but discouraged in Python 3.12+: @@ -710,16 +710,16 @@ can only be used where a dynamic value is expected: def sqrt(n: float) -> MaybeFloat: ... It is uncommon for a programmer to define their *own* function which accepts -a ``TypeExpr`` parameter or returns a ``TypeExpr`` value. Instead it is more common +a ``TypeForm`` parameter or returns a ``TypeForm`` value. Instead it is more common for a programmer to pass a literal type expression to an *existing* function -accepting a ``TypeExpr`` input which was imported from a runtime type checker +accepting a ``TypeForm`` input which was imported from a runtime type checker library. For example the ``isassignable`` function from the ``trycast`` library can be used like Python's built-in ``isinstance`` function to check whether a value matches the shape of a particular type. ``isassignable`` will accept *any* kind of type as an input because its input -is a ``TypeExpr``. By contrast ``isinstance`` only accepts a simple class object +is a ``TypeForm``. By contrast ``isinstance`` only accepts a simple class object (a ``type[]``) as input: - Yes: @@ -728,7 +728,7 @@ is a ``TypeExpr``. By contrast ``isinstance`` only accepts a simple class object from trycast import isassignable - if isassignable(some_object, MyTypedDict): # OK: MyTypedDict is a TypeExpr[] + if isassignable(some_object, MyTypedDict): # OK: MyTypedDict is a TypeForm[] ... - No: @@ -739,7 +739,7 @@ is a ``TypeExpr``. By contrast ``isinstance`` only accepts a simple class object ... There are :ref:`many other runtime type checkers ` -providing useful functions that accept a ``TypeExpr``. +providing useful functions that accept a ``TypeForm``. .. _advanced_examples: @@ -750,13 +750,13 @@ Advanced Examples If you want to write your own runtime type checker or some other kind of function that manipulates types as values at runtime, this section gives examples of how you might implement such a function -using ``TypeExpr``. +using ``TypeForm``. -Introspecting TypeExpr Values +Introspecting TypeForm Values ----------------------------- -A ``TypeExpr`` is very similar to an ``object`` at runtime, with no additional +A ``TypeForm`` is very similar to an ``object`` at runtime, with no additional attributes or methods defined. You can use existing introspection functions like ``typing.get_origin`` and @@ -767,9 +767,9 @@ like ``Origin[Arg1, Arg2, ..., ArgN]``: import typing - def strip_annotated_metadata(typx: TypeExpr[T]) -> TypeExpr[T]: + def strip_annotated_metadata(typx: TypeForm[T]) -> TypeForm[T]: if typing.get_origin(typx) is typing.Annotated: - typx = cast(TypeExpr[T], typing.get_args(typx)[0]) + typx = cast(TypeForm[T], typing.get_args(typx)[0]) return typx You can also use ``isinstance`` and ``is`` to distinguish one kind of @@ -780,11 +780,11 @@ type expression from another: import types import typing - def split_union(typx: TypeExpr) -> tuple[TypeExpr, ...]: + def split_union(typx: TypeForm) -> tuple[TypeForm, ...]: if isinstance(typx, types.UnionType): # X | Y - return cast(tuple[TypeExpr, ...], typing.get_args(typx)) + return cast(tuple[TypeForm, ...], typing.get_args(typx)) if typing.get_origin(typx) is typing.Union: # Union[X, Y] - return cast(tuple[TypeExpr, ...], typing.get_args(typx)) + return cast(tuple[TypeForm, ...], typing.get_args(typx)) if typx in (typing.Never, typing.NoReturn,): return () return (typx,) @@ -793,36 +793,36 @@ type expression from another: Combining with a type variable ------------------------------ -``TypeExpr[]`` can be parameterized by a type variable that is used elsewhere within +``TypeForm[]`` can be parameterized by a type variable that is used elsewhere within the same function definition: :: - def as_instance[T](typx: TypeExpr[T]) -> T | None: + def as_instance[T](typx: TypeForm[T]) -> T | None: return typx() if isinstance(typx, type) else None Combining with type[] --------------------- -Both ``TypeExpr[]`` and ``type[]`` can be parameterized by the same type +Both ``TypeForm[]`` and ``type[]`` can be parameterized by the same type variable within the same function definition: :: - def as_type[T](typx: TypeExpr[T]) -> type[T] | None: + def as_type[T](typx: TypeForm[T]) -> type[T] | None: return typx if isinstance(typx, type) else None Combining with TypeIs[] and TypeGuard[] --------------------------------------- -A type variable parameterizing a ``TypeExpr[]`` can also be used by a ``TypeIs[]`` +A type variable parameterizing a ``TypeForm[]`` can also be used by a ``TypeIs[]`` within the same function definition: :: - def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]: ... + def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]: ... count: int | str = ... if isassignable(count, int): @@ -834,7 +834,7 @@ or by a ``TypeGuard[]`` within the same function definition: :: - def isdefault[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]: + def isdefault[T](value: object, typx: TypeForm[T]) -> TypeGuard[T]: return (value == typx()) if isinstance(typx, type) else False value: int | str = '' @@ -848,10 +848,10 @@ or by a ``TypeGuard[]`` within the same function definition: assert_type(value, int | str) -Challenges When Accepting All TypeExprs +Challenges When Accepting All TypeForms --------------------------------------- -A function that takes an *arbitrary* ``TypeExpr`` as +A function that takes an *arbitrary* ``TypeForm`` as input must support a large variety of possible type expressions and is not easy to write. Some challenges faced by such a function include: @@ -882,7 +882,7 @@ Reference Implementation The following will be true when `mypy#9773 `__ is implemented: - The mypy type checker supports ``TypeExpr`` types. + The mypy type checker supports ``TypeForm`` types. A reference implementation of the runtime component is provided in the ``typing_extensions`` module. @@ -900,12 +900,12 @@ Widen type[C] to support all type expressions class object can always be used as the second argument of ``isinstance()`` and can usually be instantiated by calling it. -``TypeExpr`` on the other hand is typically introspected by the user in +``TypeForm`` on the other hand is typically introspected by the user in some way, is not necessarily directly instantiable, and is not necessarily directly usable in a regular ``isinstance()`` check. It would be possible to widen ``type`` to include the additional values -allowed by ``TypeExpr`` but it would reduce clarity about the user’s +allowed by ``TypeForm`` but it would reduce clarity about the user’s intentions when working with a ``type``. Different concepts and usage patterns; different spellings. @@ -931,27 +931,27 @@ parameter type or a return type: def nonsense() -> Final[object]: ... # ERROR: Final[] not meaningful here -``TypeExpr[T]`` does not allow matching such annotation expressions +``TypeForm[T]`` does not allow matching such annotation expressions because it is not clear what it would mean for such an expression to parameterized by a type variable in position ``T``: :: - def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]: ... + def ismatch[T](value: object, typx: TypeForm[T]) -> TypeGuard[T]: ... def foo(some_arg): - if ismatch(some_arg, Final[int]): # ERROR: Final[int] is not a TypeExpr + if ismatch(some_arg, Final[int]): # ERROR: Final[int] is not a TypeForm reveal_type(some_arg) # ? NOT Final[int], because invalid for a parameter Functions that wish to operate on *all* kinds of annotation expressions, -including those that are not ``TypeExpr``\ s, can continue to accept such +including those that are not ``TypeForm``\ s, can continue to accept such inputs as ``object`` parameters, as they must do so today. Accept only universal type expressions -------------------------------------- -Earlier drafts of this PEP only allowed ``TypeExpr[]`` to match the subset +Earlier drafts of this PEP only allowed ``TypeForm[]`` to match the subset of type expressions which are valid in *all* contexts, excluding :ref:`non-universal type expressions `. However doing that would effectively @@ -960,7 +960,7 @@ would have to understand, on top of all the existing distinctions between “class objects”, “type expressions”, and “annotation expressions”. To avoid introducing yet another concept that everyone has to learn, -this proposal just rounds ``TypeExpr[]`` to exactly match the existing +this proposal just rounds ``TypeForm[]`` to exactly match the existing definition of a “type expression”. @@ -977,14 +977,14 @@ Consider the following possible pattern matching syntax: :: @overload - def checkcast(typx: TypeExpr[AT=Annotated[T, *Anns]], value: str) -> T: ... + def checkcast(typx: TypeForm[AT=Annotated[T, *Anns]], value: str) -> T: ... @overload - def checkcast(typx: TypeExpr[UT=Union[*Ts]], value: str) -> Union[*Ts]: ... + def checkcast(typx: TypeForm[UT=Union[*Ts]], value: str) -> Union[*Ts]: ... @overload def checkcast(typx: type[C], value: str) -> C: ... # ... (more) -All functions observed in the wild that conceptually take a ``TypeExpr[]`` +All functions observed in the wild that conceptually take a ``TypeForm[]`` generally try to support *all* kinds of type expressions, so it doesn’t seem valuable to enumerate a particular subset. @@ -1001,7 +1001,7 @@ interior type argument and strip away the metadata: :: def checkcast( - typx: TypeExpr[T] | TypeExpr[AT=Annotated[T, *Anns]], + typx: TypeForm[T] | TypeForm[AT=Annotated[T, *Anns]], value: object ) -> T: @@ -1011,17 +1011,17 @@ The example above could be more-straightforwardly written as the equivalent: :: - def checkcast(typx: TypeExpr[T], value: object) -> T: + def checkcast(typx: TypeForm[T], value: object) -> T: .. _recognize_uniontype_as_implicit_typeexpr_value: -Recognize (T1 | T2) as an implicit TypeExpr value +Recognize (T1 | T2) as an implicit TypeForm value ------------------------------------------------- It would be nice if a value expression like ``int | str`` could be recognized -as an implicit ``TypeExpr`` value and be used directly in a context where a -``TypeExpr`` was expected. However making that possible would require making +as an implicit ``TypeForm`` value and be used directly in a context where a +``TypeForm`` was expected. However making that possible would require making changes to the rules that type checkers use for the ``|`` operator. These rules are currently underspecified and would need to be make explicit first, before making changes to them. The PEP author is not sufficiently motivated to @@ -1041,7 +1041,7 @@ Footnotes ``dataclass.make_dataclass`` accepts ``InitVar[...]`` as a special case in addition to type expressions. Therefore it may unfortunately be necessary to continue annotating its ``type`` parameter as ``object`` rather - than ``TypeExpr``. + than ``TypeForm``. .. [#forward_ref_normalization] Special forms normalize string arguments to ``ForwardRef`` instances From d33a67ec376d494e2cbcf20bd9bb96944b6d74d9 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Fri, 30 Aug 2024 08:05:54 -0700 Subject: [PATCH 02/13] Updated draft PEP 747 with updated design proposal. --- peps/pep-0747.rst | 1052 ++++++++++++++------------------------------- 1 file changed, 329 insertions(+), 723 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 04b51eed307..8006ed13857 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -1,6 +1,6 @@ PEP: 747 -Title: TypeForm: Type Hint for a Type Expression -Author: David Foster +Title: Annotating Type Forms +Author: David Foster , Eric Traut Sponsor: Jelle Zijlstra Discussions-To: https://discuss.python.org/t/pep-747-typeexpr-type-hint-for-a-type-expression/55984 Status: Draft @@ -14,121 +14,89 @@ Post-History: `19-Apr-2024 ` objects such -as the runtime object ``str | None``, even if ``C`` is an unbounded -``TypeVar``. [#type_c]_ In cases where that restriction is unwanted, this -PEP proposes a new notation ``TypeForm[T]`` where ``T`` is a type, to -refer to a either a class object or some other type expression object -that is a subtype of ``T``, allowing any kind of type to be referenced. +:ref:`Type expressions ` provide a standardized way +to specify types in the Python type system. When a type expression is +evaluated at runtime, the resulting *type form object* encodes the information +supplied in the type expression. This enables a variety of use cases including +runtime type checking, introspection, and metaprogramming. -This PEP makes no Python grammar changes. Correct usage of -``TypeForm[]`` is intended to be enforced only by static and runtime -type checkers and need not be enforced by Python itself at runtime. +Such use cases have proliferated, but there is currently no way to accurately +annotate functions that accept type form objects. Developers are forced to use +an overly-wide type like ``object``, which makes some use cases impossible and +generally reduces type safety. This PEP addresses this limitation by +introducing a new special form ``typing.TypeForm``. +This PEP makes no changes to the Python grammar. ``TypeForm`` is +intended to be enforced only by type checkers, not by the Python runtime. -.. _motivation: Motivation ========== -The introduction of ``TypeForm`` allows new kinds of metaprogramming -functions that operate on type expressions to be type-annotated and -understood by type checkers. +A function that operates on type form objects must understand how type +expression details are encoded in these objects. For example, the expressions +``int | str``, ``"int | str"``, ``list[int]``, and ``MyTypeAlias`` are all +valid type expressions, and they evaluate to instances of ``types.UnionType``, +``builtins.str``, ``types.GenericAlias``, and ``typing.TypeAliasType``, +respectively. -For example, here is a function that checks whether a value is -assignable to a variable of a particular type, and if so returns the -original value: - -:: +There is currently no way to indicate to a type checker that a function accepts +type form objects and knows how to work with them. ``TypeForm`` addresses this +limitation. For example, here is a function that checks whether a value is +assignable to a specified type and returns None if it is not:: def trycast[T](typx: TypeForm[T], value: object) -> T | None: ... -The use of ``TypeForm[]`` and the type variable ``T`` enables the return -type of this function to be influenced by a ``typx`` value passed at -runtime, which is quite powerful. +The use of ``TypeForm`` and the type variable ``T`` describes a relationship +between the type form passed to parameter ``typx`` and the function's +return type. -Here is another function that checks whether a value is assignable to a -variable of a particular type, and if so returns ``True`` (as a special -``TypeIs[]`` bool [#TypeIsPep]_): +``TypeForm`` can also be used with :ref:`TypeIs ` to define +custom type narrowing behaviors:: -:: + def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]: ... - def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]: ... - -The use of ``TypeForm[]`` and ``TypeIs[]`` together enables type -checkers to narrow the return type appropriately depending on what type -expression is passed in: - -:: + request_json: object = ... + if isassignable(request_json, MyTypedDict): + assert_type(request_json, MyTypedDict) # Type of variable is narrowed - request_json: object = ... - if isassignable(request_json, MyTypedDict): - assert_type(request_json, MyTypedDict) # type is narrowed! +The ``isassignable`` function implements something like an enhanced +``isinstance`` check. This is useful for validating whether a value decoded +from JSON conforms to a particular structure of nested ``TypedDict``\ s, +lists, unions, ``Literal``\ s, or any other type form that can be described +with a type expression. This kind of check was alluded to in :pep:`PEP 589 <589#using-typeddict-types>` +but could not be implemented without ``TypeForm``. -That ``isassignable`` function enables a kind of enhanced ``isinstance`` -check which is useful for `checking whether a value decoded from JSON -conforms to a particular structure`_ of nested ``TypedDict``\ s, -lists, unions, ``Literal``\ s, and other types. This kind -of check was alluded to in :pep:`PEP 589 <589#using-typeddict-types>` but could -not be implemented at the time without a notation similar to -``TypeForm[]``. - -.. _checking whether a value decoded from JSON conforms to a particular structure: https://mail.python.org/archives/list/typing-sig@python.org/thread/I5ZOQICTJCENTCDPHLZR7NT42QJ43GP4/ - - -Why can’t ``type[]`` be used? ------------------------------ - -One might think you could define the example functions above to take a -``type[C]`` - which is syntax that already exists - rather than a -``TypeForm[T]``. However if you were to do that then certain type -expressions like ``str | None`` - which are not class objects and -therefore not ``type``\ s at runtime - would be rejected: - -:: - # NOTE: Uses a type[C] parameter rather than a TypeForm[T] - def trycast_type[C](typ: type[C], value: object) -> T | None: ... +Why not ``type[C]``? +-------------------- - trycast_type(str, 'hi') # ok; str is a type - trycast_type(Optional[str], 'hi') # ERROR; Optional[str] is not a type - trycast_type(str | int, 'hi') # ERROR; (str | int) is not a type - trycast_type(MyTypedDict, dict(value='hi')) # questionable; accepted by mypy 1.9.0 +One might think that ``type[C]`` would suffice for these use cases. However, +only class objects (instances of the ``builtins.type`` class) are assignable +to ``type[C]``. Many type form objects do not meet this requirement:: -To solve that problem, ``type[]`` could be widened to include the -additional values allowed by ``TypeForm``. However doing so would lose -``type[]``\ ’s current ability to spell a class object which always -supports instantiation and ``isinstance`` checks, unlike arbitrary type -expression objects. Therefore ``TypeForm`` is proposed as new notation -instead. + def trycast[T](typx: type[T], value: object) -> T | None: ... -For a longer explanation of why we don’t just widen ``type[T]`` to -accept all type expressions, see -:ref:`widen_type_C_to_support_all_type_expressions`. + trycast(str, 'hi') # OK + trycast(Literal['hi'], 'hi') # Type violation + trycast(str | None, 'hi') # Type violation + trycast(MyProtocolClass, obj) # Type violation -.. _runtime_type_checkers_using_typeexpr: +TypeForm use cases +------------------ -Common kinds of functions that would benefit from TypeForm ----------------------------------------------------------- +`A survey of Python libraries`_ reveals several categories of functions that +would benefit from ``TypeForm``: -`A survey of various Python libraries`_ revealed a few kinds of commonly -defined functions which would benefit from ``TypeForm[]``: - -.. _A survey of various Python libraries: https://github.com/python/mypy/issues/9773#issuecomment-2017998886 +.. _A survey of Python libraries: https://github.com/python/mypy/issues/9773#issuecomment-2017998886 - Assignability checkers: - - - Returns whether a value is assignable to a type expression. If so - then also narrows the type of the value to match the type - expression. + - Determines whether a value is assignable to a specified type - Pattern 1: - ``def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]`` + ``def is_assignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]`` - Pattern 2: - ``def ismatch[T](value: object, typx: TypeForm[T]) -> TypeGuard[T]`` + ``def is_match[T](value: object, typx: TypeForm[T]) -> TypeGuard[T]`` - Examples: beartype.\ `is_bearable`_, trycast.\ `isassignable`_, typeguard.\ `check_type`_, xdsl.\ `isa`_ @@ -138,11 +106,12 @@ defined functions which would benefit from ``TypeForm[]``: .. _isa: https://github.com/xdslproject/xdsl/blob/ac12c9ab0d64618475efb98d1d197bdd79f593c3/xdsl/utils/hints.py#L23 - Converters: + - If a value is assignable to (or coercible to) a specified type, + a *converter* returns the value narrowed to (or coerced to) that type. + Otherwise, an exception is raised. - - If a value is assignable to (or coercible to) a type expression, - a *converter* returns the value narrowed to (or coerced to) that type - expression. Otherwise, it raises an exception. - Pattern 1: + ``def convert[T](value: object, typx: TypeForm[T]) -> T`` - Examples: cattrs.BaseConverter.\ `structure`_, trycast.\ `checkcast`_, @@ -181,13 +150,13 @@ defined functions which would benefit from ``TypeForm[]``: .. _make_dataclass: https://github.com/python/typeshed/issues/11653 .. _openapify: https://github.com/Fatal1ty/openapify/blob/c8d968c7c9c8fd7d4888bd2ddbe18ffd1469f3ca/openapify/core/models.py#L16 -The survey also identified some introspection functions that take -annotation expressions as input using plain ``object``\ s which would -*not* gain functionality by marking those inputs as ``TypeForm[]``: +The survey also identified some introspection functions that accept runtime +type forms as input. Today, these functions are annotated with ``object``: - General introspection operations: - - Pattern: ``def get_annotation_info(maybe_annx: object) -> object`` + - Pattern: ``def get_annotation_info(typx: object) -> object`` + - Examples: typing.{`get_origin`_, `get_args`_}, `typing_inspect`_.{is_*_type, get_origin, get_parameters} @@ -196,458 +165,199 @@ annotation expressions as input using plain ``object``\ s which would .. _typing_inspect: https://github.com/ilevkivskyi/typing_inspect?tab=readme-ov-file#readme -Rationale -========= - -Before this PEP existed there were already a few definitions in use to describe -different kinds of type annotations: - -.. code-block:: text - - +----------------------------------+ - | +------------------------------+ | - | | +-------------------------+ | | - | | | +---------------------+ | | | - | | | | Class object | | | | = type[C] - | | | +---------------------+ | | | - | | | Type expression object | | | = TypeForm[T] <-- new! - | | +-------------------------+ | | - | | Annotation expression object | | - | +------------------------------+ | - | Object | = object - +----------------------------------+ - -- :ref:`Class objects `, - spelled as ``type[C]``, support ``isinstance`` checks and are callable. - - - Examples: ``int``, ``str``, ``MyClass`` - -- :ref:`Type expressions ` - include any type annotation which describes a type. - - - Examples: ``list[int]``, ``MyTypedDict``, ``int | str``, - ``Literal['square']``, any class object - -- :ref:`Annotation expressions ` - include any type annotation, including those only valid in specific contexts. - - - Examples: ``Final[int]``, ``Required[str]``, ``ClassVar[str]``, - any type expression - -``TypeForm`` aligns with an existing definition from the above list - -*type expression* - to avoid introducing yet another subset of type annotations -that users of Python typing need to think about. - -``TypeForm`` aligns with *type expression* specifically -because a type expression is already used to parameterize type variables, -which are used in combination with ``TypeIs`` and ``TypeGuard`` to enable -the compelling examples mentioned in :ref:`Motivation `. - -``TypeForm`` does not align with *annotation expression* for reasons given in -:ref:`Rejected Ideas » Accept arbitrary annotation expressions `. - - Specification ============= -A ``TypeForm`` value represents a :ref:`type expression ` -such as ``str | None``, ``dict[str, int]``, or ``MyTypedDict``. -A ``TypeForm`` type is written as -``TypeForm[T]`` where ``T`` is a type or a type variable. It can also be -written without brackets as just ``TypeForm``, which is treated the same as -to ``TypeForm[Any]``. - - -Using TypeForms ---------------- - -A ``TypeForm`` is a new kind of type expression, usable in any context where a -type expression is valid, as a function parameter type, a return type, -or a variable type: - -:: - - def is_union_type(typx: TypeForm) -> bool: ... # parameter type - -:: - - def union_of[S, T](s: TypeForm[S], t: TypeForm[T]) \ - -> TypeForm[S | T]: ... # return type - -:: - - STR_TYPE: TypeForm = str # variable type - -Note however that an *unannotated* variable assigned a type expression literal -will not be inferred to be of ``TypeForm`` type by type checkers because PEP -484 :pep:`reserves that syntax for defining type aliases <484#type-aliases>`: - -- No: - - :: - - STR_TYPE = str # OOPS; treated as a type alias! - -If you want a type checker to recognize a type expression literal in a bare -assignment you’ll need to explicitly declare the assignment-target as -having ``TypeForm`` type: - -- Yes: - - :: - - STR_TYPE: TypeForm = str - -- Yes: - - :: - - STR_TYPE: TypeForm - STR_TYPE = str - -- Okay, but discouraged: - - :: - - STR_TYPE = str # type: TypeForm # the type comment is significant - -``TypeForm`` values can be passed around and assigned just like normal -values: - -:: - - def swap1[S, T](t1: TypeForm[S], t2: TypeForm[T]) -> tuple[TypeForm[T], TypeForm[S]]: - t1_new: TypeForm[T] = t2 # assigns a TypeForm value to a new annotated variable - t2_new: TypeForm[S] = t1 - return (t1_new, t2_new) - - def swap2[S, T](t1: TypeForm[S], t2: TypeForm[T]) -> tuple[TypeForm[T], TypeForm[S]]: - t1_new = t2 # assigns a TypeForm value to a new unannotated variable - t2_new = t1 - assert_type(t1_new, TypeForm[T]) - assert_type(t2_new, TypeForm[S]) - return (t1_new, t2_new) - - # NOTE: A more straightforward implementation would use isinstance() - def ensure_int(value: object) -> None: - value_type: TypeForm = type(value) # assigns a type (a subtype of TypeForm) - assert value_type == int - - -TypeForm Values ---------------- - -A variable of type ``TypeForm[T]`` where ``T`` is a type, can hold any -**type expression object** - the result of evaluating a -:ref:`type expression ` -at runtime - which is a subtype of ``T``. - -Incomplete expressions like a bare ``Optional`` or ``Union`` which do -not spell a type are not ``TypeForm`` values. - -``TypeForm[...]`` is itself a ``TypeForm`` value: - -:: - - OPTIONAL_INT_TYPE: TypeForm = TypeForm[int | None] # OK - assert isassignable(int | None, OPTIONAL_INT_TYPE) - -.. _non_universal_typeexpr: - -``TypeForm[]`` values include *all* type expressions including some -**non-universal type expressions** which are not valid in all annotation contexts. -In particular: - -- ``Self`` (valid only in some contexts) -- ``TypeGuard[...]`` (valid only in some contexts) -- ``TypeIs[...]`` (valid only in some contexts) - - -Explicit TypeForm Values -'''''''''''''''''''''''' - -The syntax ``TypeForm(T)`` (with parentheses) can be used to -spell a ``TypeForm[T]`` value explicitly: - -:: - - NONE = TypeForm(None) - INT1 = TypeForm('int') # stringified type expression - INT2 = TypeForm(int) - -At runtime the ``TypeForm(...)`` callable returns its single argument unchanged. - - -.. _implicit_typeexpr_values: - -Implicit TypeForm Values -'''''''''''''''''''''''' - -Historically static type checkers have only needed to recognize -*type expressions* in contexts where a type expression was expected. -Now *type expression objects* must also be recognized in contexts where a -value expression is expected. - -Static type checkers already recognize **class objects** (``type[C]``): - -- As a value expression, ``C`` has type ``type[C]``, - for each of the following values of C: - - - ``name`` (where ``name`` must refer to a valid in-scope class, type alias, or TypeVar) - - ``name '[' ... ']'`` - - `` '[' ... ']'`` - -The following **unparameterized type expressions** can be recognized unambiguously: - -- As a value expression, ``X`` has type ``TypeForm[X]``, - for each of the following values of X: - - - ```` - - ```` - - ```` - - ```` - - ```` +When a type expression is evaluated at runtime, the resulting value is a +*type form* object. This value encodes the information supplied in the type +expression, and it represents the type described by that type expression. + +``TypeForm`` is a special form that, when used in a type expression, describes +a set of type form objects. It accepts a single type argument, which must be a +valid type expression. ``TypeForm[T]`` describes the set of all type form +objects that represent the type ``T`` or types that are +:term:`assignable to ` ``T``. For example, +``TypeForm[str | None]`` describes the set of all type form objects +that represent a type assignable to ``str | None``:: + + ok1: TypeForm[str | None] = str | None # OK + ok2: TypeForm[str | None] = str # OK + ok3: TypeForm[str | None] = None # OK + ok4: TypeForm[str | None] = Literal[None] # OK + ok5: TypeForm[str | None] = Optional[str] # OK + ok6: TypeForm[str | None] = "str | None" # OK + ok7: TypeForm[str | None] = Any # OK + + err1: TypeForm[str | None] = str | int # Error + err2: TypeForm[str | None] = list[str | None] # Error + +By this same definition, ``TypeForm[Any]`` describes a type form object +that represents the type ``Any`` or any type that is assignable to ``Any``. +Since all types in the Python type system are assignable to ``Any``, +``TypeForm[Any]`` describes the set of all type form objects +evaluated from all valid type expressions. + +The type expression ``TypeForm``, with no type argument provided, is +equivalent to ``TypeForm[Any]``. + + +Implicit ``TypeForm`` Evaluation +'''''''''''''''''''''''''''''''' + +When a static type checker encounters an expression that follows all of the +syntactic, semantic and contextual rules for a type expression as detailed +in the typing spec, it should evaluate the *actual type* of this expression +using its normal rules for value expressions, and it should *also* evaluate +its ``TypeForm`` type. -**None**: The type expression ``None`` (``NoneType``) is ambiguous with the value ``None``, -so must use the explicit ``TypeForm(...)`` syntax: +For example, if a static type checker encounters the expression ``str | None``, +it should evaluate the *actual type* of this expression to be ``UnionType``. +Because this expression is a valid type expression, a type checker should +*also* evaluate its type as ``TypeForm[str | None]``. The resulting type is +assignable to both ``UnionType`` and ``TypeForm``. -- As a value expression, ``TypeForm(None)`` has type ``TypeForm[None]``. -- As a value expression, ``None`` continues to have type ``None``. +When a type checker reports the resulting type (for example, in error messages +or in response to a ``reveal_type`` call), it may reveal its actual type, +the ``TypeForm`` type, or both. Internally, the type checker should retain both +types:: -The following **parameterized type expressions** can be recognized unambiguously: + v1 = str | None + reveal_type(v1) # Revealed type is "UnionType" or + # Revealed type is "TypeForm[str | None]" or + # Revealed type is "UnionType & TypeForm[str | None]" -- As a value expression, ``X`` has type ``TypeForm[X]``, - for each of the following values of X: + v1_actual: UnionType = v1 # OK + v1_type_form: TypeForm[str | None] = v1 # OK - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` + v2 = list[int] + reveal_type(v2) # Revealed type is "type[list[int]]" or + # Revealed type is "TypeForm[list[int]]" or + # Revealed type is "type[list[int]] & TypeForm[list[int]]" -.. _recognizing_annotated: + v2_actual: type = v2 # OK + v2_type_form: TypeForm = v2 # OK -**Annotated**: The type expression ``Annotated[...]`` is ambiguous with -the annotation expression ``Annotated[...]``, -so must be disambiguated based on its argument type: +The ``Annotated`` special form is allowed in type expressions, so it should +also be evaluated as a ``TypeForm`` type. Consistent with the typing spec's +rules for ``Annotated``, a static type checker may choose to ignore any +``Annotated`` metadata that it does not understand:: -- As a value expression, ``Annotated[x, ...]`` has type ``type[C]`` - if ``x`` has type ``type[C]``. -- As a value expression, ``Annotated[x, ...]`` has type ``TypeForm[T]`` - if ``x`` has type ``TypeForm[T]``. -- As a value expression, ``Annotated[x, ...]`` has type ``object`` - if ``x`` has a type that is not ``type[C]`` or ``TypeForm[T]``. + v3 = Annotated[int | str, "metadata"] + reveal_type(v3) # Revealed type is "_AnnotatedAlias & TypeForm[int | str]" or + # Revealed type is "UnionType & TypeForm[int | str]" + v4: TypeForm[Annotated[int | str, "metadata"]] = int | str # OK -**Union**: The type expression ``T1 | T2`` is ambiguous with -the value ``int1 | int2``, ``set1 | set2``, ``dict1 | dict2``, and more, -so must use the explicit ``TypeForm(...)`` syntax: +A string literal expression containing a valid type expression should likewise +be evaluated as a ``TypeForm`` type:: -- Yes: - - :: - - if isassignable(value, TypeForm(int | str)): ... - -- No: - - :: - - if isassignable(value, int | str): ... - -Future PEPs may make it possible to recognize the value expression ``T1 | T2`` directly as an -implicit TypeForm value and avoid the need to use the explicit ``TypeForm(...)`` syntax, -but that work is :ref:`deferred for now `. - -The **stringified type expression** ``"T"`` is ambiguous with both -the stringified annotation expression ``"T"`` -and the string literal ``"T"``, -so must use the explicit ``TypeForm(...)`` syntax: - -- As a value expression, ``TypeForm("T")`` has type ``TypeForm[T]``, - where ``T`` is a valid type expression -- As a value expression, ``"T"`` continues to have type ``Literal["T"]``. - -No other kinds of type expressions currently exist. - -New kinds of type expressions that are introduced should define how they -will be recognized in a value expression context. - - -Literal[] TypeForms -''''''''''''''''''' - -A value of ``Literal[...]`` type is *not* considered assignable to -a ``TypeForm`` variable even if all of its members spell valid types because -dynamic values are not allowed in type expressions: - -:: - - STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str' - STRS_TYPE: TypeForm = STRS_TYPE_NAME # ERROR: Literal[] value is not a TypeForm - -However ``Literal[...]`` itself is still a ``TypeForm``: - -:: - - DIRECTION_TYPE: TypeForm[Literal['left', 'right']] = Literal['left', 'right'] # OK - - -Static vs. Runtime Representations of TypeForms -''''''''''''''''''''''''''''''''''''''''''''''' - -A ``TypeForm`` value appearing statically in a source file may be normalized -to a different representation at runtime. For example string-based -forward references are normalized at runtime to be ``ForwardRef`` instances -in some contexts: [#forward_ref_normalization]_ - -:: - - >>> IntTree = list[typing.Union[int, 'IntTree']] - >>> IntTree - list[typing.Union[int, ForwardRef('IntTree')]] - -The runtime representations of ``TypeForm``\ s are considered implementation -details that may change over time and therefore static type checkers are -not required to recognize them: - -:: - - INT_TREE: TypeForm = ForwardRef('IntTree') # ERROR: Runtime-only form - -Runtime type checkers that wish to assign a runtime-only representation -of a type expression to a ``TypeForm[]`` variable must use ``cast()`` to -avoid errors from static type checkers: - -:: - - INT_TREE = cast(TypeForm, ForwardRef('IntTree')) # OK - - -Subtyping ---------- - -Whether a ``TypeForm`` value can be assigned from one variable to another is -determined by the following rules: - -Relationship with type -'''''''''''''''''''''' - -``TypeForm[]`` is covariant in its argument type, just like ``type[]``: - -- ``TypeForm[T1]`` is a subtype of ``TypeForm[T2]`` iff ``T1`` is a - subtype of ``T2``. -- ``type[C1]`` is a subtype of ``TypeForm[C2]`` iff ``C1`` is a subtype - of ``C2``. - -An unparameterized ``type`` can be assigned to an unparameterized ``TypeForm`` -but not the other way around: - -- ``type[Any]`` is assignable to ``TypeForm[Any]``. (But not the - other way around.) - -Relationship with object -'''''''''''''''''''''''' - -``TypeForm[]`` is a kind of ``object``, just like ``type[]``: - -- ``TypeForm[T]`` for any ``T`` is a subtype of ``object``. - -``TypeForm[T]``, where ``T`` is a type variable, is assumed to have all -the attributes and methods of ``object`` and is not callable. + v5 = "set[str]" + reveal_type(v5) # Revealed type is "Literal['set[str]'] & TypeForm[set[str]]" or + # Revealed type is "str & TypeForm[set[str]]" +Expressions that violate one or more of the syntactic, semantic, or contextual +rules for type expressions should not evaluate to a ``TypeForm`` type. The rules +for type expression validity are explained in detail within the typing spec, so +they are not repeated here:: -Interactions with isinstance() and issubclass() ------------------------------------------------ + bad1: TypeForm = tuple() # Error: Call expression not allowed in type expression + bad2: TypeForm = (1, 2) # Error: Tuple expression not allowed in type expression + bad3: TypeForm = 1 # Non-class object not allowed in type expression + bad4: TypeForm = Self # Error: Self not allowed outside of a class + bad5: TypeForm = Literal[var] # Error: Variable not allowed in type expression + bad6: TypeForm = Literal[f""] # Error: f-strings not allowed in type expression + bad7: TypeForm = ClassVar[int] # Error: ClassVar not allowed in type expression + bad8: TypeForm = Required[int] # Error: Required not allowed in type expression + bad9: TypeForm = Final[int] # Error: Final not allowed in type expression + bad10: TypeForm = Unpack[Ts] # Error: Unpack not allowed in this context + bad11: TypeForm = Optional # Error: Invalid use of Optional special form + bad12: TypeForm = T # Error if T is an out-of-scope TypeVar + bad13: TypeForm = "int + str" # Error: invalid quoted type expression -The ``TypeForm`` special form cannot be used as the ``type`` argument to -``isinstance``: -:: - - >>> isinstance(str, TypeForm) - TypeError: typing.TypeForm cannot be used with isinstance() +Explicit ``TypeForm`` Evaluation +'''''''''''''''''''''''''''''''' - >>> isinstance(str, TypeForm[str]) - TypeError: isinstance() argument 2 cannot be a parameterized generic +``TypeForm`` also acts as a function that can be called with a single argument. +Type checkers should validate that this argument is a valid type expression:: -The ``TypeForm`` special form cannot be used as any argument to -``issubclass``: + x1 = TypeForm(str | None) + reveal_type(v1) # Revealed type is "UnionType & TypeForm[str | None]" -:: + x2 = TypeForm("list[int]") + revealed_type(v2) # Revealed type is "Literal['list[int]'] & TypeForm[list[int]]" - >>> issubclass(TypeForm, object) - TypeError: issubclass() arg 1 must be a class + x3 = TypeForm('type(1)') # Error: invalid type expression - >>> issubclass(object, TypeForm) - TypeError: typing.TypeForm cannot be used with issubclass() +At runtime the ``TypeForm(...)`` callable simply returns the value passed to it. +This explicit syntax serves two purposes. First, it documents the developer's +intent to use the value as a type form object. Second, static type checkers +validate that all rules for type expressions are followed:: -Affected signatures in the standard library -------------------------------------------- + x4 = type(int) # No error, evaluates to "type[int]" + + x5 = TypeForm(type(int)) # Error: call not allowed in type expression -Changed signatures -'''''''''''''''''' -The following signatures related to type expressions introduce -``TypeForm`` where previously ``object`` or ``Any`` existed: +Assignability +------------- -- ``typing.cast`` -- ``typing.assert_type`` +``TypeForm`` has a single type parameter, which is covariant. That means +``TypeForm[B]`` is assignable to ``TypeForm[A]`` if ``B`` is assignable to +``A``:: -Unchanged signatures -'''''''''''''''''''' + def get_type_form() -> TypeForm[int]: ... -The following signatures related to annotation expressions continue to -use ``object`` and remain unchanged: + t1: TypeForm[int | str] = get_type_form() # OK + t2: TypeForm[str] = get_type_form() # Error -- ``typing.get_origin`` -- ``typing.get_args`` +``type[T]`` is a subtype of ``TypeForm[T]``, which means that ``type[B]`` is +assignable to ``TypeForm[A]`` if ``B`` is assignable to ``A``:: -The following signatures related to class objects continue to use -``type`` and remain unchanged: + def get_type() -> type[int]: ... -- ``builtins.isinstance`` -- ``builtins.issubclass`` -- ``builtins.type`` + t3: TypeForm[int | str] = get_type() # OK + t4: TypeForm[str] = get_type() # Error -``typing.get_type_hints(..., include_extras=False)`` nearly returns only type -expressions in Python 3.12, stripping out most type qualifiers -(``Required, NotRequired, ReadOnly, Annotated``) but currently preserves a -few type qualifiers which are only allowed in annotation expressions -(``ClassVar, Final, InitVar, Unpack``). It may be desirable to alter the -behavior of this function in the future to also strip out those -qualifiers and actually return type expressions, although this PEP does -not propose those changes now: +``TypeForm`` is a subtype of ``object`` and is assumed to have all of the +attributes and methods of ``object``. -- ``typing.get_type_hints(..., include_extras=False)`` +If a value with a ``TypeForm`` type is used within an ``typing.assert_type`` +call, the assertion should succeed for both its actual type and its +``TypeForm`` type:: - - Almost returns only type expressions, but not quite + t1 = int | str + assert_type(t1, UnionType) # OK + assert_type(t1, TypeForm[int | str]) # OK + assert_type(t1, type[int] | type[str]) # Error + assert_type(t1, TypeForm[int]) # Error -- ``typing.get_type_hints(..., include_extras=True)`` + t2 = "int | str" + assert_type(t2, str) # OK + assert_type(t2, TypeForm[int | str]) # OK + assert_type(t2, type[int | str]) # Error - - Returns annotation expressions +Backward Compatibility +====================== -Backwards Compatibility -======================= - -The rules for recognizing type expression objects -in a value expression context were not previously defined, so static type checkers -`varied in what types were assigned `_ -to such objects. Existing programs manipulating type expression objects -were already limited in manipulating them as plain ``object`` values, -and such programs should not break with -:ref:`the newly-defined rules `. +This PEP clarifies static type checker behaviors when evaluating type +expressions in "value expression" contexts (that is, contexts where type +expressions are not mandated by the typing spec). It augments the *actual +type* of these expressions with a ``TypeForm`` type. This approach retains +backward compatibility because the resulting type is compatible with the old +(non-augmented) type. For example, if a static type checker previously +evaluated the type of expression ``str | None`` as ``UnionType``, it will +now evaluate the type of this expression as ``UnionType`` *and* +``TypeForm[str | None]``. How to Teach This ================= -Normally when using type annotations in Python you're concerned with defining -the shape of values allowed to be passed to a function parameter, returned -by a function, or stored in a variable: +Type expressions are typically used in annotations to describe the set of +values accepted by a function parameter, returned by a function, or stored +in a variable: .. code-block:: text @@ -662,65 +372,37 @@ by a function, or stored in a variable: return sum -However type annotations themselves are valid values in Python and can be +Type expressions evaluate to valid *type form* objects at runtime and can be assigned to variables and manipulated like any other data in a program: .. code-block:: text - a variable a type + a variable a type expression | | v v - MAYBE_INT_TYPE: TypeForm = int | None + int_type_form: TypeForm = int | None ^ | - the type of a type - -``TypeForm[]`` is how you spell the type of a variable containing a -type annotation object describing a type. + the type of a type form object -``TypeForm[]`` is similar to ``type[]``, but ``type[]`` can only -spell simple **class objects** like ``int``, ``str``, ``list``, or ``MyClass``. -``TypeForm[]`` by contrast can additionally spell more complex types, -including those with brackets (like ``list[int]``) or pipes (like ``int | None``), -and including special types like ``Any``, ``LiteralString``, or ``Never``. +``TypeForm[]`` is how you spell the type of a *type form* object, which is +a runtime representation of a type. -A ``TypeForm`` variable (``maybe_float: TypeForm``) looks similar to -a ``TypeAlias`` definition (``MaybeFloat: TypeAlias``), but ``TypeForm`` -can only be used where a dynamic value is expected: +``TypeForm`` is similar to ``type``, but ``type`` is compatible only with +**class objects** like ``int``, ``str``, ``list``, or ``MyClass``. +``TypeForm`` accommodates any type form that can be expressed using +a valid type expression, including those with brackets (``list[int]``), union +operators (``int | None``), and special forms (``Any``, ``LiteralString``, +``Never``, etc.). -- No: - - :: - - maybe_float: TypeForm = float | None - def sqrt(n: float) -> maybe_float: ... # ERROR: Can't use TypeForm value in a type annotation - -- Okay, but discouraged in Python 3.12+: - - :: +Most programmer will not define their *own* functions that accept a ``TypeForm`` +parameter or returns a ``TypeForm`` value. It is more common to pass a type +form object to a library function that knows how to decode and use such objects. - MaybeFloat: TypeAlias = float | None - def sqrt(n: float) -> MaybeFloat: ... - -- Yes: - - :: - - type MaybeFloat = float | None - def sqrt(n: float) -> MaybeFloat: ... - -It is uncommon for a programmer to define their *own* function which accepts -a ``TypeForm`` parameter or returns a ``TypeForm`` value. Instead it is more common -for a programmer to pass a literal type expression to an *existing* function -accepting a ``TypeForm`` input which was imported from a runtime type checker -library. - -For example the ``isassignable`` function from the ``trycast`` library +For example, the ``isassignable`` function in the ``trycast`` library can be used like Python's built-in ``isinstance`` function to check whether -a value matches the shape of a particular type. -``isassignable`` will accept *any* kind of type as an input because its input -is a ``TypeForm``. By contrast ``isinstance`` only accepts a simple class object -(a ``type[]``) as input: +a value matches the shape of a particular type. ``isassignable`` accepts *any* +type form object an input. - Yes: @@ -739,29 +421,22 @@ is a ``TypeForm``. By contrast ``isinstance`` only accepts a simple class object ... There are :ref:`many other runtime type checkers ` -providing useful functions that accept a ``TypeForm``. - +providing useful functions that accept type form objects as input. -.. _advanced_examples: Advanced Examples ================= -If you want to write your own runtime type checker or some other -kind of function that manipulates types as values at runtime, -this section gives examples of how you might implement such a function -using ``TypeForm``. - +If you want to write your own runtime type checker or a function that +manipulates type form objects as values at runtime, this section provides +examples of how such a function can use ``TypeForm``. -Introspecting TypeForm Values ------------------------------ -A ``TypeForm`` is very similar to an ``object`` at runtime, with no additional -attributes or methods defined. +Introspecting type form objects +------------------------------- -You can use existing introspection functions like ``typing.get_origin`` and -``typing.get_args`` to extract the components of a type expression that looks -like ``Origin[Arg1, Arg2, ..., ArgN]``: +Functions like ``typing.get_origin`` and ``typing.get_args`` can be used to +extract components of some type form objects. :: @@ -772,8 +447,8 @@ like ``Origin[Arg1, Arg2, ..., ArgN]``: typx = cast(TypeForm[T], typing.get_args(typx)[0]) return typx -You can also use ``isinstance`` and ``is`` to distinguish one kind of -type expression from another: +``isinstance`` and ``is`` can also be used to distinguish between different +kinds of type form objects: :: @@ -781,44 +456,43 @@ type expression from another: import typing def split_union(typx: TypeForm) -> tuple[TypeForm, ...]: - if isinstance(typx, types.UnionType): # X | Y - return cast(tuple[TypeForm, ...], typing.get_args(typx)) - if typing.get_origin(typx) is typing.Union: # Union[X, Y] - return cast(tuple[TypeForm, ...], typing.get_args(typx)) - if typx in (typing.Never, typing.NoReturn,): + if isinstance(typ, types.UnionType): # X | Y + return cast(tuple[TypeForm, ...], typing.get_args(typ)) + if typing.get_origin(typ) is typing.Union: # Union[X, Y] + return cast(tuple[TypeForm, ...], typing.get_args(typ)) + if typ in (typing.Never, typing.NoReturn,): return () - return (typx,) + return (typ,) Combining with a type variable ------------------------------ -``TypeForm[]`` can be parameterized by a type variable that is used elsewhere within -the same function definition: +``TypeForm`` can be parameterized by a type variable that is used elsewhere +within the same function definition: :: def as_instance[T](typx: TypeForm[T]) -> T | None: - return typx() if isinstance(typx, type) else None + return typ() if isinstance(typ, type) else None -Combining with type[] ---------------------- +Combining with ``type`` +----------------------- -Both ``TypeForm[]`` and ``type[]`` can be parameterized by the same type +Both ``TypeForm`` and ``type`` can be parameterized by the same type variable within the same function definition: :: def as_type[T](typx: TypeForm[T]) -> type[T] | None: - return typx if isinstance(typx, type) else None + return typ if isinstance(typ, type) else None -Combining with TypeIs[] and TypeGuard[] ---------------------------------------- +Combining with ``TypeIs`` and ``TypeGuard`` +------------------------------------------- -A type variable parameterizing a ``TypeForm[]`` can also be used by a ``TypeIs[]`` -within the same function definition: +A type variable can also be used by a ``TypeIs`` or ``TypeGuard`` return type: :: @@ -830,59 +504,32 @@ within the same function definition: else: assert_type(count, str) -or by a ``TypeGuard[]`` within the same function definition: - -:: - - def isdefault[T](value: object, typx: TypeForm[T]) -> TypeGuard[T]: - return (value == typx()) if isinstance(typx, type) else False - - value: int | str = '' - if isdefault(value, int): - assert_type(value, int) - assert 0 == value - elif isdefault(value, str): - assert_type(value, str) - assert '' == value - else: - assert_type(value, int | str) - Challenges When Accepting All TypeForms --------------------------------------- -A function that takes an *arbitrary* ``TypeForm`` as -input must support a large variety of possible type expressions and is -not easy to write. Some challenges faced by such a function include: +A function that takes an *arbitrary* ``TypeForm`` as input must support a +variety of possible type form object. Such functions are not easy to write. -- An ever-increasing number of typing special forms are introduced with - each new Python version which must be recognized, with special - handling required for each one. -- Stringified type annotations [#strann_less_common]_ (like ``'list[str]'``) - must be *parsed* (to something like ``typing.List[str]``) to be introspected. - - - In practice it is extremely difficult for stringified type - annotations to be handled reliably at runtime, so runtime type - checkers may opt to not support them at all. - -- Resolving string-based forward references inside type - expressions to actual values must typically be done using ``eval()``, - which is difficult/impossible to use in a safe way. -- Recursive types like ``IntTree = list[typing.Union[int, 'IntTree']]`` - are not possible to fully resolve. -- Supporting user-defined generic types (like Django’s - ``QuerySet[User]``) requires user-defined functions to - recognize/parse, which a runtime type checker should provide a - registration API for. +- New special forms are introduced with each new Python version, and + special handling may be required for each one. +- Quoted annotations [#quoted_less_common]_ (like ``'list[str]'``) + must be *parsed* (to something like ``list[str]``). +- Resolving quoted forward references inside type expressions is typically + done with ``eval()``, which is difficult to use in a safe way. +- Recursive types like ``IntTree = list[int | 'IntTree']`` are difficult + to resolve. +- User-defined generic types (like Django’s ``QuerySet[User]``) can introduce + non-standard behaviors that requite runtime support. Reference Implementation ======================== -The following will be true when -`mypy#9773 `__ is implemented: +Pyright (version 1.1.379) provides a reference implementation for ``TypeForm``. - The mypy type checker supports ``TypeForm`` types. +Mypy contributors also `plan to implement ` +support for ``TypeForm``. A reference implementation of the runtime component is provided in the ``typing_extensions`` module. @@ -891,81 +538,59 @@ A reference implementation of the runtime component is provided in the Rejected Ideas ============== -.. _widen_type_C_to_support_all_type_expressions: +Alternative names +----------------- -Widen type[C] to support all type expressions ---------------------------------------------- +Alternate names were considered for ``TypeForm``. ``TypeObject`` +and ``TypeType`` were deemed too generic. ``TypeExpression`` and ``TypeExpr`` +were also considered, but these were considered confusing because these objects +are not themselves "expressions" but rather the result of evaluating a type +expression. -``type`` was `designed`_ to only be used to describe class objects. A -class object can always be used as the second argument of ``isinstance()`` -and can usually be instantiated by calling it. -``TypeForm`` on the other hand is typically introspected by the user in -some way, is not necessarily directly instantiable, and is not -necessarily directly usable in a regular ``isinstance()`` check. +Widen ``type[C]`` to support all type expressions +------------------------------------------------- -It would be possible to widen ``type`` to include the additional values -allowed by ``TypeForm`` but it would reduce clarity about the user’s -intentions when working with a ``type``. Different concepts and usage -patterns; different spellings. +``type`` was `designed`_ to describe class objects, subclasses of the +``type`` class. A value with the type ``type`` is assumed to be instantiable +through a constructor call. Widening the meaning of ``type`` to represent +arbitrary type form objects would present backward compatibility problems +and would eliminate a way to describe the set of values limited to subclasses +of ``type``. .. _designed: https://mail.python.org/archives/list/typing-sig@python.org/message/D5FHORQVPHX3BHUDGF3A3TBZURBXLPHD/ -.. _accept_arbitrary_annotation_expressions: - Accept arbitrary annotation expressions --------------------------------------- -Certain typing special forms can be used in *some* but not *all* -annotation contexts: +Certain special forms act as type qualifiers and can be used in +*some* but not *all* annotation contexts: -For example ``Final[]`` can be used as a variable type but not as a -parameter type or a return type: +For example. the type qualifier ``Final`` can be used as a variable type but +not as a parameter type or a return type: :: some_const: Final[str] = ... # OK - def foo(not_reassignable: Final[object]): ... # ERROR: Final[] not allowed here - - def nonsense() -> Final[object]: ... # ERROR: Final[] not meaningful here + def foo(not_reassignable: Final[object]): ... # Error: Final not allowed here -``TypeForm[T]`` does not allow matching such annotation expressions -because it is not clear what it would mean for such an expression -to parameterized by a type variable in position ``T``: + def nonsense() -> Final[object]: ... # Error: Final not alowed here -:: +With the exception of ``Annotated``, type qualifiers are not allowed in type +expressions. ``TypeForm`` is limited to type expressions because its +assignability rules are based on the assignability rules for types. It is +nonsensical to ask whether ``Final[int]`` is assignable to ``int`` because the +former is not a valid type expression. - def ismatch[T](value: object, typx: TypeForm[T]) -> TypeGuard[T]: ... +Functions that wish to operate on objects that are evaluated from *any* +annotation expressions can continue to accept such inputs as ``object`` +parameters. - def foo(some_arg): - if ismatch(some_arg, Final[int]): # ERROR: Final[int] is not a TypeForm - reveal_type(some_arg) # ? NOT Final[int], because invalid for a parameter -Functions that wish to operate on *all* kinds of annotation expressions, -including those that are not ``TypeForm``\ s, can continue to accept such -inputs as ``object`` parameters, as they must do so today. - - -Accept only universal type expressions --------------------------------------- - -Earlier drafts of this PEP only allowed ``TypeForm[]`` to match the subset -of type expressions which are valid in *all* contexts, excluding -:ref:`non-universal type expressions `. -However doing that would effectively -create a new subset of annotation expressions that Python typing users -would have to understand, on top of all the existing distinctions between -“class objects”, “type expressions”, and “annotation expressions”. - -To avoid introducing yet another concept that everyone has to learn, -this proposal just rounds ``TypeForm[]`` to exactly match the existing -definition of a “type expression”. - - -Support pattern matching on type expressions --------------------------------------------- +Pattern matching on type forms +------------------------------ It was asserted that some functions may wish to pattern match on the interior of type expressions in their signatures. @@ -977,71 +602,53 @@ Consider the following possible pattern matching syntax: :: @overload - def checkcast(typx: TypeForm[AT=Annotated[T, *Anns]], value: str) -> T: ... + def checkcast(typx: TypeForm[AT=Annotated[T, *A]], value: str) -> T: ... @overload def checkcast(typx: TypeForm[UT=Union[*Ts]], value: str) -> Union[*Ts]: ... @overload def checkcast(typx: type[C], value: str) -> C: ... # ... (more) -All functions observed in the wild that conceptually take a ``TypeForm[]`` -generally try to support *all* kinds of type expressions, so it doesn’t -seem valuable to enumerate a particular subset. +All functions observed in the wild that conceptually accept type form +objects generally try to support *all* kinds of type expressions, so it +doesn’t seem valuable to enumerate a particular subset. -Additionally the above syntax isn’t precise enough to fully describe the -actual input constraints for a typical function in the wild. For example -many functions recognize un-stringified type expressions like -``list[Movie]`` but may not recognize type expressions with stringified -subcomponents like ``list['Movie']``. +Additionally, the above syntax isn’t precise enough to fully describe the +input constraints for a typical function in the wild. For example, many +functions do not support type expressions with quoted subexpressions +like ``list['Movie']``. -A second use case for pattern matching on the interior of type -expressions is to explicitly match an ``Annotated[]`` form to pull out the -interior type argument and strip away the metadata: +A second use case for pattern matching is to explicitly match an ``Annotated`` +form to extract the interior type argument and strip away any metadata: :: def checkcast( - typx: TypeForm[T] | TypeForm[AT=Annotated[T, *Anns]], + typx: TypeForm[T] | TypeForm[AT=Annotated[T, *A]], value: object ) -> T: -However ``Annotated[T, metadata]`` is already treated equivalent to ``T`` anyway. -There’s no additional value in being explicit about this behavior. -The example above could be more-straightforwardly written as the equivalent: +However, ``Annotated[T, metadata]`` is already treated equivalent to ``T`` +by static type checkers. There’s no additional value in being explicit about +this behavior. The example above could more simply be written as the equivalent: :: def checkcast(typx: TypeForm[T], value: object) -> T: -.. _recognize_uniontype_as_implicit_typeexpr_value: - -Recognize (T1 | T2) as an implicit TypeForm value -------------------------------------------------- - -It would be nice if a value expression like ``int | str`` could be recognized -as an implicit ``TypeForm`` value and be used directly in a context where a -``TypeForm`` was expected. However making that possible would require making -changes to the rules that type checkers use for the ``|`` operator. These rules -are currently underspecified and would need to be make explicit first, -before making changes to them. The PEP author is not sufficiently motivated to -take on that specification work at the time of writing. - - Footnotes ========= -.. [#type_c] - :pep:`Type[C] spells a class object <484#the-type-of-class-objects>` +.. [#type_t] + :ref:`Type[T] spells a class object ` -.. [#TypeIsPep] - :pep:`TypeIs[T] is similar to bool <742>` +.. [#TypeIs] + :ref:`TypeIs[T] is similar to bool ` .. [#DataclassInitVar] - ``dataclass.make_dataclass`` accepts ``InitVar[...]`` as a special case - in addition to type expressions. Therefore it may unfortunately be necessary - to continue annotating its ``type`` parameter as ``object`` rather - than ``TypeForm``. + ``dataclass.make_dataclass`` allows the type qualifier ``InitVar[...]``, + so ``TypeForm`` cannot be used in this case. .. [#forward_ref_normalization] Special forms normalize string arguments to ``ForwardRef`` instances @@ -1049,12 +656,11 @@ Footnotes Runtime type checkers may wish to implement similar functions when working with string-based forward references. -.. [#strann_less_common] - Stringified type annotations are expected to become less common - starting in Python 3.14 when :pep:`deferred annotations <649>` - become available. However there is a large amount of existing code from - earlier Python versions relying on stringified type annotations that will - still need to be supported for several years. +.. [#quoted_less_common] + Quoted annotations are expected to become less common starting in Python + 3.14 when :pep:`deferred annotations <649>` is implemented. However, + code written for earlier Python versions relies on quoted annotations and + will need to be supported for several years. Copyright From 57367660b2341434c2f62cb2a5c9e204fc43f119 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Sun, 1 Sep 2024 13:37:24 -0700 Subject: [PATCH 03/13] Fixed rst/sphynx formatting issues found by CI tests. --- peps/pep-0747.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 8006ed13857..d1f06c81416 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -1,6 +1,6 @@ PEP: 747 Title: Annotating Type Forms -Author: David Foster , Eric Traut +Author: David Foster , Eric Traut Sponsor: Jelle Zijlstra Discussions-To: https://discuss.python.org/t/pep-747-typeexpr-type-hint-for-a-type-expression/55984 Status: Draft @@ -528,7 +528,7 @@ Reference Implementation Pyright (version 1.1.379) provides a reference implementation for ``TypeForm``. -Mypy contributors also `plan to implement ` +Mypy contributors also `plan to implement `__ support for ``TypeForm``. A reference implementation of the runtime component is provided in the From c87d2619ad92dc5431802d6db555de185548f6c3 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Sun, 1 Sep 2024 13:40:48 -0700 Subject: [PATCH 04/13] Fix more CI errors. --- peps/pep-0747.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index d1f06c81416..16af0ffdbbf 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -94,9 +94,13 @@ would benefit from ``TypeForm``: - Assignability checkers: - Determines whether a value is assignable to a specified type - Pattern 1: + ``def is_assignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]`` + - Pattern 2: + ``def is_match[T](value: object, typx: TypeForm[T]) -> TypeGuard[T]`` + - Examples: beartype.\ `is_bearable`_, trycast.\ `isassignable`_, typeguard.\ `check_type`_, xdsl.\ `isa`_ From 694d0944830fd6f97e59f6ad3a70536ce2a41417 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Sun, 1 Sep 2024 13:43:00 -0700 Subject: [PATCH 05/13] Fixed another indentation issue. --- peps/pep-0747.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 16af0ffdbbf..1f32e65760f 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -110,6 +110,7 @@ would benefit from ``TypeForm``: .. _isa: https://github.com/xdslproject/xdsl/blob/ac12c9ab0d64618475efb98d1d197bdd79f593c3/xdsl/utils/hints.py#L23 - Converters: + - If a value is assignable to (or coercible to) a specified type, a *converter* returns the value narrowed to (or coerced to) that type. Otherwise, an exception is raised. From 7705161d81c1804c277dd45a5354dd0ffb8bf5a2 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Sun, 1 Sep 2024 13:49:00 -0700 Subject: [PATCH 06/13] Made title levels consistent. --- peps/pep-0747.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 1f32e65760f..670ccfb2892 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -207,7 +207,7 @@ equivalent to ``TypeForm[Any]``. Implicit ``TypeForm`` Evaluation -'''''''''''''''''''''''''''''''' +-------------------------------- When a static type checker encounters an expression that follows all of the syntactic, semantic and contextual rules for a type expression as detailed @@ -280,7 +280,7 @@ they are not repeated here:: Explicit ``TypeForm`` Evaluation -'''''''''''''''''''''''''''''''' +-------------------------------- ``TypeForm`` also acts as a function that can be called with a single argument. Type checkers should validate that this argument is a valid type expression:: From 5141ffdc40966bc90c5fb7b4fc17cec1730d23a8 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Sun, 1 Sep 2024 14:01:22 -0700 Subject: [PATCH 07/13] Another attempt to appease the sphynx gods. --- peps/pep-0747.rst | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 670ccfb2892..d0234b2dc74 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -51,8 +51,8 @@ The use of ``TypeForm`` and the type variable ``T`` describes a relationship between the type form passed to parameter ``typx`` and the function's return type. -``TypeForm`` can also be used with :ref:`TypeIs ` to define -custom type narrowing behaviors:: +``TypeForm`` can also be used with `TypeIs `__ +to define custom type narrowing behaviors:: def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]: ... @@ -425,9 +425,6 @@ type form object an input. if isinstance(some_object, MyTypedDict): # ERROR: MyTypedDict is not a type[] ... -There are :ref:`many other runtime type checkers ` -providing useful functions that accept type form objects as input. - Advanced Examples ================= @@ -646,10 +643,10 @@ Footnotes ========= .. [#type_t] - :ref:`Type[T] spells a class object ` + `Type[T] `__ spells a class object .. [#TypeIs] - :ref:`TypeIs[T] is similar to bool ` + `TypeIs[T] ` is similar to bool .. [#DataclassInitVar] ``dataclass.make_dataclass`` allows the type qualifier ``InitVar[...]``, From 2831f8e6b7499eeb7c0d4368030951eb47e9da60 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Sun, 1 Sep 2024 14:04:22 -0700 Subject: [PATCH 08/13] Another attempt to appease the sphynx gods. --- peps/pep-0747.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index d0234b2dc74..9a0e1bfcf14 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -643,10 +643,12 @@ Footnotes ========= .. [#type_t] - `Type[T] `__ spells a class object + `Type[T] `__ + spells a class object .. [#TypeIs] - `TypeIs[T] ` is similar to bool + `TypeIs[T] `__ + is similar to bool .. [#DataclassInitVar] ``dataclass.make_dataclass`` allows the type qualifier ``InitVar[...]``, From 2df0bd3c315ff751efddbab6b578dbbea039827f Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Mon, 2 Sep 2024 08:58:30 -0700 Subject: [PATCH 09/13] Update peps/pep-0747.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0747.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 9a0e1bfcf14..9443c5925e7 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -51,7 +51,7 @@ The use of ``TypeForm`` and the type variable ``T`` describes a relationship between the type form passed to parameter ``typx`` and the function's return type. -``TypeForm`` can also be used with `TypeIs `__ +``TypeForm`` can also be used with :ref:`typing:typeis` to define custom type narrowing behaviors:: def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]: ... From dd9293086146dd99270fd0ac8877cf25c3f34700 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Mon, 2 Sep 2024 08:58:36 -0700 Subject: [PATCH 10/13] Update peps/pep-0747.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0747.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 9443c5925e7..5111d2b58fd 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -647,8 +647,7 @@ Footnotes spells a class object .. [#TypeIs] - `TypeIs[T] `__ - is similar to bool + :ref:`TypeIs[T] ` is similar to bool .. [#DataclassInitVar] ``dataclass.make_dataclass`` allows the type qualifier ``InitVar[...]``, From 390d958d8b9e0367d0c4da02ab72563364529280 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Mon, 2 Sep 2024 08:58:40 -0700 Subject: [PATCH 11/13] Update peps/pep-0747.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0747.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 5111d2b58fd..5929bd31fa6 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -643,8 +643,7 @@ Footnotes ========= .. [#type_t] - `Type[T] `__ - spells a class object + :ref:`Type[T] ` spells a class object .. [#TypeIs] :ref:`TypeIs[T] ` is similar to bool From d443f1fa1cb06d871e35813c8fbbed10027ff2d4 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Fri, 13 Sep 2024 15:45:40 -0700 Subject: [PATCH 12/13] Incorporated PR feedback from last draft. --- peps/pep-0747.rst | 133 ++++++++++++++++------------------------------ 1 file changed, 47 insertions(+), 86 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 5929bd31fa6..65c2d3b6654 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -34,9 +34,9 @@ Motivation ========== A function that operates on type form objects must understand how type -expression details are encoded in these objects. For example, the expressions -``int | str``, ``"int | str"``, ``list[int]``, and ``MyTypeAlias`` are all -valid type expressions, and they evaluate to instances of ``types.UnionType``, +expression details are encoded in these objects. For example, ``int | str``, +``"int | str"``, ``list[int]``, and ``MyTypeAlias`` are all valid type +expressions, and they evaluate to instances of ``types.UnionType``, ``builtins.str``, ``types.GenericAlias``, and ``typing.TypeAliasType``, respectively. @@ -51,8 +51,8 @@ The use of ``TypeForm`` and the type variable ``T`` describes a relationship between the type form passed to parameter ``typx`` and the function's return type. -``TypeForm`` can also be used with :ref:`typing:typeis` -to define custom type narrowing behaviors:: +``TypeForm`` can also be used with :ref:`typing:typeis` to define custom type +narrowing behaviors:: def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]: ... @@ -64,8 +64,9 @@ The ``isassignable`` function implements something like an enhanced ``isinstance`` check. This is useful for validating whether a value decoded from JSON conforms to a particular structure of nested ``TypedDict``\ s, lists, unions, ``Literal``\ s, or any other type form that can be described -with a type expression. This kind of check was alluded to in :pep:`PEP 589 <589#using-typeddict-types>` -but could not be implemented without ``TypeForm``. +with a type expression. This kind of check was alluded to in +:pep:`PEP 589 <589#using-typeddict-types>` but could not be implemented without +``TypeForm``. Why not ``type[C]``? @@ -92,6 +93,7 @@ would benefit from ``TypeForm``: .. _A survey of Python libraries: https://github.com/python/mypy/issues/9773#issuecomment-2017998886 - Assignability checkers: + - Determines whether a value is assignable to a specified type - Pattern 1: @@ -165,6 +167,9 @@ type forms as input. Today, these functions are annotated with ``object``: - Examples: typing.{`get_origin`_, `get_args`_}, `typing_inspect`_.{is_*_type, get_origin, get_parameters} +These functions accept values evaluated from arbitrary annotation expressions, +not just type expressions, so they cannot be altered to use ``TypeForm``. + .. _get_origin: https://docs.python.org/3/library/typing.html#typing.get_origin .. _get_args: https://docs.python.org/3/library/typing.html#typing.get_args .. _typing_inspect: https://github.com/ilevkivskyi/typing_inspect?tab=readme-ov-file#readme @@ -211,53 +216,27 @@ Implicit ``TypeForm`` Evaluation When a static type checker encounters an expression that follows all of the syntactic, semantic and contextual rules for a type expression as detailed -in the typing spec, it should evaluate the *actual type* of this expression -using its normal rules for value expressions, and it should *also* evaluate -its ``TypeForm`` type. +in the typing spec, the evaluated type of this expression should be assignable +to ``TypeForm[T]`` if the type it describes is assignable to ``T``. For example, if a static type checker encounters the expression ``str | None``, -it should evaluate the *actual type* of this expression to be ``UnionType``. -Because this expression is a valid type expression, a type checker should -*also* evaluate its type as ``TypeForm[str | None]``. The resulting type is -assignable to both ``UnionType`` and ``TypeForm``. - -When a type checker reports the resulting type (for example, in error messages -or in response to a ``reveal_type`` call), it may reveal its actual type, -the ``TypeForm`` type, or both. Internally, the type checker should retain both -types:: - - v1 = str | None - reveal_type(v1) # Revealed type is "UnionType" or - # Revealed type is "TypeForm[str | None]" or - # Revealed type is "UnionType & TypeForm[str | None]" - - v1_actual: UnionType = v1 # OK - v1_type_form: TypeForm[str | None] = v1 # OK - - v2 = list[int] - reveal_type(v2) # Revealed type is "type[list[int]]" or - # Revealed type is "TypeForm[list[int]]" or - # Revealed type is "type[list[int]] & TypeForm[list[int]]" - - v2_actual: type = v2 # OK - v2_type_form: TypeForm = v2 # OK - -The ``Annotated`` special form is allowed in type expressions, so it should -also be evaluated as a ``TypeForm`` type. Consistent with the typing spec's -rules for ``Annotated``, a static type checker may choose to ignore any -``Annotated`` metadata that it does not understand:: - - v3 = Annotated[int | str, "metadata"] - reveal_type(v3) # Revealed type is "_AnnotatedAlias & TypeForm[int | str]" or - # Revealed type is "UnionType & TypeForm[int | str]" +it may normally evaluate its type as ``UnionType`` because it produces a +runtime value that is an instance of ``types.UnionType``. However, because +this expression is a valid type expression, it is also assignable to the +type ``TypeForm[str | None]``. + +The ``Annotated`` special form is allowed in type expressions, so it can +also appear in an expression that is assignable to ``TypeForm``. Consistent +with the typing spec's rules for ``Annotated``, a static type checker may choose +to ignore any ``Annotated`` metadata that it does not understand:: + + v3: TypeForm[int | str] = Annotated[int | str, "metadata"] # OK v4: TypeForm[Annotated[int | str, "metadata"]] = int | str # OK A string literal expression containing a valid type expression should likewise -be evaluated as a ``TypeForm`` type:: +be assignable to ``TypeForm``:: - v5 = "set[str]" - reveal_type(v5) # Revealed type is "Literal['set[str]'] & TypeForm[set[str]]" or - # Revealed type is "str & TypeForm[set[str]]" + v5: TypeForm[set[str]] = "set[str]" # OK Expressions that violate one or more of the syntactic, semantic, or contextual rules for type expressions should not evaluate to a ``TypeForm`` type. The rules @@ -286,10 +265,10 @@ Explicit ``TypeForm`` Evaluation Type checkers should validate that this argument is a valid type expression:: x1 = TypeForm(str | None) - reveal_type(v1) # Revealed type is "UnionType & TypeForm[str | None]" + reveal_type(v1) # Revealed type is "TypeForm[str | None]" x2 = TypeForm("list[int]") - revealed_type(v2) # Revealed type is "Literal['list[int]'] & TypeForm[list[int]]" + revealed_type(v2) # Revealed type is "TypeForm[list[int]]" x3 = TypeForm('type(1)') # Error: invalid type expression @@ -327,42 +306,25 @@ assignable to ``TypeForm[A]`` if ``B`` is assignable to ``A``:: ``TypeForm`` is a subtype of ``object`` and is assumed to have all of the attributes and methods of ``object``. -If a value with a ``TypeForm`` type is used within an ``typing.assert_type`` -call, the assertion should succeed for both its actual type and its -``TypeForm`` type:: - - t1 = int | str - assert_type(t1, UnionType) # OK - assert_type(t1, TypeForm[int | str]) # OK - assert_type(t1, type[int] | type[str]) # Error - assert_type(t1, TypeForm[int]) # Error - - t2 = "int | str" - assert_type(t2, str) # OK - assert_type(t2, TypeForm[int | str]) # OK - assert_type(t2, type[int | str]) # Error - Backward Compatibility ====================== This PEP clarifies static type checker behaviors when evaluating type expressions in "value expression" contexts (that is, contexts where type -expressions are not mandated by the typing spec). It augments the *actual -type* of these expressions with a ``TypeForm`` type. This approach retains -backward compatibility because the resulting type is compatible with the old -(non-augmented) type. For example, if a static type checker previously -evaluated the type of expression ``str | None`` as ``UnionType``, it will -now evaluate the type of this expression as ``UnionType`` *and* -``TypeForm[str | None]``. +expressions are not mandated by the typing spec). In the absence of a +``TypeForm`` type annotation, existing type evaluation behaviors persist, +so no backward compatibility issues are anticipated. For example, if a static +type checker previously evaluated the type of expression ``str | None`` as +``UnionType``, it will continue to do so unless this expression is assigned +to a variable or parameter whose type is annotated as ``TypeForm``. How to Teach This ================= -Type expressions are typically used in annotations to describe the set of -values accepted by a function parameter, returned by a function, or stored -in a variable: +Type expressions are used in annotations to describe which values are accepted +by a function parameter, returned by a function, or stored in a variable: .. code-block:: text @@ -382,9 +344,9 @@ assigned to variables and manipulated like any other data in a program: .. code-block:: text - a variable a type expression - | | - v v + a variable a type expression + | | + v v int_type_form: TypeForm = int | None ^ | @@ -400,14 +362,14 @@ a valid type expression, including those with brackets (``list[int]``), union operators (``int | None``), and special forms (``Any``, ``LiteralString``, ``Never``, etc.). -Most programmer will not define their *own* functions that accept a ``TypeForm`` -parameter or returns a ``TypeForm`` value. It is more common to pass a type +Most programmers will not define their *own* functions that accept a ``TypeForm`` +parameter or return a ``TypeForm`` value. It is more common to pass a type form object to a library function that knows how to decode and use such objects. For example, the ``isassignable`` function in the ``trycast`` library can be used like Python's built-in ``isinstance`` function to check whether a value matches the shape of a particular type. ``isassignable`` accepts *any* -type form object an input. +type form object as input. - Yes: @@ -511,7 +473,7 @@ Challenges When Accepting All TypeForms --------------------------------------- A function that takes an *arbitrary* ``TypeForm`` as input must support a -variety of possible type form object. Such functions are not easy to write. +variety of possible type form objects. Such functions are not easy to write. - New special forms are introduced with each new Python version, and special handling may be required for each one. @@ -522,7 +484,7 @@ variety of possible type form object. Such functions are not easy to write. - Recursive types like ``IntTree = list[int | 'IntTree']`` are difficult to resolve. - User-defined generic types (like Django’s ``QuerySet[User]``) can introduce - non-standard behaviors that requite runtime support. + non-standard behaviors that require runtime support. Reference Implementation @@ -586,9 +548,8 @@ assignability rules are based on the assignability rules for types. It is nonsensical to ask whether ``Final[int]`` is assignable to ``int`` because the former is not a valid type expression. -Functions that wish to operate on objects that are evaluated from *any* -annotation expressions can continue to accept such inputs as ``object`` -parameters. +Functions that wish to operate on objects that are evaluated from annotation +expressions can continue to accept such inputs as ``object`` parameters. Pattern matching on type forms From 0db78259a2f53d672f8b079ce133c48035c721b7 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Sun, 15 Sep 2024 08:13:50 -0700 Subject: [PATCH 13/13] Incorporated PR feedback. --- peps/pep-0747.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 65c2d3b6654..dd3a053ae69 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -223,7 +223,13 @@ For example, if a static type checker encounters the expression ``str | None``, it may normally evaluate its type as ``UnionType`` because it produces a runtime value that is an instance of ``types.UnionType``. However, because this expression is a valid type expression, it is also assignable to the -type ``TypeForm[str | None]``. +type ``TypeForm[str | None]``: + + v1_actual: UnionType = str | None # OK + v1_type_form: TypeForm[str | None] = str | None # OK + + v2_actual: type = list[int] # OK + v2_type_form: TypeForm = list[int] # OK The ``Annotated`` special form is allowed in type expressions, so it can also appear in an expression that is assignable to ``TypeForm``. Consistent