Calling `PyDateTime_IMPORT` can lead to an unhandled exception

[ngoldbaum]

The PyDateTime_IMPORT macro doesn't do error checking and isn't documented to return anything, so I think to fix this there would need to be a new function or macro that can fail that people can migrate too.

c.f. python/cpython#122184 (comment)

The PyDateTime_IMPORT macro is defined like this:

#define PyDateTime_IMPORT \
    PyDateTimeAPI = (PyDateTime_CAPI *)PyCapsule_Import(PyDateTime_CAPSULE_NAME, 0)

This is problematic because PyCapsule_Import can fail. And indeed if I create a C extension with the following content:

Details

#include "Python.h"
#include "datetime.h"

PyModuleDef test_module = {
    .m_base=PyModuleDef_HEAD_INIT,
    .m_name="test_datetimebug",
    .m_doc=NULL,
    .m_size=-1,
    .m_methods=NULL,
    .m_slots=NULL,
    .m_traverse=NULL,
    .m_clear=NULL,
    .m_free=NULL
};


PyMODINIT_FUNC
PyInit_test_datetime(void)
{
  PyDateTime_IMPORT;
  return PyModule_Create(&test_module);
}

build it with the following meson.build:

project(
  'test_datetime',
  'c',
)

py_mod = import('python')
py = py_mod.find_installation()

srcs = [
  'test_datetime/test.c'
]

py.extension_module(
  'test_datetime',
  srcs,
  install: true,
  subdir: 'test_datetime',
)

and pyproject.toml:

[build-system]
requires = [
    "meson>=0.63.0",
    "meson-python",
]
build-backend = "mesonpy"

[project]
name = "test_datetime"
requires-python = ">=3.9.0"
version = "0.0.1"%

and then if I put a file named datetime.py somewhere in my import path (e.g. next to the C extension), I get the following error about an unhandled exception:

$ touch datetime.py
$ python
Python 3.13.0 experimental free-threading build (main, Oct 10 2024, 10:28:54) [Clang 16.0.0 (clang-1600.0.26.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import test_datetime
AttributeError: module 'datetime' has no attribute 'datetime_CAPI' (consider renaming '/Users/goldbaum/Documents/test-datetime/build/datetime.py' since it has the same name as the standard library module named 'datetime' and the import system gives it precedence)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import test_datetime
SystemError: initialization of test_datetime raised unreported exception

It looks like the same trick doesn't work on CPython itself if I create a datetime.py file and do import zoneinfo, it's only problematic for extensions.

[serhiy-storchaka]

You should check PyDateTimeAPI after PyDateTime_IMPORT.

https://github.com/python/cpython/blob/ec4021c6d73407fd4d22ee1a4f49d68835ef0770/Modules/_zoneinfo.c#L2712-L2715

    PyDateTime_IMPORT;
    if (PyDateTimeAPI == NULL) {
        goto error;
    }

But there is other issue. If PyDateTime_IMPORT is called multiple times, and the former call succeeds, passing the check, but the latter call fails and sets PyDateTimeAPI to NULL, the code that depends on the passed check can crash. Reproducer:

>>> import sys
>>> import _zoneinfo
>>> sys.modules['_datetime'] = None
>>> del sys.modules['datetime']
>>> del sys.modules['_zoneinfo']
>>> import _zoneinfo as _zoneinfo2
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    import _zoneinfo as _zoneinfo2
AttributeError: module 'datetime' has no attribute 'datetime_CAPI'
>>> _zoneinfo.ZoneInfo('Europe/Paris')