IndexError on type definition

[hoijui]

yapf version: 0.43.0
python version: 3.12

to reproduce:

$ echo "type foo = str" | yapf
Traceback (most recent call last):
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf/yapflib/yapf_api.py", line 198, in FormatCode
    tree = pytree_utils.ParseCodeToTree(unformatted_source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf/pytree/pytree_utils.py", line 113, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf_third_party/_ylib2to3/pgen2/driver.py", line 187, in parse_string
    return self.parse_tokens(tokens, debug)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf_third_party/_ylib2to3/pgen2/driver.py", line 156, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf_third_party/_ylib2to3/pgen2/parse.py", line 230, in addtoken
    return self._addtoken(ilabel, type, value, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf_third_party/_ylib2to3/pgen2/parse.py", line 313, in _addtoken
    raise ParseError('bad input', type, value, context)
yapf_third_party._ylib2to3.pgen2.parse.ParseError: bad input: type=1, value='foo', context=(' ', (1, 5))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf/__init__.py", line 113, in main
    reformatted_source, _ = yapf_api.FormatCode(
                            ^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf/yapflib/yapf_api.py", line 201, in FormatCode
    raise errors.YapfError(errors.FormatErrorMsg(e))
                           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf/yapflib/errors.py", line 37, in FormatErrorMsg
    return '{}:{}:{}: {}'.format(e.args[1][0], e.args[1][1], e.args[1][2], e.msg)
                                 ~~~~~~^^^
IndexError: tuple index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/bin/yapf", line 8, in <module>
    sys.exit(run_main())
             ^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf/__init__.py", line 377, in run_main
    sys.exit(main(sys.argv))
             ^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf/__init__.py", line 121, in main
    raise errors.YapfError(errors.FormatErrorMsg(e))
                           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/krawl-_WstNwwR-py3.12/lib/python3.12/site-packages/yapf/yapflib/errors.py", line 37, in FormatErrorMsg
    return '{}:{}:{}: {}'.format(e.args[1][0], e.args[1][1], e.args[1][2], e.msg)
                                 ~~~~~~^^^
IndexError: tuple index out of range

[jrd]

ylib2to3 needs to catch up with the fork maintained by black : https://github.com/psf/black/commits/main/src/blib2to3

Especially here, the following commits should be merge:

• psf/black@3aad6e3
• psf/black@722735d (fix)

I applied the diff to Grammar.txt and it’s working.

[stav-bitanski]

it happens to me too, @hoijui did you managed to fix it?