Unable to change column comment with Alembic

[reyley]

We're using databricks-sqlalchemy with Alembic, and have found that we are unable to add or change comments on columns. Creating a new column with a comment works fine though.

Let me know if there's any more information I can provide, or if I should open this issue with SQLAlchemy or Alembic.

from alembic import op

op.alter_column(
    'some_table',
    'some_column',
    comment='my column comment',
)

An exception is thrown:

 can't render element of type <class 'alembic.ddl.base.ColumnComment'>: <class 'alembic.ddl.base.ColumnComment'> construct has no default compilation handler.

Click to See full backtrace

Traceback (most recent call last):
  File "<redacted>lib/python3.12/site-packages/sqlalchemy/sql/visitors.py", line 137, in _compiler_dispatch
    meth = getter(visitor)
           ^^^^^^^^^^^^^^^
AttributeError: 'DatabricksDDLCompiler' object has no attribute 'visit_clause'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<redacted>lib/python3.12/site-packages/sqlalchemy/ext/compiler.py", line 523, in _wrap_existing_dispatch
    return existing_dispatch(element, compiler, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>lib/python3.12/site-packages/sqlalchemy/sql/visitors.py", line 139, in _compiler_dispatch
    return visitor.visit_unsupported_compilation(self, err, **kw)  # type: ignore  # noqa: E501
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>lib/python3.12/site-packages/sqlalchemy/sql/compiler.py", line 902, in visit_unsupported_compilation
    raise exc.UnsupportedCompilationError(self, type(element)) from err
sqlalchemy.exc.UnsupportedCompilationError: Compiler <databricks.sqlalchemy._ddl.DatabricksDDLCompiler object at 0x140c5f980> can't render element of type <class 'alembic.ddl.base.ColumnComment'> (Background on this error at: https://sqlalche.me/e/20/l7de)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<redacted>bin/alembic", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "<redacted>/lib/python3.12/site-packages/alembic/config.py", line 988, in main
    CommandLine(prog=prog).main(argv=argv)
  File "<redacted>/lib/python3.12/site-packages/alembic/config.py", line 978, in main
    self.run_cmd(cfg, options)
  File "<redacted>/lib/python3.12/site-packages/alembic/config.py", line 912, in run_cmd
    fn(
  File "<redacted>/lib/python3.12/site-packages/alembic/command.py", line 483, in upgrade
    script.run_env()
  File "<redacted>/lib/python3.12/site-packages/alembic/script/base.py", line 551, in run_env
    util.load_python_file(self.dir, "env.py")
  File "<redacted>/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 114, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 134, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/Users/<redacted>/PycharmProjects/up-ingest-pipeline/databricks/migrations/env.py", line 220, in <module>
    run_migrations_online()
  File "/Users/<redacted>/PycharmProjects/up-ingest-pipeline/databricks/migrations/env.py", line 208, in run_migrations_online
    context.run_migrations(
  File "<string>", line 8, in run_migrations
  File "<redacted>/lib/python3.12/site-packages/alembic/runtime/environment.py", line 946, in run_migrations
    self.get_context().run_migrations(**kw)
  File "<redacted>/lib/python3.12/site-packages/alembic/runtime/migration.py", line 623, in run_migrations
    step.migration_fn(**kw)
  File "<redacted>", line 29, in upgrade
    op.alter_column(
  File "<string>", line 8, in alter_column
  File "<string>", line 3, in alter_column
  File "<redacted>/lib/python3.12/site-packages/alembic/operations/ops.py", line 1973, in alter_column
    return operations.invoke(alt)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/alembic/operations/base.py", line 441, in invoke
    return fn(self, operation)
           ^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/alembic/operations/toimpl.py", line 53, in alter_column
    operations.impl.alter_column(
  File "<redacted>/lib/python3.12/site-packages/alembic/ddl/impl.py", line 343, in alter_column
    self._exec(
  File "<redacted>/lib/python3.12/site-packages/alembic/ddl/impl.py", line 246, in _exec
    return conn.execute(construct, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1416, in execute
    return meth(
           ^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 180, in _execute_on_connection
    return connection._execute_ddl(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1524, in _execute_ddl
    compiled = ddl.compile(
               ^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 309, in compile
    return self._compiler(dialect, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 69, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/sql/compiler.py", line 870, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/sql/compiler.py", line 915, in process
    return obj._compiler_dispatch(self, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/ext/compiler.py", line 538, in <lambda>
    lambda *arg, **kw: existing(*arg, **kw),
                       ^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/ext/compiler.py", line 591, in __call__
    expr = fn(element, compiler, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted>/lib/python3.12/site-packages/sqlalchemy/ext/compiler.py", line 525, in _wrap_existing_dispatch
    raise exc.UnsupportedCompilationError(
sqlalchemy.exc.UnsupportedCompilationError: Compiler <databricks.sqlalchemy._ddl.DatabricksDDLCompiler object at 0x140c5f980> can't render element of type <class 'alembic.ddl.base.ColumnComment'>: <class 'alembic.ddl.base.ColumnComment'> construct has no default compilation handler. (Background on this error at: https://sqlalche.me/e/20/l7de)

NOTE: A very similar issue was opened in sqlalchemy-redshift and I wonder if the solution is similar