Skip to content

ForeignKeyViolation error when doing airflow db clean #56192

@hket-ianchu

Description

@hket-ianchu

Apache Airflow version

3.1.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

The command is used to clean up the records created for 30 days+ :

airflow db clean --clean-before-timestamp "$(date -d "30 days ago" +%Y-%m-%dT%H:%M)" --skip-archive --yes

The error :

Checking table dag_version
Found 155 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_version__20250929024208
    self.dialect.do_execute(
  File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.ForeignKeyViolation: update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance"
DETAIL:  Key (id)=(0198edf9-f9f1-7d73-92c1-fac5359c10b7) is still referenced from table "task_instance".

For the DAG Version ID 0198edf9-f9f1-7d73-92c1-fac5359c10b7, I found :-

  • 1 record in dag_version with last_updated = 2025-08-28 (more than 1 month ago, should be scoped)
  • some records in task_instance with updated_at = 2025-09-02 (less than 1 month ago, should be not scoped)

Seem I can workaround by retrying it after 2025-10-02 so all mentioned records will be scoped to remove.

What you think should happen instead?

It should skip the problematic records / tables, or aggressively remove the referencing records as well.

How to reproduce

n/a

Operating System

AlmaLinux release 9.6 (Sage Margay)

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions