-
Notifications
You must be signed in to change notification settings - Fork 15.9k
Open
Labels
affected_version:3.0Issues Reported for 3.0Issues Reported for 3.0affected_version:3.1Issues Reported for 3.1Issues Reported for 3.1area:MetaDBMeta Database related issues.Meta Database related issues.area:corekind:bugThis is a clearly a bugThis is a clearly a bug
Description
Apache Airflow version
3.1.0
If "Other Airflow 2 version" selected, which one?
No response
What happened?
The command is used to clean up the records created for 30 days+ :
airflow db clean --clean-before-timestamp "$(date -d "30 days ago" +%Y-%m-%dT%H:%M)" --skip-archive --yes
The error :
Checking table dag_version
Found 155 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_version__20250929024208
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.ForeignKeyViolation: update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance"
DETAIL: Key (id)=(0198edf9-f9f1-7d73-92c1-fac5359c10b7) is still referenced from table "task_instance".
For the DAG Version ID 0198edf9-f9f1-7d73-92c1-fac5359c10b7, I found :-
- 1 record in
dag_versionwithlast_updated= 2025-08-28 (more than 1 month ago, should be scoped) - some records in
task_instancewithupdated_at= 2025-09-02 (less than 1 month ago, should be not scoped)
Seem I can workaround by retrying it after 2025-10-02 so all mentioned records will be scoped to remove.
What you think should happen instead?
It should skip the problematic records / tables, or aggressively remove the referencing records as well.
How to reproduce
n/a
Operating System
AlmaLinux release 9.6 (Sage Margay)
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
obarisklamiskin
Metadata
Metadata
Assignees
Labels
affected_version:3.0Issues Reported for 3.0Issues Reported for 3.0affected_version:3.1Issues Reported for 3.1Issues Reported for 3.1area:MetaDBMeta Database related issues.Meta Database related issues.area:corekind:bugThis is a clearly a bugThis is a clearly a bug