Skip to content

Allow configurable delay between create table and insert for athena.to_iceberg #3224

@AntoniaBlanchflower

Description

@AntoniaBlanchflower

Occasionally we experience a delay in metadata propagation between the create table operation and insert operation. This results in the following exception when executing the INSERT INTO:

File "/opt/python/awswrangler/athena/_executions.py", line 230, in wait_query
raise exceptions.QueryFailed(response["Status"].get("StateChangeReason"))
awswrangler.exceptions.QueryFailed: Table xxxxx not found in database xxxxx

Can we have a configurable delay or retry between the create table operation and INSERT INTO operation e.g. as an environmental variable such as how the wait query polling interval can be configured

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions