Skip to content

Safe DAGs generation and deployment #288

@beczkowb

Description

@beczkowb

There are 2 problems when generating and deploying dags:

  1. Some workflows can be skipped when generating DAGs, due to errors on module import
  2. When we are deploying DAGs, we are not checking if there is an image with the proper version. There are 2 common reasons for DAGs and image mismatch – broken/forgotten image build and broken versioning.

First, we should throw an error when we can't import some module, and secondly, we should add a check in the DAGs deployment process, so it can throw an exception if there is no expected image. I'm not sure if we should make it as default (due to backward compatibility).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions