Skip to content

[Feature] flags.INVOCATION_COMMAND for programmatic dbt invocations #10313

@dsillman2000

Description

@dsillman2000

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When programmatically invoking dbt in a separate python script (i.e. a Pytest function), the CLI args passed to that external python script are in sys.argv, which causes flags.INVOCATION_COMMAND to be equal to dbt {sys.argv}, even when the arguments may have originally been passed to the external Python script.

For instance, when logging the flags.INVOCATION_COMMAND from a model during programmatic invocation from a Pytest function, I see

INVOCATION_COMMAND="dbt tests/my_test.py -sxv"

Expected Behavior

In reality, the programmatic invocation was more like

dbt_runner.invoke(["compile", "--empty", "--select", "my_empty_test_model"], project_dir=..., profiles_dir=...)

I would have expected INVOCATION_COMMAND to be "dbt compile --empty --select my_empty_test_model", but instead it was "dbt tests/my_test.py -sxv" (my arguments passed to pytest).

Steps To Reproduce

  1. Initialize an empty dbt project
  2. Inside one of the default models, add a Jinja block for logging the flags variable:
{% do log("INVOCATION_COMMAND = " ~ flags.INVOCATION_COMMAND, info=true) %}
  1. Create a python script in the root project folder, called main.py
  2. Fill that main.py script with these contents:
from dbt.cli.main import dbtRunner
from pathlib import Path

DBT_PROJECT_DIR = Path(__file__).parent

def cli():
    dbt_runner = dbtRunner()
    dbt_runner.invoke(["compile"], project_dir=DBT_PROJECT_DIR, profiles_dir=DBT_PROJECT_DIR)


if __name__ == "__main__":
    cli()
  1. Call the python script directly, passing any additional arguments you want:
python main.py hello world

The results show that the INVOCATION_COMMAND is not "dbt compile" like we'd expect, but "dbt hello world."

Relevant log output

14:06:00  Running with dbt=1.8.2
14:06:01  Registered adapter: snowflake=1.8.3
14:06:01  Found 2 models, 4 data tests, 444 macros
14:06:01  
14:06:02  Concurrency: 4 threads (target='dev')
14:06:02  
14:06:02  INVOCATION_COMMAND = dbt hello world

Environment

- OS: Mac OSX Ventura 13.6
- Python: 3.11.2 (via pyenv 2.3.15)
- dbt:
    - dbt-core: 1.8.2
    - dbt-snowflake: 1.8.3

Which database adapter are you using with dbt?

snowflake

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp_wantedTrickier changes, with a clear starting point, good for previous/experienced contributors

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions