Skip to content

Conversation

@acrylJonny
Copy link
Collaborator

@acrylJonny acrylJonny commented Nov 13, 2025

  1. Fixing issues caused by text panels not having upstream lineage
  2. Cater for rawSQL, rawSql inconsistencies across different datasources - different datasources return different items from the API.
  3. Emit dataset lineage even when column level lineage cannot be determined.

@github-actions github-actions bot added the ingestion PR or Issue related to the ingestion of metadata label Nov 13, 2025
@acrylJonny acrylJonny linked an issue Nov 13, 2025 that may be closed by this pull request
@codecov
Copy link

codecov bot commented Nov 13, 2025

Codecov Report

❌ Patch coverage is 90.39548% with 17 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...ion/src/datahub/ingestion/source/grafana/models.py 90.16% 12 Missing ⚠️
...rc/datahub/ingestion/source/grafana/field_utils.py 88.57% 4 Missing ⚠️
...rc/datahub/ingestion/source/grafana/grafana_api.py 80.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@datahub-cyborg datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label Nov 13, 2025
Comment on lines +68 to +72
raw_sql = ""
for key, value in target.items():
if key.lower() == "rawsql" and value:
raw_sql = value
break
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about ?

Suggested change
raw_sql = ""
for key, value in target.items():
if key.lower() == "rawsql" and value:
raw_sql = value
break
raw_sql = target.get("rawSql") or target.get("rawSQL") or target.get("rawsql")

any other variant to be considered?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The short answer is that this is dictated by the implementation of the individual connector - so, although we've found Athena to be different to Postgres, there may be more, hence why leaving it as flexible would I think be best.

def _extract_raw_sql_fields_fallback(target: Dict[str, Any]) -> List[SchemaFieldClass]:
"""Fallback basic SQL parsing for when sqlglot fails."""
raw_sql = target.get("rawSql", "").lower()
# Handle case variations: rawSql, rawSQL, etc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as before

@datahub-cyborg datahub-cyborg bot added pending-submitter-response Issue/request has been reviewed but requires a response from the submitter and removed needs-review Label for PRs that need review from a maintainer. labels Nov 24, 2025
for target in query_targets:
if target.get("rawSql"):
return target["rawSql"]
# Handle case variations: rawSql, rawSQL, etc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed pending-submitter-response Issue/request has been reviewed but requires a response from the submitter labels Nov 24, 2025
@acrylJonny acrylJonny merged commit 2b48b22 into master Nov 24, 2025
84 of 85 checks passed
@acrylJonny acrylJonny deleted the grafana-text-panel-fix branch November 24, 2025 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ingestion PR or Issue related to the ingestion of metadata pending-submitter-merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Grafana ingestion fails with text panels

3 participants